Vision Agents Skill Reference

Product Summary

Vision Agents is an open-source Python framework for building real-time voice and video AI agents. You write an Agent that joins a call session, connects to AI providers through swappable plugins (LLM, STT, TTS, vision models), and responds in real time. The framework handles call lifecycle, audio/video routing, turn-taking, and deployment. Key files: agent.py (agent definition), .env (API keys), pyproject.toml (dependencies). CLI: uv run agent.py run (console mode), uv run agent.py serve (HTTP server). Plugins: 35+ integrations for LLMs, speech, vision, avatars, and infrastructure. Primary docs: https://visionagents.ai

When to Use

Reach for this skill when:

• Building voice agents: Custom STT/LLM/TTS pipelines or realtime models (OpenAI, Gemini, Qwen)
• Building video agents: VLMs, YOLO processors, or realtime video understanding
• Adding tools and knowledge: Function calling, MCP servers, or RAG (Gemini FileSearch, TurboPuffer)
• Deploying to production: HTTP server, Docker, Kubernetes, horizontal scaling with Redis
• Phone integration: Inbound/outbound calls via Twilio or Telnyx
• Testing agent behavior: Text-only tests without audio/video infrastructure
• Monitoring: OpenTelemetry metrics, Prometheus, Jaeger tracing
• Choosing providers: Swapping LLMs, STT, TTS, or vision models without rewriting logic

Quick Reference