Vision Agents Skill

Product Summary

Vision Agents is an open-source Python framework for building real-time voice and video AI agents. Agents join calls via an edge transport (Stream Video RTC, local, or Tencent), connect to swappable AI plugins (LLMs, STT, TTS, vision models), and respond in real time. The framework handles call lifecycle, audio/video routing, turn-taking, and deployment. Key files: agent.py (your agent definition), pyproject.toml (dependencies), .env (API keys). CLI: uv run agent.py run (console mode), uv run agent.py serve (HTTP server). See visionagents.ai for full documentation.

When to Use

Reach for this skill when:

• Building voice agents: Custom STT/LLM/TTS pipelines, realtime models (Gemini, OpenAI, Qwen), phone integration with Twilio/Telnyx
• Building video agents: VLMs, YOLO processors, real-time video analysis, pose detection, object recognition
• Adding tools and knowledge: Function calling, MCP servers, RAG (Gemini FileSearch or TurboPuffer)
• Deploying to production: Docker, Kubernetes, horizontal scaling with Redis, metrics and observability
• Testing agents: Text-only testing without audio/video infrastructure
• Integrating AI providers: Swapping between 35+ plugins (LLMs, STT, TTS, vision, avatars, turn detection)

Quick Reference