understand-image-video

Two solutions:

1. agnes (recommended) — Agnes 2.5 Flash cloud vision LLM. No local model, no max-token truncation: 512K context, up to 65.5K output tokens. Videos supported via automatic ffmpeg frame sampling. Needs network + API key.
2. mlx-vlm (local fallback) — offline, private, but limited by --max-tokens (default ~2K, truncates long descriptions / OCR / JSON dumps).

prefer: agnes service solution

Prerequisites

• API key resolution (checked in order):
  1. AGNES_API_KEY environment variable
  2. ~/.pi/agent/auth.json → "agnes": { "type": "api_key", "key": "..." }
  3. ~/.pi/agent/models.json → "providers" → "agnes" → "apiKey"
• ffmpeg/ffprobe required for video inputs (frames are extracted and sent chronologically).
• Script is Python stdlib, run via uvx python3 (no global install).

basic