rtvi-vlm-customize-model
Installation
SKILL.md
VLM Customization — VSS Alerts Blueprint
When to use
Use this skill when the user wants to:
- repoint the VSS Alerts Blueprint to a different VLM endpoint,
- run
rtvi-vlmstandalone with either an OpenAI-compatible endpoint or the in-container vLLM path, - fix the assumption that changing one VLM config automatically updates
rtvi-vlm,vlm-as-verifier, andvss-agent.
Do not use this skill for CV detector swaps inside vss-rt-cv; use rtvi-cv-customize-model for those.
Instructions
- Treat
rtvi-vlm,vlm-as-verifier, andvss-agentas three separate VLM consumers. Do not imply that changing onlyRTVI_VLM_*automatically repoints the verifier or the agent UI. - Treat all paths in this skill as relative to a checkout of the public VSS Blueprint repository. Blueprint deployment files live under
deploy/docker/...; customer-accessible RT-VLM source and standalone deployment files live underservices/rtvi/rt-vlm/.... None of these paths resolve inside the DeepStream repository. - For blueprint-side OpenAI-compatible routing, cover all three surfaces:
RTVI_VLM_*env vars,${VSS_PROFILE_DIR}/vlm-as-verifier/configs/config.yml, and thevss-agentVLM_MODEL_TYPE/VLM_NAME/VLM_BASE_URLsettings, then mention force-recreate plus log checks. - For standalone
vllm-compatible, setVLM_MODEL_TO_USE=vllm-compatible, pointMODEL_PATHat the weights source, mentionHF_TOKENonly when the model actually needs auth. Do not treat a log line as proof of a working deployment: v3.2.1 logsWarmup VlmProcess-0 doneeven when warm-up raised an exception. Require the absence of anError during warmupline, readiness, and one successful/v1/chat/completionsresponse. - Never paste credentials into chat or logs; capture with a silent prompt (
read -rsp),chmod 600the env file, and never commitgenerated.env.