rtvi-vlm-customize-model

Installation
SKILL.md

VLM Customization — VSS Alerts Blueprint

When to use

Use this skill when the user wants to:

  • repoint the VSS Alerts Blueprint to a different VLM endpoint,
  • run rtvi-vlm standalone with either an OpenAI-compatible endpoint or the in-container vLLM path,
  • fix the assumption that changing one VLM config automatically updates rtvi-vlm, vlm-as-verifier, and vss-agent.

Do not use this skill for CV detector swaps inside vss-rt-cv; use rtvi-cv-customize-model for those.

Instructions

  • Treat rtvi-vlm, vlm-as-verifier, and vss-agent as three separate VLM consumers. Do not imply that changing only RTVI_VLM_* automatically repoints the verifier or the agent UI.
  • Treat all paths in this skill as relative to a checkout of the public VSS Blueprint repository. Blueprint deployment files live under deploy/docker/...; customer-accessible RT-VLM source and standalone deployment files live under services/rtvi/rt-vlm/.... None of these paths resolve inside the DeepStream repository.
  • For blueprint-side OpenAI-compatible routing, cover all three surfaces: RTVI_VLM_* env vars, ${VSS_PROFILE_DIR}/vlm-as-verifier/configs/config.yml, and the vss-agent VLM_MODEL_TYPE / VLM_NAME / VLM_BASE_URL settings, then mention force-recreate plus log checks.
  • For standalone vllm-compatible, set VLM_MODEL_TO_USE=vllm-compatible, point MODEL_PATH at the weights source, mention HF_TOKEN only when the model actually needs auth. Do not treat a log line as proof of a working deployment: v3.2.1 logs Warmup VlmProcess-0 done even when warm-up raised an exception. Require the absence of an Error during warmup line, readiness, and one successful /v1/chat/completions response.
  • Never paste credentials into chat or logs; capture with a silent prompt (read -rsp), chmod 600 the env file, and never commit generated.env.
Installs
2
GitHub Stars
203
First Seen
2 days ago
rtvi-vlm-customize-model — nvidia/deepstream