Deploy vLLM Inference

Use this skill when deploying, configuring, scaling, and benchmarking production LLM serving workloads with the vLLM v1 engine on Kubernetes.

When to Use

• deploying self-hosted open-weight models (GLM-5, DeepSeek-V4, Qwen3.8, Kimi K2/K3, gpt-oss; DeepSeek-R1/Qwen2.5/Llama 3.3 are legacy generations) with the vLLM V1 engine
• configuring PagedAttention v3 with FP8 KV cache memory bounds (<2.4% fragmentation)
• configuring dynamic chunked prefill to eliminate decode latency jitter (TPOT)
• enabling hierarchical Radix Tree prefix caching for multi-turn conversations
• allocating GPU accelerators via Kubernetes 1.31+ Dynamic Resource Allocation (DRA) or NVIDIA MIG hardware slicing
• setting up queue-depth driven Horizontal Pod Autoscaling (HPA) via Prometheus metrics
• configuring high-speed /dev/shm tmpfs mounts and HugePages for RDMA RoCEv2 transfers

Core Rules