vLLM Server Management

Deploy production-grade LLM inference servers with vLLM — the fastest open-source LLM serving engine with PagedAttention and continuous batching.

When to Use This Skill

Use this skill when:

• Serving open-source LLMs (Llama, Mistral, Qwen, Gemma) at scale
• Building an OpenAI-compatible API endpoint for self-hosted models
• Optimizing LLM throughput and latency for production traffic
• Running multi-GPU inference with tensor or pipeline parallelism
• Deploying quantized models to reduce GPU memory requirements

Version and exposure boundary

Pin a reviewed vLLM release, model revision and container digest. Validate the exact release’s Python/CUDA/hardware requirements and flags; the versions, memory figures and model/quantization examples below are historical sketches, not fit guarantees. Model weights, activations and KV cache must fit together. Keep examples loopback-only; remote service needs approved authentication, TLS, quotas and network policy. Never use the sample key as a real secret.

Prerequisites