vLLM High-Throughput Inference Engine AI Skill Guide (Claude)

Overview & Engine Architecture

vLLM is an open-source high-throughput LLM serving engine. PagedAttention stores KV cache in non-contiguous blocks (OS-page analogy), cutting fragmentation. Continuous batching schedules at iteration granularity; chunked prefill (default on V1 when possible) interleaves long prefills with decode so TTFT spikes do not starve live streams. Agents size GPU memory utilization vs max model length vs concurrency, expose the OpenAI-compatible HTTP API, and treat preemption warnings as capacity signals—not noise.