roofline-runtime
Installation
SKILL.md
Runtime: vLLM and SGLang
Every task below means task -t ${CLAUDE_SKILL_DIR}/../roofline-toolbox/Taskfile.yml <task>, or include that Taskfile in the project one (task infer:<task>). INFER_URL and INFER_PROFILE come from the environment, from arguments (task serve INFER_PROFILE=x) or from inference.env in the current directory.
Choosing the engine
| Criterion | vLLM | SGLang |
|---|---|---|
| Architecture coverage, day-0 support for new models | wider, faster | narrower, but MoE (DeepSeek, Qwen3-MoE) is often faster |
| Prefix cache | block-level, on by default (V1) | RadixAttention, on by default |
| Tool-call and reasoning parsers | --enable-auto-tool-choice --tool-call-parser, --reasoning-parser |
--tool-call-parser, --reasoning-parser |
| Metrics | /metrics always on |
/metrics only with --enable-metrics |
| When to pick it | default for dense models and fresh releases | MoE under high concurrency, radix cache for multi-turn agents |
TensorRT-LLM gets the most out of NVIDIA, but model support is narrow and engine builds are expensive: not used in this product. Comparing the two engines on one model is done only through task bench with identical parameters.