inference_perf_hillclimb

Installation
SKILL.md

Inference Performance Hill-Climbing

Close the gap between the TorchTitan unified model (in vLLM) and vLLM's native model on inference throughput, one profiler-found optimization at a time. Harness: torchtitan/rl/generate.py --benchmark. Append-only running log + full per-rung numbers live in torchtitan/rl/docs/inference_gap_ablation.md. Goal: match vLLM native 100%.

Scope: close the IMPLEMENTATION gap, not the serving knobs

Installs
14
GitHub Stars
5.9K
First Seen
Oct 21, 2026