modelopt-speculative-decoding

Installation
SKILL.md

Speculative Decoding Draft-Model Training

Everything needed to take a target model from "no draft head" to "validated acceptance rate" lives in this directory. Work through the stages below in order for a new model; jump straight to a stage when you already know which one you need.

Two axes: stage and algorithm

The pipeline is the same shape for every draft-model algorithm — synthesize data, dump base-model hidden states, train the draft, benchmark acceptance rate. What changes between algorithms is which training script and recipe run, which knobs matter, and which failures are typical.

So this skill is split along those two axes, and you almost always read one file from each:

Installs
1
GitHub Stars
5.6K
First Seen
1 day ago