nvalchemi-distillation
Installation
SKILL.md
nvalchemi Distillation
Overview
Distillation trains a small student to reproduce a large frozen
teacher. In nvalchemi it is a TrainingStrategy subclass over two named
models, so everything from nvalchemi-training-api applies: optimizers,
schedulers, validation, hooks, checkpoints, DDP. What distillation adds is a
teacher whose outputs become loss targets.
Read nvalchemi-training-api and nvalchemi-loss-api first.
docs/userguide/distillation.md is the concept guide and
docs/modules/training/distillation.rst is the symbol-by-symbol reference.
import torch