nemo-mbridge-perf-cpu-offloading
Originally fromnvidia/skills
Installation
SKILL.md
CPU Offloading
References
- Stable docs: @docs/training/cpu-offloading.md
- Structured metadata: @skills/nemo-mbridge-perf-cpu-offloading/card.yaml
What It Is
Two independent mechanisms to move data from GPU to CPU memory:
| Mechanism | Config namespace | What gets offloaded | PP restriction |
|---|---|---|---|
| Activation offloading | model.cpu_offloading* |
Activations (and optionally weights) per transformer layer | PP must be 1 |
| Optimizer offloading | optimizer.optimizer_cpu_offload |
Adam optimizer states (momentum + variance) via HybridDeviceOptimizer |
None |