ai-distributed-training
Installation
SKILL.md
Distributed Training - Systems Performance Skill
Operational focus: picking and implementing the right parallelism strategy, not the theory. Covers data parallelism through FSDP/ZeRO/tensor+pipeline parallelism, memory-efficient attention, mixed precision at scale, activation checkpointing, rented-GPU cost discipline, and reproducing GPT-2 124M as the canonical sanity check.
Profile before you scale. Debug on the smallest GPU that fits. Stop the instance when done.
When to Use This Skill
Activate when the user asks about: