ai-distributed-training

Installation
SKILL.md

Distributed Training - Systems Performance Skill

Operational focus: picking and implementing the right parallelism strategy, not the theory. Covers data parallelism through FSDP/ZeRO/tensor+pipeline parallelism, memory-efficient attention, mixed precision at scale, activation checkpointing, rented-GPU cost discipline, and reproducing GPT-2 124M as the canonical sanity check.

Profile before you scale. Debug on the smallest GPU that fits. Stop the instance when done.

When to Use This Skill

Activate when the user asks about:

Installs
18
GitHub Stars
102
First Seen
Aug 12, 2026