The Agent Skills Directory

[EXTERNAL_DOWNLOADS]: The skill instructs the user to download the TorchTitan repository from the official PyTorch GitHub organization. This is a standard procedure for using this specific training framework.
[EXTERNAL_DOWNLOADS]: The skill facilitates the download of model assets, such as tokenizers, from the official Meta Llama repositories on Hugging Face using the download_hf_assets.py script provided by the framework.
[REMOTE_CODE_EXECUTION]: The skill includes instructions to install the torchao library directly from the official PyTorch GitHub repository using pip install git+https://github.com/pytorch/ao.git to enable Float8 training features.
[COMMAND_EXECUTION]: The skill uses torchrun and SLURM (srun, sbatch) for managing distributed training processes across multiple GPUs and nodes. These are standard tools in high-performance computing and machine learning environments.
[CREDENTIALS_UNSAFE]: The skill references the use of Hugging Face tokens (hf_token) for accessing model assets. It correctly utilizes placeholders (e.g., YOUR_HF_TOKEN) and directs users to the official Hugging Face settings page to generate their own credentials, adhering to safe secret management practices.

distributed-llm-pretraining-torchtitan