The Agent Skills Directory

[EXTERNAL_DOWNLOADS]: The skill fetches resources from well-known technology organizations and platforms.
Downloads the torchtitan repository and its requirements from PyTorch's official GitHub organization.
Installs the torchao library directly from the official PyTorch GitHub repository.
Fetches model assets and tokenizers from the well-known HuggingFace service.
[COMMAND_EXECUTION]: The instructions utilize standard machine learning orchestration tools.
Executes pip for dependency management.
Uses torchrun and srun (SLURM) to launch distributed training jobs across multiple GPUs and nodes.
Runs Python scripts for asset downloading and checkpoint conversion.
[DYNAMIC_EXECUTION]: The skill utilizes runtime optimization and code generation patterns typical for machine learning workflows.
Employs torch.compile to generate optimized kernels at runtime for faster training.
Provides structural templates for defining custom model architectures and parallelism strategies, which are intended to be implemented and executed as Python scripts.
[INDIRECT_PROMPT_INJECTION]: The skill possesses an attack surface for indirect injection via the ingestion of large-scale training datasets (e.g., C4).
Ingestion points: Training datasets are loaded during the pretraining phase (SKILL.md).
Boundary markers: None explicitly defined for dataset content in these instructions.
Capability inventory: The training process involves significant compute resources, file-system access for checkpoints, and standard network calls for logging/monitoring.
Sanitization: Standard machine learning data loaders generally treat dataset content as raw tokens rather than executable instructions, significantly mitigating the risk of accidental obedience.

distributed-llm-pretraining-torchtitan