CoreWeave Core Workflow: GPU Training

Community-contributed. Not affiliated with, endorsed by, or sponsored by CoreWeave, Inc. CoreWeave is a registered trademark of CoreWeave, Inc.

Overview

Run distributed GPU training on CoreWeave: single-node multi-GPU and multi-node training with PyTorch DDP, Slurm-on-Kubernetes, and shared storage.

Prerequisites

• CKS cluster with multi-GPU node pools (8xA100 or 8xH100)
• Shared storage (CoreWeave PVC or NFS)
• Training container with PyTorch and NCCL

Instructions

Step 1: Single-Node Multi-GPU Training