GenAI / GPU / ML Workloads on Amazon ECS

End-to-end opinionated guidance for running GPU-accelerated, ML-training, and GenAI inference workloads on Amazon ECS-on-EC2. This skill is scoped to the compute-and-capacity mechanics that are unique to ECS: the GPU-optimized AMI + NVIDIA container runtime, the one-homogeneous-ASG-per-GPU-type capacity-provider pattern — cluster auto scaling supports multiple instance types in one ASG, but managed scaling has no instance weighting and bin-packs and protects on the smallest instance type, so mixing GPU types (with different GPU counts / VRAM) breaks the scaling math; one homogeneous ASG per GPU type is therefore the best practice, not a hard limit — plus EC2 Capacity Blocks for ML, AWS Neuron (Inferentia/Trainium) on ECS, container inference/serving, distributed ML, and accelerator observability.

The single most important constraint, stated first: AWS Fargate has no GPU support. GPUs and AWS accelerators (Inferentia/Trainium) are available only on ECS-on-EC2, ECS Managed Instances, and ECS Anywhere/External — never on Fargate. Every GPU/ML answer on ECS begins by ruling Fargate out for the accelerated container. See service-boundaries.md for the exact evidence and the "use EKS / SageMaker / Bedrock instead" routing.

For "which ECS launch model should I use" with no accelerator or ML workload, use ecs-architect. For Kubernetes-based GenAI, use eks-genai. This skill is the GPU/ML workload layer on ECS specifically.

When to Use This Skill

Activate when the user wants to:

• Run a GPU workload on ECS — choose an instance family (g4dn/g5/g6/g6e/p3/p4d/p5), the ECS GPU-optimized AMI, and the NVIDIA container runtime
• Serve an LLM / model inference container on ECS, or run ML training/fine-tuning on ECS
• Design GPU capacity on ECS at scale — the separate-ASG-per-GPU-type + capacity-provider-strategy pattern, Managed Instances, Spot, and Capacity Blocks for ML
• Use AWS Neuron (Inferentia/Trainium) on ECS — device allocation, compilation, Inf/Trn instance selection
• Wire GPU/accelerator observability on ECS — agentless DCGM metrics via Container Insights enhanced observability are Managed-Instances-only; the EC2 launch type needs the CloudWatch agent (nvidia_smi, host-level) or a DCGM exporter for per-task metrics
• Decide when NOT to use ECS — when EKS (eks-genai), SageMaker, or Bedrock is the better home