PyTorch Training Loop

Overview

A correct PyTorch loop has a precise sequence of operations. Getting the order or the modes wrong produces silent bugs (no gradients, dropout active at eval, leaked compute graphs). This skill encodes the canonical, production-ready loop.

When to use

• Writing a training loop from scratch.
• Debugging a model that won't learn or OOMs.
• Reviewing PyTorch training code.

Canonical loop

import torch
from torch.amp import autocast, GradScaler