PyTorch Engineering

Use this contract for the concrete task. Apply a short relevant check for a small change; expand investigation when the failure, risk or requested artifact warrants it. Resolve facts from the repository and runtime before imposing a process. Delegation is optional and should answer a bounded unresolved question.

Implementation contract

Inspect the failing code, dependency/CUDA versions and actual error/trace. For routine syntax or a local change, use only the relevant check. API and backend behavior must be verified against the installed version and upstream documentation when uncertain.

• Track tensor shape, stride/layout, dtype, device, lifetime and autograd ownership at the failing boundary. Retained graphs and hooks can keep memory alive after variables are gone.
• Register parameters/buffers and update optimizer membership deliberately. A tensor stored on a module is not automatically a parameter or portable state.
• For AMP, distinguish FP16/BF16/other toolchains; unscale before clipping and keep accumulation/step/scaler order consistent. torch.amp is not an FP8 autocast recipe.
• Isolate eager behavior from torch.compile graph breaks, guards and recompilation before attributing numerical/performance changes to the optimizer.
• Profile a representative warmed workload with appropriate GPU synchronization and memory evidence. Separate host, compute, memory and communication bottlenecks.
• Check distributed collective ordering, rank agreement, sharding/state-dict semantics and failure cleanup. Select DDP/FSDP/DTensor APIs supported by the target version.
• Checkpoint all state required by the claimed resume contract: model, optimizer, scheduler, scaler, RNG and relevant sampler/data position. Use safe-loading-compatible serialization; do not bypass safety merely to make an example load.
• Verify custom gradients with appropriate numerical/analytic checks and assess transforms/compile interoperability separately.

Evidence and output