PyTorch Debugging Guide

This guide provides systematic approaches to debugging PyTorch models, from common tensor errors to complex training issues.

Common Error Patterns

1. CUDA Out of Memory (OOM)

Error Message:

RuntimeError: CUDA out of memory. Tried to allocate X.XX GiB

Causes:

• Batch size too large for GPU memory
• Accumulating gradients without clearing
• Storing tensors on GPU unnecessarily
• Memory leaks from not detaching tensors