AI Scaling Laws — Compute-Optimal Sizing Skill

Functional reference for pre-training researchers and engineers who need to reason cold about compute, token, and parameter tradeoffs. Covers Kaplan et al. (2020), Chinchilla / Hoffmann et al. (2022), GPT-3 sizing, over-training for inference efficiency, and the mechanics of budget allocation for a from-scratch run.

This is a standard interview probe. Know the key ratios and be ready to work through a concrete sizing calculation without a lookup.