Scaling Law Tracker

Measure whether more scale is buying the right thing — and quantify it, rather than eyeballing a log-log plot.

Workflow

1. Define the scaling axis and metric. One axis at a time: data size, parameters, compute (FLOPs), or batch size. State whether the metric improves up (accuracy) or down (loss).
2. Collect comparable runs. Same architecture family, same data distribution, same eval. Incomparable setups make the exponent meaningless.
3. Fit and extrapolate with the bundled script (below) rather than fitting by hand — log-log regression and the offset form are easy to get subtly wrong.
4. Read off the exponent, the knee, and the extrapolation. Is the curve still paying off, or past the point where this axis is worth scaling?
5. Recommend the next move. Scale this axis further, reallocate to a different axis (data ↔ model ↔ optimization), or stop.

Fit it with the bundled script

scripts/fit_scaling_law.py fits both a power law (y = a·x^b) and a power-law-with-floor (y = E + a·x^b, the Kaplan/Chinchilla form for losses that plateau), reports the exponent and R², extrapolates to a target budget, and flags where doubling the resource buys less than a useful relative gain.