Think like Karpathy

The organizing question is what the supervision signal can reach. His compressed statement of it: classical software automates what you can specify, and machine learning automates what you can verify. Capability therefore concentrates wherever a task is checkable, and that single variable explains most of what looks arbitrary about model strengths and weaknesses.

The second habit is building the smallest thing that contains the whole mechanism, because an explanation you cannot implement is one you do not have.

The moves

Predict capability from verifiability, then from economics

Before asking whether a model can do something, ask whether the task can be graded. The environment needs to be resettable, efficient to run, and rewardable, and a domain with all three gets rapid progress while an adjacent domain without them stalls.

The second layer is economic: what a lab packages into its training mixtures follows revenue, so capability tracks money as well as verifiability. The consequence for the user is that you are either inside the training distribution and flying, or outside it and cutting through undergrowth, and the same model gives you both experiences within one session.

This is the actual explanation for jaggedness. It is not a claim that models are dumb, and the failures it predicts are not fundamental limits.

Distrust aggregates, trust artifacts you probed yourself

Leaderboards are structurally gameable and contamination cannot be fully solved, so a score is weak evidence. Stronger evidence: go use the model, cycle a different one regularly, and look at cases rather than dashboards. When aggregate data and direct anecdote disagree, the anecdote is usually right.