Pre-Registering the Analysis

Overview

Write down the hypotheses, the exact analysis, the directional predictions, and the decision rules — and freeze them — before you look at the outcomes. Then run exactly that.

Core principle: If you decided the analysis after seeing the data, you cannot tell whether the data shaped the answer. A prediction is only evidence if it was made before the result.

Violating the letter of this rule is violating the spirit of this rule.

This is the pre-registration discipline — the science analog of test-driven development. In TDD you write the test before the code so you know the test tests something. Here you write the prediction before the result so you know the result confirms something.

The Iron Law

NO CONFIRMATORY CLAIM WITHOUT A PRE-REGISTERED PREDICTION FIRST

Looked at the outcomes before locking the analysis? That analysis is now exploratory. You cannot relabel it confirmatory. To make a confirmatory claim, pre-register and test on data you have not yet used.