Expectation-Driven Science

This is the discipline that turns data exploration from an implicit process into a verifiable hypothesis → prediction → verification loop. It is the data-science equivalent of TDD's red→green: the same root principle — put the expectation before the trust — applied to three distinct objects (hypothesis in the business, premise in the data, function in the code). This skill is the reference that makes that loop produce analyses worth keeping: what a good expectation is, where it lives, the anti-patterns, and the rules of the loop.

When exploring the domain, read the project's context documentation (if it exists) so hypothesis names, metrics, and interfaces match the business domain language, and respect the ADRs in the area you're touching.

What a good expectation (hypothesis/prediction) is

An expectation declares what the observer believes to be true about the world before looking at the data, and can therefore be confirmed or refuted. It reads like a research question: "customers with abandoned carts have lower long-term savings" tells you exactly which relationship you're testing — and it survives model swaps, column changes, or tool changes, because it depends on the relationship of interest, not on the internal mechanism of implementation.

A good expectation must be falsifiable (it can be proven wrong) and quantifiable (it has a threshold/acceptance limit defined before running). When the expectation fails, treat the deviation as information, not failure. See the Rule of the loop below.

Seams: where expectations live

A seam is the object on which you declare an expectation — the point where you observe reality without "reaching inside" it. Expectations live at seams, never against internal details.

Only declare expectations at pre-agreed seams. Before writing any hypothesis, write down the seams under test and confirm them with the user. No expectation is written at an unconfirmed seam. You can't test everything, so agreeing the seams up front is how verification effort lands on the critical hypotheses and complex logic instead of every edge case.

There are three seam families, aligned with the three layers on which a data scientist relies: