Data analysis

A number is easy to produce and hard to trust. Most analytical errors are not statistical — they are a filter nobody mentioned, a join that double-counted, a date range that clipped a week, or test accounts left in.

Check the data before you analyse it. The most common way to be confidently wrong is to answer a well-posed question with a badly-built dataset.

1. Pin down the question

Vague questions produce numbers nobody can act on. Establish:

• The metric, defined precisely. "Active users" — active how, over what window, counted per what? Two reasonable definitions give different numbers and both are "right"
• The population: who is included, and who is deliberately excluded
• The period, and what it is being compared against
• The decision this feeds. An analysis with no decision attached usually produces a number nobody uses