Quantitative analysis

A wrong answer here is rarely an arithmetic error. It is a correct calculation on the wrong comparison, or on data that does not mean what the field name suggests.

Frame the question so that an answer changes something

Start from the decision. "How is retention doing" has no answer; "is the cohort we changed onboarding for retaining better than the one before it, enough to justify rolling it out" does.

Write down what you expect to find and what you would do in each case before you look. If every possible result leads to the same action, the analysis is not worth running — and knowing that in advance is worth more than the analysis would have been.

Choose the comparison before the metric

Almost every meaningful number is a comparison, and the choice of what to compare against does more work than the calculation.