Measurement Gate — Reliability Is Not Validity

Skill type: DISCIPLINE-ENFORCING. Before any hypothesis test consumes a scale, this gate checks that the scale measures what it claims, reliably, and comparably across groups. It does not fit factor models itself — it requires the right evidence and routes execution to the psychometrics skill.

The Core Rule

A RELIABLE MEASURE OF THE WRONG CONSTRUCT IS STILL WRONG. RELIABILITY ≠ VALIDITY.

High internal consistency only means the items move together; it says nothing about whether they capture the intended construct. Evidence for validity (content, criterion, construct) is separate and mandatory. And the usual reliability number — Cronbach alpha — rests on an assumption (tau-equivalence: equal item loadings) that real scales rarely meet, so alpha is a lower bound that can under- or (with correlated errors) over-state reliability.