usability-testing
Usability Testing
Concept of the skill
Usability testing is the evaluative research practice of watching representative real people attempt realistic tasks on a prototype or product to surface the obstacles they encounter — making it empirical observation rather than expert opinion. Its core instrument is the think-aloud protocol (Ericsson & Simon), where participants narrate their reasoning so the divergence between their mental model and the design becomes visible at the moment it happens. Sessions are built around task scenarios that state a goal without prescribing steps ("find out how much you owe in taxes this quarter"), run by a moderator whose discipline is neutrality: frame the goal, prompt only with open questions, let silence and struggle sit, and never rescue or defend the design.
The craft has four hard gates. First, the artifact must exist at a fidelity where a real person can attempt the task. Second, the participant must match the study's target audience closely enough that the result transfers. Third, the task must be goal-framed rather than path-framed: "find out how much you owe this quarter," not "click Tax Summary, then View Details." Fourth, the claim type must match the study design: small formative rounds are excellent for discovering and diagnosing problems; summative benchmark claims require larger samples, metrics, and statistical justification.
Sample size is governed by purpose — the Nielsen/Landauer 5-user rule (~85% of major problems per homogeneous segment, per discrete task) holds for formative, diagnostic, iterative testing, but it is a long-run average with real per-round variance (Faulkner 2003), and summative benchmark claims require much larger statistical samples; conflating the two is a classic, invalidating error. Findings are triaged by Nielsen's 0–4 severity scale — itself a composite of frequency, impact, and persistence — complemented when needed by quantitative instruments (task success reported with a small-sample confidence interval, SEQ, SUS or UMUX-Lite). Current tooling — remote moderated sessions, unmoderated platforms, automatic transcripts, AI-assisted highlights, and LLM-agent simulations — has changed the logistics, not the epistemology: it can reduce operational cost, but it does not replace observing real humans. (For the standard practitioner reference behind these mechanics, see Rubin & Chisnell, Handbook of Usability Testing, 2008.)
Coverage
Usability testing covers session planning, participant selection, task design, moderation or unmoderated setup, observation, severity triage, and claim-appropriate reporting.