evaluating-systems

Installation
SKILL.md

Evaluating systems

Evaluate a system by what it produces. A system can operate exactly as designed and still fail its intended outcome.

State what should happen

Express the intervention as a causal hypothesis:

Changing X should improve Y under Z conditions because of M.

Establish what happens before the intervention. Choose an observation window long enough for its proposed mechanism to produce evidence.

Separate operation from outcome

Verification determines whether the system behaves as designed. Validation determines whether that behavior improves the intended outcome.

An apparent improvement can still fail when its cost is unacceptable or it cannot be sustained. Treat those as distinct claims rather than folding them into a single success metric.

Observe at the proof seam

Installs
6
GitHub Stars
3
First Seen
Aug 16, 2026