human-in-the-loop-review
Installation
SKILL.md
Human-in-the-Loop Review
Human-in-the-loop (HITL) review is the process of having a human check, edit, or approve AI-generated output before it reaches a customer or enters a system of record. Every production AI agent that produces customer-facing output needs HITL. The question is not whether to include human review. It's how much, at what stage, and how to reduce it over time without reducing quality.
The principle: start at 100% human review. Earn the right to reduce it through demonstrated quality. 100% → 50% → 25% → 10% spot-check. Never 0%. Even the most reliable agent produces occasional bad output. The cost of one bad email to a Tier 1 ABM account exceeds the cost of reviewing 1,000 good emails.
The HITL Spectrum
| Level | What the human does | When to use | Review rate |
|---|---|---|---|
| Full review | Human reads every output. Approves, edits, or rejects before it goes out | First 2-4 weeks of any new agent or prompt | 100% |
| Selective review | Human reviews flagged outputs (low confidence, edge cases) + random sample | After 2-4 weeks of stable quality. Automated checks pass consistently | 25-50% |
| Spot-check | Human reviews a random 10-20% sample. The rest goes out automatically | After 30+ days of stable quality with < 2% error rate | 10-20% |
| Exception-only | Human reviews only outputs that fail automated checks | Mature agents with 60+ days of production data and < 1% error rate | 1-5% |