Human-in-the-Loop

Human-in-the-loop design defines when, where, and how humans intervene in automated workflows. Too little human involvement and the system makes dangerous mistakes. Too much and you've just built an expensive notification system.

Intervention Types

• Approval gates: The system pauses and waits for human approval before proceeding
• Review checkpoints: The system presents results for human review but can continue if no objection
• Correction opportunities: The system shows its work and the human can edit before it's finalised
• Override controls: The human can stop, redirect, or undo the system's actions at any time
• Monitoring dashboards: The human passively observes the system and intervenes only when needed

When to Require Human Intervention

• High stakes: Actions that are expensive, irreversible, or affect many people
• Low confidence: The system is uncertain about the right action
• Novel situations: The input or context is outside the system's training distribution
• Ethical judgments: Decisions that require moral reasoning or value trade-offs
• Legal requirements: Regulatory or compliance requirements mandate human review
• User request: The user explicitly asks for human involvement

Designing Intervention Points

For each intervention point:

• Trigger: What causes the intervention? (confidence threshold, stakes level, user request, policy requirement)
• Presentation: What does the human see? (summary, full context, recommendations, options)
• Time constraint: How quickly must the human respond? What happens if they don't?