Multi-Agent Eval Harness

Overview

Eval that works for a single agent does not catch coordination failures: dropped handoffs, role confusion, blame loops, partial state transfer. This harness adds role-level metrics, coordination-failure scenarios, and full transcript grading.

When to use

• ≥2 agents collaborating (orchestrator + workers, peer agents, A2A)
• Bugs that only appear in production multi-agent runs
• Need to compare orchestration patterns (hierarchical vs peer vs blackboard)
• A2A protocol integration (see a2a-protocol)

What to measure