evals-analyze

What this skill does

Provides cross-functional team elevation and closed-loop feedback following EDD Principle VIII (Close the Production Loop) by deep-analyzing trajectory failure traces and routing them to correct resolution pathways.

Output:

1. Trajectory Analysis - Full multi-turn trace analysis with tool calls and context preservation (EDD Principle V)
2. Failure Routing:
   • Specification Failures (agent logic missing/ambiguous) → Automatically triggers a local call to levelup-specify to propose new context rules in .adlc/drafts/cdr/ to fix agent behavior.
   • Generalization Failures (grader flawed or lacks edge-case coverage) → Appends evaluator backlog items to the project backlog for ongoing monitoring.
3. Cross-Functional PR - Creates a team-ai-directives PR with insights and rule updates (EDD Principle X)

Key EDD Principles Applied:

• Principle VIII: Close Production Loop - Spec failures → fix directives; Gen failures → evaluator backlog
• Principle V: Trajectory Observability - Full multi-turn traces, not just outputs
• Principle X: Cross-Functional Observability - PMs, domain experts, and AI engineers collaborate

When to use