eval-trace-rca
Installation
SKILL.md
Eval RCA — Root Cause Analysis from Production Trace Signal
Perform structured root cause analysis on production LLM traces. Supports two modes depending on what signal is available:
| Mode | Signal used | When to use |
|---|---|---|
| Eval Signal | LLM judge verdicts and reasoning (pass/fail rates, scoring) | App has evaluators configured; goal is to understand why evals are failing |
| Error Signal | Runtime errors in traces (@status:error, error types, stack traces) |
No evals configured, or user explicitly wants to analyze crashes/exceptions/tool failures |
If the mode cannot be inferred from context, ask one clarifying question before proceeding: "Would you like me to analyze eval pass/fail patterns, or look at runtime errors and exceptions in traces?"
Methodology
Context → Observe → Open Coding → Axial Coding → Root Cause Analysis → Recommendations