eval-trace-rca
Eval RCA — Root Cause Analysis from Production Trace Signal
Perform structured root cause analysis on production LLM traces. Supports two modes depending on what signal is available:
| Mode | Signal used | When to use |
|---|---|---|
| Eval Signal | LLM judge verdicts and reasoning (pass/fail rates, scoring) | App has evaluators configured; goal is to understand why evals are failing |
| Error Signal | Runtime errors in traces (@status:error, error types, stack traces) |
No evals configured, or user explicitly wants to analyze crashes/exceptions/tool failures |
If the mode cannot be inferred from context, ask one clarifying question before proceeding: "Would you like me to analyze eval pass/fail patterns, or look at runtime errors and exceptions in traces?"
Methodology
Context → Observe → Open Coding → Axial Coding → Root Cause Analysis → Recommendations
Usage
What's wrong with <ml_app> based on its evals over the last <timeframe>
More from datadog-labs/agent-skills
dd-pup
Datadog CLI (Rust). OAuth2 auth with token refresh.
655dd-apm
APM - install, onboard, instrument, enable, set up, configure, traces, services, dependencies, performance analysis. Use for any request involving Datadog APM setup, instrumentation (SSI, ddtrace, agent install), or analysis.
568dd-logs
Log management - search, archives, metrics, and cost control.
568dd-monitors
Monitor management - list, search, file-based create, and alerting best practices.
550agent-skills
Datadog skills for AI agents. Essential monitoring, logging, tracing and observability.
545dd-docs
Datadog docs lookup using docs.datadoghq.com/llms.txt and linked Markdown pages.
539