AI Eval Regression Ops Review

Use this skill to convert a AI eval regression operations, datasets, rubrics, judge drift, release gates, monitoring, and rollback question into a concrete artifact with owners, gates, metrics, and recovery paths.

Workflow

1. Identify AI job, autonomy level, harm if wrong, datasets, eval owners, model/provider dependencies, release cadence, and user-facing acceptance criteria.
2. Read references/ai-eval-regression-ops-patterns.md.
3. Classify eval types: golden cases, adversarial/safety, rubric judge, human review, offline replay, live canary, cost/latency, and task-success metrics.
4. Define dataset governance, rubric versioning, judge calibration, release thresholds, failure triage, monitoring, rollback, and review cadence.
5. Produce eval ops plan, state machine, decision table, event schema, regression checklist, and model-change policy.

When not to use

• Do not use for generic advice the base model already handles without this skill's specific artifact contract.

Guardrails