evals-validate

What this skill does

Conducts comprehensive validation of the implemented evaluation system following EDD principles to ensure production readiness through statistical analysis, performance verification, and quality assurance.

Output:

1. Statistical Validation - TPR/TNR analysis, accuracy metrics, confidence intervals
2. Performance Validation - SLA compliance verification for evaluation pyramid tiers
3. Quality Assurance - Goldset integrity, example balance, coverage analysis
4. Holdout Dataset Validation - Unbiased accuracy assessment on reserved test set
5. Auto-handoff to /evals-analyze for closed loop trajectory analysis

Key EDD Principles Applied:

• Principle IV: Evaluation Pyramid - Tier performance SLA validation (Tier 1 <30s, Tier 2 <5min)
• Principle II: Binary Pass/Fail - Statistical compliance verification
• Principle IX: Test Data as Code - Holdout dataset validation integrity
• Principle III: Error Analysis - Pattern stability validation