evals-init

What this skill does

Initialize the project-level evaluation directory structure following EDD (Eval-Driven Development) principles to prepare for systematic evaluation development. This is completely standalone with zero spec-kit dependencies.

Output:

1. Directory Structure - evals/{system}/ with proper organization (promptfoo | deepeval)
2. Security Baseline - Auto-created graders for PII leakage, prompt injection, hallucination detection, misinformation detection
3. Configuration Files - Standalone config.yml and goldset templates under .adlc/evals/
4. Auto-handoff to /evals-specify to begin error analysis

Key EDD Principles Applied:

• Principle I: Spec-Driven Contracts - Evals validate spec compliance
• Principle II: Binary Pass/Fail - No Likert scales in grader templates
• Principle IV: Evaluation Pyramid - Tier 1 (fast) + Tier 2 (goldset) structure
• Principle IX: Test Data as Code - Version control setup for datasets

When to use