eval-harness
Eval Harness
Design Philosophy
Evaluation-Driven Development
Evaluation-driven development (EDD) is a methodology where evaluations are defined before or alongside implementation, ensuring that success criteria are explicit, measurable, and testable from the start.
Core Principles:
- Define Success First: Before implementing a feature, define what "working correctly" means through explicit evaluations
- Measure Continuously: Run evals throughout development, not just at the end
- Automate Where Possible: Prefer automated graders for speed and consistency
- Human Review for Nuance: Use human graders when quality judgments require context or subjectivity
- Track Regressions: Every capability added should become a regression test
- Iterate on Failures: Failed evals provide specific, actionable feedback for improvement
Benefits of EDD:
More from mhylle/claude-skills-collection
skill-visualizer
Generate interactive HTML visualizations of the skills collection, codebase structure, or dependency graphs. Uses D3.js for interactive visualization with collapsible nodes, color-coded categories, and hover details. Triggers on "visualize skills", "generate skill map", "codebase visualization", or "show skill dependencies".
11verification-loop
Comprehensive 6-check verification framework for validating implementation quality across build, types, lint, tests, security, and diff review. This skill ensures code meets all quality gates before phase completion. Triggers on "verify implementation", "run verification", "/verification-loop", or automatically as part of implement-phase Step 2.
8context-saver
Save session context to disk for seamless continuation in new chat sessions. This skill should be used when the user asks to save context, preserve work state, checkpoint progress, or prepare for session handoff. Triggers on "save context", "checkpoint", "save progress", "preserve state", or when explicitly asked to create a context file for later resumption. Optimizes for correctness, completeness, minimal size, and trajectory preservation.
8strategic-compact
Strategic compaction suggestion framework that monitors session complexity and suggests context compaction at optimal logical boundaries rather than arbitrary thresholds.
8implement-plan
Orchestrate the execution of complete implementation plans, delegating each phase to implement-phase skill. This skill manages the full plan lifecycle including phase sequencing, user confirmation between phases, and overall progress tracking. Triggers on "implement the plan", "execute the implementation plan", or when given a path to a plan file.
8create-plan
Create detailed implementation plans through interactive research and iteration. This skill should be used when creating new implementation plans, designing feature specifications, planning technical work, or when the user asks to plan an implementation. Triggers on requests like "create a plan", "plan the implementation", "design how to implement", or when given a feature/task that needs structured planning before implementation.
7