eval-harness

Installation
SKILL.md
  • Define test cases with known-correct outputs
  • Run agent against each test case
  • Score: accuracy, completeness, relevance
  • Compare against baseline performance
  • Track performance over time

2. Skill Quality Testing

  • Verify skill instructions produce expected outcomes
  • Test edge cases and boundary conditions
  • Measure consistency across multiple runs
  • Check for harmful or incorrect outputs
  • Validate against ground truth

3. Regression Suite

  • Collection of previously-passing test cases
  • Run after any agent/skill modification
  • Flag regressions with before/after comparison
  • Maintain pass rate threshold (>= 95%)
Installs
1
GitHub Stars
1.8K
First Seen
Sep 21, 2026