Metric Design

Select, configure, and combine evaluation graders into a working pipeline. You choose the right tool for each evaluation dimension — from zero-cost code checks to LLM judges — and produce executable GradingRunner code that runs on OpenJudge.

Requires OpenJudge (pip install py-openjudge). This skill is intentionally SDK-centric — grader selection, GradingRunner, and aggregators are OpenJudge APIs. The design/decision logic still applies if you use another harness; only the code does not.

When to Activate

• User has eval dimensions/principles but doesn't know which grader type to use
• User wants to write an LLM-as-judge prompt for a specific failure mode
• User needs a composite score combining multiple evaluation dimensions
• User wants to auto-generate graders from labeled data instead of writing them manually
• User's current evaluation is all LLM-based and too expensive/too slow

Checklist