Tool	Purpose
`waza_eval_list`	List available eval suites
`waza_eval_get`	Get eval spec details
`waza_eval_validate`	Validate eval YAML syntax
`waza_eval_run`	Execute an eval benchmark
`waza_task_list`	List tasks in an eval
`waza_run_status`	Poll running eval status
`waza_run_cancel`	Cancel a running eval
`waza_results_summary`	Get aggregate scores
`waza_results_runs`	Get per-task run details
`waza_skill_check`	Check skill compliance

Scenario 1: Create a New Eval

When user wants to create an eval suite for their skill:

Ask which skill to evaluate — get the skill name and path
Call waza_eval_list to check for existing evals for this skill
If none exist, run waza init <directory> via terminal to scaffold
Explain the generated eval.yaml structure — name, skill, executor, tasks
Help define tasks: ask what behaviors to test, suggest validators (code, regex)
For each task, help write the prompt and expected output
Call waza_eval_validate to confirm the YAML is valid
Suggest running with waza_eval_run to verify the first task passes

Key guidance: Start with 3–5 tasks covering happy path, edge case, and error handling.

Scenario 2: Run and Interpret Results

When user wants to run evals and understand scores:

Call waza_eval_run with the eval spec path and context dir
Poll waza_run_status until complete (check every 10s)
Call waza_results_summary to get aggregate scores
Interpret the results for the user:
- Pass rate — percentage of tasks that passed all validators
- Weighted score — 0.0–1.0 aggregate across all tasks
- Duration — total and per-task execution time
If pass rate < 80%, identify which tasks failed and why
Call waza_results_runs for per-task details on failures
Suggest specific improvements: prompt rewording, validator tuning, fixture updates

Thresholds: ≥90% pass rate = strong, 70–89% = needs work, <70% = significant issues.

Scenario 3: Compare Models

When user wants to compare model performance:

Ask which models to compare (e.g., gpt-4o vs claude-sonnet-4)
Call waza_eval_run with model A — save results
Call waza_eval_run with model B — save results
Compare results side by side:
- Per-task pass/fail differences
- Score deltas (which model scores higher on which tasks)
- Duration differences (speed vs quality tradeoff)
Provide a recommendation: which model is better for this skill and why
Suggest next steps: try a third model, tune prompts for the weaker model, or adjust validators

Guidance: Run each model 2–3 times to account for variance before drawing conclusions.

Scenario 4: Debug a Failing Skill

When user's skill is failing evals or behaving unexpectedly:

Call waza_skill_check to verify skill compliance (frontmatter, triggers, token count)
If compliance issues found, fix those first — they affect routing
Call waza_eval_run with --verbose and --transcript-dir flags
Call waza_results_runs to get per-task failure details
Analyze failure patterns:
- All tasks fail → prompt or fixture issue, check skill instructions
- Some tasks fail → specific edge cases, review failed task prompts
- Validator failures → regex too strict, code validator language mismatch
Suggest targeted fixes based on the pattern
Re-run with waza_eval_run to verify the fix

Scenario 5: Ship Readiness Check

When user asks "is my skill ready?" or wants a pre-ship checklist:

Call waza_skill_check — verify compliance score ≥ medium-high
Call waza_eval_validate — confirm eval YAML is valid
Call waza_eval_run — execute full eval suite
Call waza_results_summary — check aggregate scores
Render the readiness verdict:

SHIP READINESS CHECKLIST:
☐ Skill compliance: [score] (need: medium-high+)
☐ Eval YAML valid: [yes/no]
☐ Pass rate: [X]% (need: ≥90%)
☐ Weighted score: [X.XX] (need: ≥0.85)
☐ No task timeouts
☐ Consistent across 2+ runs

VERDICT: [READY / NOT READY — fix items marked ✗]

If NOT READY, route to the appropriate scenario (Scenario 4 for failures, Scenario 1 for missing evals)

Conversation Style

Always explain why before what — context before commands
After every tool call, interpret the result in plain language
When something fails, diagnose before suggesting fixes
Offer the next logical step — don't wait to be asked
Use the checklist format for multi-step validations

waza-interactive

Waza Interactive

Available MCP Tools