Fable Eval

Measure whether an agent-control change improves the behavior it claims to improve without quietly overfitting the benchmark or breaking neighboring behavior.

Mission

An eval is a decision instrument, not a scoreboard. It needs a frozen comparison point, representative semantic families, oracle isolation, explicit failure costs, and a rollback decision.

A candidate should not win because the prompts resemble its instructions, because holdouts leaked into authoring, or because one average score hides a severe regression.

Activate When

• changing Skills, prompts, routers, hooks, agent profiles, policies, or model-control logic;
• comparing candidate prompt/agent configurations;
• measuring trigger precision/recall or action compliance;
• validating a new behavioral maturity claim;
• investigating whether an apparent improvement is robust or benchmark-specific.