eval-guide
Eval Guide
Iron Law
NEVER generate eval tool code from memory. ALWAYS query Context7 MCP for the official API before writing any eval code. Every metric class name, constructor signature, and YAML provider ID must be verified against current official docs — these APIs change between minor versions.
Prefer LangChain-free eval paths. Ragas, Giskard, and Promptfoo all have LangChain-free paths — prefer them to reduce dependency surface and avoid version conflicts.
Consistent LLM judge. Best practice: use the same LLM provider as your main stack for LLM-as-judge to reduce vendor sprawl. For Gemini-based stacks: GeminiModel("gemini-2.5-flash") for DeepEval, Generator(model="google/gemini-3.1-flash") for Giskard, google:gemini-2.5-pro for Promptfoo.
Dispatch eval-reviewer agent after writing any eval code — same mandate as dispatching adk-reviewer after ADK agent code.