Iron Law

NO EVALUATION CONCLUSION WITHOUT A BASELINE — comparing model A to model B means nothing without a defined baseline and statistical test. Every eval claim requires: metric name, sample size, statistical significance (p < 0.05), and effect size.

LLM Evaluation Skill — Python 3.14 + LangGraph + Google ADK

When to Use This Skill

• Measuring quality of a RAG agent's retrieval or generation output
• Comparing two prompt variants or model versions before promoting to production
• Detecting metric regressions in CI after code or model changes
• Building an automated eval harness for a LangGraph or Google ADK agent
• Selecting the right metric family for a given eval task (retrieval vs generation vs classification)
• Establishing baseline scores before any optimization work
• Setting up inter-rater agreement to validate human evaluation labels

Metric Selection Quick Reference