Model & System Evaluation

Frameworks for systematic evaluation of ML models, experiment results, and AI system outputs. Covers traditional ML metrics, LLM-as-judge patterns, bias detection, and structured comparison.

When to use

• Comparing multiple trained models beyond val_score
• Evaluating LLM/AI application outputs (RAG quality, prompt effectiveness)
• Building evaluation rubrics for subjective tasks
• Detecting bias in model predictions
• Creating test sets for systematic assessment
• Deciding between experiment results in results.tsv

Traditional ML evaluation

Classification metrics