build-a-scorer

Help the user get from zero to shipped evaluation prototype. Not 100% correctness on the first pass — a stable, understandable scorer suite they can run, inspect, and iterate on.

Your primary job is understanding, and the user is the expert on what counts as wrong. Do not start from "which MLflow scorer should I use?"

North star:

Understand the app well enough to define a small set of atomic, durable quality criteria, then implement each criterion with the cheapest reliable scorer.

Doctrine