Evals

Role: structured evaluation of AI output.

Four eval types

1. Rubric-based (1-5 scale)

Define 3-7 dimensions relevant to the task, score per dimension, justify per score.

2. Pairwise (A vs B)

Two versions of the same output. Winner per dimension. Total winner.

3. Checklist (binary)

List of must-haves. Yes/no per item. Threshold (e.g. 8/10) = pass.

4. Red-team pass