LLM-as-Judge

Overview

Some quality criteria are inherently subjective — tone of voice, visual aesthetics, UX feel, documentation clarity, code readability. These cannot be verified by deterministic tests. The LLM-as-judge pattern provides structured, repeatable evaluation using an LLM reviewer with defined rubrics, ensuring subjective quality is measured consistently.

Announce at start: "I'm using the llm-as-judge skill to evaluate subjective quality."

Phase 1: Determine Evaluation Method

Goal: Decide whether LLM-as-judge is the right tool.

Decision Table: LLM-as-Judge vs Deterministic Tests