Langfuse LLM Quality Audit

Degree of freedom: MIXED — Phases 0–1, 4 [HIGH freedom]; Phases 2–3 CLI traces and playwright [LOW freedom — run exactly]. Read protocol-browser-anti-stall before any browser step. Never skip Phase 0.

How to reason

1. Observe — quote the trace, prompt version, token/cost, or live output
2. Interpret — is quality, cost, or the pipeline actually broken?
3. Classify — missing-trace / prompt / cost / eval / grounding / correct
4. Severity — pipeline break or untraced prod feature = P0; cost/eval gap = P1

Worked example