Agent evals

An eval answers one question: did the agent do the job correctly? Everything below is a way to ask that question more reliably, more cheaply, and more often.

Built from the evals masterclass by Alex Lieberman with Viv (LangChain), 2026: youtube.com/watch?v=zLeG-XJtbIE. The structure (easy, hard, god mode) is theirs. Mistakes in this skill are ours.

Start by asking the user three things

1. What real job does the agent do? Name the system it touches (CRM, inbox, repo, database).
2. Who in the company is best at that job today? Their judgment is the ground truth.
3. Does the agent run in production yet? If yes, are traces stored anywhere?

Pick the mode from the answers. Do not skip to god mode without tasks and verifiers.

Easy mode: tasks and verifiers