Run Experiments for Your Agent

Experiments are pre-deployment batch tests. They run an application over a dataset and compare outputs with reusable evaluators. They are appropriate for prompt and model comparisons, regression tests, benchmarks, and CI quality gates.

Hand Off Production Evaluation Requests

If the user wants to score live traces or threads, monitor production quality, or block unsafe traffic, this is the wrong workflow.

1. If the online-evaluations skill is available, load it and follow it now.

2. Otherwise, tell the user to install it with:

   npx skills@1.5.19 add langwatch/skills/online-evaluations
   

Do not configure a monitor or guardrail from this skill.

Experiments and Scenarios