Set Up Online Evaluations and Guardrails

Online evaluations apply reusable evaluators to production traffic:

• An online evaluation measures live traces or threads asynchronously.
• A guardrail runs synchronously and can stop or replace unsafe traffic.

Hand Off Batch Testing Requests

If the user wants to test a dataset, compare prompts or models, benchmark, or create a CI quality gate, this is the wrong workflow.

1. If the experiments skill is available, load it and follow it now.
2. Otherwise, tell the user to install it with:

   npx skills@1.5.19 add langwatch/skills/experiments
   

Do not create a batch experiment from this skill.