agento11y-prod-setup
Installation
SKILL.md
Agent Observability — production evals & guards setup
The production counterpart to agento11y-test-starter (which runs pre-ship, on code alone,
producing an offline test suite). This skill runs after ship, when the agent has real
traffic, and sets up the two production surfaces the starter deliberately leaves out:
- Online eval rules — evaluators that score ingested live conversations, so regressions surface without hand-reviewing every conversation.
- Guards (hook-rules) — policies on the request path that
warn(and can later be promoted todeny) in real time. A guard decides via one of three shapes:evaluator_ids(an evaluator judges),redact(regex redaction), ortool_filter(block tool calls). See Step 4.
What this skill does that agento11y doesn't
The sibling agento11y skill is the mechanics layer: exact CLI flags, evaluator/rule YAML
shapes, create-or-update semantics, the online-eval setup steps. It assumes you already know
what to create.