AI Observability

AI Observability is Grafana's AI observability platform. It records what LLM-powered applications do in production and scores the quality of their output.

Applications send generations (individual LLM API calls — request, response, model, tokens, tool calls) to AI Observability. Generations belonging to the same user session are grouped into a conversation.

Evaluators are scoring functions (LLM judge, regex, heuristic, JSON schema, etc.) that assess generation quality. Rules bind evaluators to production traffic, they select which generations to evaluate (e.g. only user-visible turns), filter by agent/model, and control sampling rate. When a rule matches a generation, AI Observability runs the bound evaluators and writes scores.

All commands live under gcx aio11y. Use gcx aio11y <subcommand> --help for flags and usage.

Command Groups