Silent Failure Detection

Detect when LLM agents fail silently - appearing to work while producing incorrect results.

Overview

• Detecting when agents skip expected tool calls
• Identifying gibberish or degraded output quality
• Monitoring for infinite loops and token consumption spikes
• Setting up statistical baselines for anomaly detection
• Alerting on non-error failures (service up but logic broken)

Quick Reference

Tool Skipping Detection

from langfuse import Langfuse