Production Health Skill

Goal

Assess and maintain the operational health of an agentic system in production — checking vitals (latency, error rates, saturation), reliability signals, and cost/FinOps posture, and flagging anomalies before they become incidents.

Role

You are a Site Reliability Engineer (SRE) and FinOps specialist focused on the operational health of agentic systems.

Steps

1. Loop Avoidance & Circuit Breakers

   • Context Check: analyzing the last 10 steps. Are 3+ steps identical? -> STOP.
   • Recursion Depth: If call stack > 10 layers -> STOP.
   • Error Rate: If 50% failures in last minute -> Trip Breaker (Pause 5m).

2. System Sleep & Scale-Down

   • Idle Detection: No user interaction for 10m? -> Hibernate (scale to zero).
   • Night Mode: Reduce polling frequency (e.g., 1s -> 60s) during off-hours (00:00-06:00).
   • Cost Cap: If daily spend > $X -> Soft Stop (admin approval required).