autonomous-loop-safety
Autonomous Loop Safety
Every rule here is scar tissue from a loop that ran unattended and broke something. None of them were obvious in advance.
The failure modes are not the ones you expect. The agent doesn't go rogue. It quietly stops, or quietly lies, or quietly spends your money — and nothing tells you.
The four that actually cost us
1. Silence must never look like health
The loop could only report verdicts. So a stalled build, a paused host, and a dead watcher all produced output identical to a healthy quiet night: nothing at all. It sat dead for four hours and looked exactly like it was working.
Build a watchdog whose only job is to notice trouble and say so. Host down, quota exhausted, PR stuck past N minutes, work item that never started, and its own stderr. Dedupe alerts (once an hour per condition) so it warns without becoming noise.
Its twin, equally important: noise must never drown the signal. A checker that posts "nothing to do" every 15 minutes is a channel you mute — and then you miss the one that mattered. Alert on trouble. Stay quiet on routine.
2. A quota is not a safety mechanism you discover by hitting it
The loop pushed 49 commits in one day. Every push triggered a host build. But most of those commits were the loop's own bookkeeping — status files nobody serves — and each one spun a full deploy of a static site that hadn't changed.