Rate Limiting

Pain Signals — You Need This Pattern When:

• Bursts of agent activity hit the model API's RPM/TPM ceiling and 429s spike
• One greedy session starves others on a shared API key
• Backoff retries pile on during incidents — making outages worse
• Tool calls (external APIs, DB) have their own rate limits and the harness ignores them
• Cost spikes because retries on 429 doubled the bill before respecting Retry-After

Rate limiting is bidirectional. The harness must respect upstream limits and fairly allocate its own capacity across concurrent users/sessions.

Core Principle

Treat upstream limits as shared resources the harness must schedule against. Three interacting concerns: