LangChain Rate Limits (Python)

Overview

A team deploys 10 Cloud Run workers. Each worker initializes its ChatAnthropic with InMemoryRateLimiter(requests_per_second=10) — they read the docs, they picked a safe-looking number, they shipped. Thirty seconds later the dashboard lights up with 429s: the cluster is pushing 100 RPS to Anthropic's 50 RPM tier-1 ceiling, not the 10 RPS they configured. The name is the fix — InMemoryRateLimiter is in-process. Each worker has its own counter. Ten workers × 10 RPS = 100 RPS to the provider. This is pain-catalog entry P29 and it lands on every team that scales past one pod.

Three more traps wait on the same code path: