Job Queues — What Claude Gets Wrong

You write job processors that work when everything goes right. In production: workers crash mid-job, malformed jobs block the queue forever, the same job runs twice producing duplicate side effects, and nobody notices when the dead letter queue fills up.

The Six Rules

1. Idempotent processing — reprocessing a job MUST produce the same result, not duplicates
2. Poison pill protection — malformed jobs get quarantined, not retried forever
3. Bounded retries with backoff — retry N times with exponential delay, then dead-letter
4. Dead letter handling — permanently failed jobs go somewhere observable with alerting
5. Backpressure — monitor queue depth, shed load when saturated
6. Graceful shutdown — in-progress jobs complete or are released on SIGTERM

Anti-Patterns You Default To