Celery Distributed Task Queue

Celery 5.x with Django. Covers distributed workers, task routing, beat scheduling, result backends, and deployment patterns (Compose, K8s, VMs).

Core Architecture

• Beat: Single-instance scheduler. Never scale beyond 1 replica.
• Workers: Stateless consumers. Scale horizontally. Can run off-host.
• Broker: Redis or RabbitMQ. The only shared dependency between beat and workers.

Two Worker Models

1. ORM Workers (DB Access)

Use full Django settings. Tasks import models directly. Must have DB connectivity. Best for: data migrations, aggregation queries, ORM-heavy operations.