staff-engineering-skills-streams-vs-batch
Installation
SKILL.md
Streams vs Batch Trap
Batch-to-stream is a rewrite, not a refactor. Before writing a processing pipeline, ask: what's the latency requirement, and will it change?
Decision Framework
Ask these questions before writing the first line of processing code:
| Question | Batch | Stream |
|---|---|---|
| Acceptable latency? | Minutes to hours | Seconds or less |
| Throughput trajectory? | Stable or slow-growing | Growing fast or unpredictable |
| Failure isolation? | Whole batch can retry | Must handle per-item failure |
| Ordering matters? | No, or within batch is fine | Yes, across items |
| "Real-time" ever mentioned? | No | Yes -- build for it now |
If the answer to ANY row points to stream, build for streaming from the start. You cannot cheaply add streaming later.