Data Flow Diagram

Visualize data movement, transformations, and storage across systems to identify bottlenecks and design efficient pipelines.

Context

You are designing how data flows between systems. Document data sources, transformations, sinks, and timing. Read existing architecture documents and data pipelines.

Domain Context

Based on enterprise data architecture patterns and streaming frameworks:

• Batch Processing: High-latency, high-throughput; good for analytics and reports
• Stream Processing: Low-latency, event-driven; good for realtime features and monitoring
• Lambda Architecture: Batch + streaming for both completeness and speed
• Transformation Stages: Extract, transform (business logic), load; validate at each stage
• Data at Rest vs In Motion: Storage systems vs message queues and streams

Instructions