Apache Beam Unified Batch And Stream

Overview

Use this skill when Apache Beam is the abstraction layer for both batch and streaming data processing. It helps agents preserve portability without hiding time semantics or runner-specific constraints.

When to Use

• building Apache Beam pipelines for batch, streaming, or unified workloads
• targeting multiple runners such as Dataflow, Flink, Spark, or Direct Runner
• sharing transform logic across batch and streaming modes
• managing windowing, watermarks, triggers, and late data handling
• designing portable pipelines that must run across environments

Do not use this when the workload is locked to a single runner and Beam portability is not a goal.

Workflow