Java Development in Apache Beam

Project Structure

Key Directories

• sdks/java/core - Core Java SDK (PCollection, PTransform, Pipeline)
• sdks/java/harness - SDK harness (container entrypoint)
• sdks/java/io/ - I/O connectors (51+ connectors including BigQuery, Kafka, JDBC, etc.)
• sdks/java/extensions/ - Extensions (SQL, ML, protobuf, etc.)
• runners/ - Runner implementations:
  • runners/direct-java - Direct Runner (local execution)
  • runners/flink/ - Flink Runner
  • runners/spark/ - Spark Runner
  • runners/google-cloud-dataflow-java/ - Dataflow Runner
• examples/java/ - Java examples including WordCount