Python Development in Apache Beam

Project Structure

Key Directories

• sdks/python/ - Python SDK root
  • apache_beam/ - Main Beam package
    • transforms/ - Core transforms (ParDo, GroupByKey, etc.)
    • io/ - I/O connectors
    • ml/ - Beam ML code (RunInference, etc.)
    • runners/ - Runner implementations and wrappers
    • runners/worker/ - SDK worker harness
  • container/ - Docker container configuration
  • test-suites/ - Test configurations
  • scripts/ - Utility scripts