StreamSets Engine

Flow authoring: This skill covers engine infrastructure and operations only. For building or modifying StreamSets flows, use your path's flow authoring skill.

When to Use This Skill

• Configuring or troubleshooting StreamSets environments and engines
• Engine deployment (Docker/Podman), communication methods (tunneling vs. direct)
• Job lifecycle: execution, monitoring, failover, offset management
• High availability setup and resource management

Engine Characteristics

• Container-based: Runs as Docker/Podman containers in your network
• Data ownership: All processing occurs within your infrastructure
• Continuous processing: Jobs run continuously, processing data as it arrives
• High availability: Multiple engines support failover and load distribution
• Flexible communication: Tunneling (default) or direct connection methods