Redpanda Connect

Redpanda Connect (formerly Benthos) is a declarative stream processor: you write a YAML config that specifies an input, an optional pipeline of processors, and an output. Connect reads from the input, runs each record through the processors, and writes to the output — with at-least-once delivery guarantees. It ships as the redpanda-connect binary and is also accessible via rpk connect.

The component library is large (hundreds of inputs, processors, outputs, caches, buffers, rate limits). This skill teaches the config model, how to run pipelines, how to discover components, Bloblang for transformations, error handling, and several canonical patterns. For CDC-specific pipelines see the connect-cdc-* skills; for debugging see connect-debugging.

Quickstart

1. Minimal kafka-to-kafka pipeline with a Bloblang transform

# my-pipeline.yaml
input:
  redpanda:
    seed_brokers: ["localhost:9092"]
    topics: ["raw-events"]
    consumer_group: connect-demo
    start_offset: earliest