CDC streaming pipeline (Kinesis -> Redshift Serverless / S3)

This skill captures the lived patterns for building a real-time CDC pipeline: a transactional source publishes change events to Kinesis, a Lambda consumer applies them to a sink (Redshift Serverless event log, or S3 / Iceberg via Firehose). It is source-agnostic: the producer-side notes are split into "any CDC source" and "Aurora DSQL preview" callouts.

When to use

• You are wiring up a CDC pipeline from Aurora DSQL, Aurora Postgres, or RDS to Kinesis Data Streams (or MSK) with a Lambda consumer.
• You need to write the consumer Lambda that pushes CDC rows into Redshift Serverless via the Redshift Data API.
• You are deciding how to model CDC events at the sink (append-only vs in-place upsert, how to absorb schema drift).
• You want the operational gotchas (parameter limits, async statement polling, retries, poison records) before you debug them.