Kafka Resilience And Schema Evolution

Overview

Generic streaming guidance is not enough for production Kafka. Agents routinely introduce breaking schema changes, under-provisioned durability settings, and missing poison-message isolation. This skill mandates enforceable broker, producer, consumer, and registry guardrails before any production change ships.

When to Use

• creating or modifying Kafka topics, producers, or consumers
• setting or changing schema registry compatibility policies
• designing dead-letter queue (DLQ) routing for poison pill messages
• hardening producer durability (acks, retries, idempotence)
• reviewing consumer lag, replay, or failover behavior on Kafka-backed pipelines

Pair with streaming-and-messaging-systems for broader event design. Pair with avro-protobuf-json-schema-registry when registry subjects and compatibility CI are in scope.

Workflow