arch-event-driven

Purpose

This skill implements event-driven architectures using Kafka, RabbitMQ, and related patterns like event sourcing, CQRS, pub/sub, dead letter queues, and schema registries. It helps design scalable, decoupled systems for real-time event processing in microservices environments.

When to Use

Use this skill for scenarios requiring asynchronous communication, such as microservices that need to react to events without direct dependencies. Apply it in high-volume data pipelines, real-time analytics, or when decoupling producers and consumers is essential, like in e-commerce order processing or IoT data streams. Avoid it for simple synchronous operations where polling suffices.

Key Capabilities

• Set up Kafka topics and partitions for event streaming.
• Implement event sourcing by storing events in Kafka for state reconstruction.
• Apply CQRS to separate read and write models, using Kafka for commands and queries.
• Manage pub/sub with Kafka consumer groups for fan-out scenarios.
• Handle failures via dead letter queues in Kafka or RabbitMQ.
• Enforce schema validation using Confluent Schema Registry for Avro schemas.

Usage Patterns

To implement pub/sub, create a Kafka topic and have producers publish events; consumers subscribe via groups. For event sourcing, store all state changes as events in a Kafka stream and replay them to build current state. In CQRS, route commands to a write service (e.g., via Kafka producer) and queries to a read service (e.g., from a materialized view). Use dead letter queues by configuring Kafka topics to redirect failed messages. Always define event schemas in JSON or Avro format for consistency.