04 / 05

How would you explain Kafka's role in an event-driven microservices architecture?

Difficulty: 8/10
Core concepts

Kafka is the durable event backbone that decouples producers from consumers in time, space and scale

In an event-driven architecture, services publish facts about what happened (OrderPlaced, PaymentCaptured) and other services react. Kafka's role is to be the durable, ordered, replayable backbone that carries those facts. The producer does not know who consumes its events, so adding a new consumer such as a fraud or recommendation service requires no change to the producer. That is loose coupling. Because Kafka retains events, consumers can be down, slow or newly created and still catch up, which is temporal decoupling. Because processing is asynchronous, a spike in orders does not cascade into synchronous failures across services.

Replay is what separates this from plain messaging. A consumer with a bug can be fixed and rewound, a new read model can be rebuilt from history, and an audit trail comes almost for free. Keys give per-entity ordering, and consumer groups give horizontal scaling. Compacted topics let you keep the latest state per key, which supports data sharing and cache or read-model bootstrapping.

javascript
  1. 1

    Trade-off: eventual consistency and harder debugging. A synchronous call is easier to trace and reason about, so I keep request/response for queries that need an immediate answer and use events for state changes other services care about.

  2. 2

    Common mistake: the dual-write problem. Updating a database and publishing to Kafka as two separate steps can leave them inconsistent. Use the transactional outbox pattern with CDC, or a single source of truth.

  3. 3

    Common mistake: assuming exactly-once end to end. At-least-once delivery is the realistic default, so consumers must be idempotent (dedupe on eventId). Kafka transactions give exactly-once only within a Kafka read-process-write pipeline.

  4. 4

    Common mistake: treating events as commands or leaking internal DB schemas. Events should be stable, versioned contracts managed with a schema registry and compatibility rules.

  5. 5

    Alternative: Kafka versus event sourcing. Kafka can be the event store, but many teams keep the system of record in a database and use Kafka for propagation. I pick event sourcing only when auditability and rebuildable state are core requirements.

  6. 6

    Version note: idempotence is on by default in newer clients (3.0+), and KRaft replaces ZooKeeper in 4.0. Confirm what your platform runs.

Share

Share via WhatsApp, X, Facebook, LinkedIn or copy link. Open Graph preview enabled.