05 / 05

A business asks for global ordering across millions of events per day. How would you challenge that requirement?

Difficulty: 9/10
Partitioning and ordering

Quantify the volume, find the real ordering domain, and offer per-entity ordering unless a single serialized stream is truly justified

I would not refuse; I would turn the requirement into numbers and consequences. First, millions of events per day is small: 10 million per day is about 115 events per second on average. A single Kafka partition can handle that comfortably, so raw throughput is not the argument. The real cost of global ordering is elsewhere: one partition means one leader and one consumer processing serially for the whole stream. That gives no horizontal scaling, head-of-line blocking (one slow or poison message stalls everything behind it), and a single point of unavailability during leader failover. If volume later grows by 100x, you are stuck with a serialized pipeline and no easy way out.

Then I ask what question the order answers. Almost always, order matters per entity (per account, per order, per device), not across unrelated entities. Order between two different customers' events rarely has business meaning. So I propose the smallest ordering domain that satisfies the real invariant and key by it. Where a true global sequence is needed, I would ask whether it must be enforced at write time or only reconstructed at read time.

  1. 1

    Per-key ordering: key by the entity whose events must be ordered. This scales with partitions and is the default recommendation.

  2. 2

    Version or sequence numbers: producers attach a per-entity sequence, and consumers detect gaps or apply only newer versions. This tolerates reordering without serializing the stream.

  3. 3

    Event-time ordering in the consumer: stream processing with windows and watermarks can sort by event time, accepting bounded lateness. Note that this is event-time order, not arrival order.

  4. 4

    Single-partition topic: legitimate for a low-volume, high-value stream such as a ledger or configuration changes, where simplicity beats scale. I would add an alert on lag and a runbook for failover, and document the growth limit.

  5. 5

    Sequencer or database as the authority: if a strict global sequence is needed for something like invoice numbering, generate it in a transactional store and publish events afterwards, rather than relying on Kafka arrival order.

  6. 6

    Important nuance: even a single partition orders records by the order the broker received them, not by when business events happened. With multiple producers, global order equals arrival order, which may not match real-world order.

  7. 7

    Common mistake: believing you can get global order across topics or by adding consumers on a multi-partition topic. You cannot. Another is accepting the requirement without asking what breaks if two unrelated events swap.

  8. 8

    Version note: idempotent producers (default in clients 3.0+) preserve order within a partition across retries, but check the client version and explicit overrides before relying on that.

javascript

Scenario Questions

Share

Share via WhatsApp, X, Facebook, LinkedIn or copy link. Open Graph preview enabled.