What ordering guarantee does Kafka provide?

Kafka guarantees ordering only within a single partition. Records written to the same partition with the same producer are appended to the log in the order they were sent, and consumers read them in that same order. There is no ordering guarantee across partitions. If a topic has multiple partitions, records with different keys may be written to different partitions, and there is no defined order between them. This is not a limitation to work around; it is the fundamental design that allows Kafka to scale horizontally. If you need global ordering across all records in a topic, you must use a single partition, which means a single consumer and no parallelism. Most systems do not need global ordering; they need ordering per entity, such as per customer, per order, or per account. That is achieved by choosing a key such that all records for the same entity go to the same partition.

The mechanism for per-partition ordering is the partition log itself. A partition is an append-only sequence of records, each with a monotonically increasing offset. A producer writing to a partition appends records in order, and the broker assigns offsets in that order. A consumer reading from a partition reads records in offset order. If the producer retries a batch, the idempotent producer (enabled by default in recent clients) ensures that retries do not create duplicates or reorder records within the partition. Without idempotence, a retry could cause duplicates, and with max.in.flight.requests.per.connection greater than 1, it could cause reordering on retry. This is why enable.idempotence=true is important when ordering matters. The trade-off is between ordering and throughput: a single partition gives strict ordering but limits throughput to what one broker and one consumer can handle. More partitions give more throughput but only per-key ordering. Version note: enable.idempotence defaults to true in Kafka 3.0 and later; in earlier versions it defaulted to false, and you had to set it explicitly to get ordering-safe retries.

A common mistake is to assume that Kafka provides global ordering because it is a log. It does not, unless you use one partition. Another mistake is to use a null key, which causes the producer to distribute records round-robin across partitions; this gives no ordering guarantee at all, even for records that belong to the same entity. A third mistake is to assume that ordering is preserved across a consumer rebalance. During a rebalance, partitions are reassigned, and a consumer that takes over a partition starts from the last committed offset. Records that were in-flight but not committed may be reprocessed, which can break ordering if the consumer is not idempotent. The trade-off is between ordering and availability: strict global ordering requires a single partition, which is a single point of failure and a throughput bottleneck. Per-key ordering with multiple partitions gives scalability but requires that the key correctly captures the ordering boundary. Version note: Kafka 3.x has improved rebalance protocols (cooperative sticky assignors) that reduce the impact of rebalances, but the fundamental ordering guarantee remains per-partition.