Design partitioning for per-customer ordering with a few extremely large customers.

The fundamental tension is that per-customer ordering requires all events for a customer to go to the same partition, but a few extremely large customers will create hot partitions that dominate the topic's throughput and consumer lag. The design must balance three goals: preserve per-customer ordering for all customers, prevent a few large customers from starving the rest, and keep operational complexity manageable. There is no single perfect solution; the right design depends on how large the large customers are relative to the rest, whether ordering is required for all event types, and how much complexity the team can operate. The first step is to quantify the skew: what percentage of total traffic comes from the top 1% of customers? If the top 1% is less than 10% of traffic, the default key-by-customer approach may be fine. If the top 1% is 50% or more, you need a more sophisticated design.

The most common approach for extreme skew is to split large customers into their own topics or partitions, while keeping small customers in a shared topic keyed by customer ID. Large customers get a dedicated topic (or a set of dedicated partitions) where their events are keyed by customer ID and ordered. Small customers share a topic with many partitions, keyed by customer ID, so each small customer's events go to one partition but the load is spread across many partitions. This preserves per-customer ordering for everyone and isolates the large customers so they do not starve the small ones. The trade-off is complexity: you now have multiple topics, and consumers must subscribe to all of them. You also need a routing layer that decides which topic a customer's events go to, and that layer must be consistent and highly available. This is essentially a sharding strategy at the topic level, and it is the approach used by many large-scale platforms. Another approach is to shard the large customer's events by a sub-key, such as order ID or region, and accept that ordering is preserved only within the sub-key, not across the entire customer. This is a trade-off between ordering granularity and balance. If the business can tolerate per-order ordering instead of per-customer ordering for large customers, this is simpler and more scalable.

A common mistake is to shard all customers by a composite key, which breaks per-customer ordering for everyone. Another mistake is to create a dedicated topic per large customer without a routing layer, which leads to a proliferation of topics and operational burden. A third mistake is to assume that adding partitions to the shared topic will fix the skew; it will not, because the large customer's key still maps to one partition. The trade-off is between ordering, balance, and complexity. The design must be explicit about which customers get which treatment and how the routing layer handles changes (e.g., when a customer grows from small to large). A hybrid approach is often best: a shared topic for the long tail, a small number of dedicated topics for the largest customers, and a routing layer that is configurable and observable. Version note: Kafka does not provide built-in sharding or routing; you implement it in the producer or in a stream processor. Managed services may have topic limits, so check the quota before designing a dedicated-topic-per-customer approach. For very large customers, you might also consider a separate Kafka cluster entirely, which gives the strongest isolation at the highest operational cost.