02 / 05

How would you decide whether two event types should share a topic?

Difficulty: 4/10
Event-driven platform, Event contracts, Idempotency

Deciding Whether Two Event Types Should Share a Topic

The decision to share a topic or use separate topics depends on five factors: consumers, schemas, retention, partitioning, and ownership. If two event types have the same consumers, the same schema family, the same retention requirements, the same partitioning key, and the same owning team, they can share a topic. If any of these differ, separate topics are usually better. The most important factor is consumers: if the same consumer group needs both event types, sharing a topic reduces the number of topics and consumers. But if different consumers need different event types, sharing a topic forces every consumer to filter out the events it does not need, which is wasteful and error-prone. The second factor is schemas: if the two event types have different schemas, sharing a topic means the topic has multiple schema types, which complicates schema registry management and can confuse consumers. The third factor is retention: if one event type needs to be retained for 7 days and the other for 30 days, sharing a topic forces a single retention policy, which is either too short for one or too long for the other. The trade-off is between simplicity and isolation. Sharing a topic reduces the number of topics but couples the event types; separate topics increase the number of topics but decouple them.

The mechanism for sharing a topic is that the topic's schema subject covers multiple event types, usually with a union schema or with a common envelope that has a type field. Consumers must inspect the type field and deserialize accordingly. This works but adds complexity: the schema registry must handle the union, and consumers must handle multiple event types. The mechanism for separate topics is that each event type has its own schema subject, its own retention, and its own consumer groups. This is simpler for consumers but increases the number of topics and the operational overhead. The trade-off is between coupling and operational cost. For a small number of event types with the same consumers, sharing is fine. For a large number of event types with different consumers, separate topics are better. Version note: some platforms use a single topic per domain with an envelope schema, while others use a topic per event type. Both patterns are valid; the choice depends on the platform's conventions and the team's preferences. Confluent recommends one topic per event type for clarity and isolation; some event-driven architectures use a topic per aggregate with multiple event types. The key is to be consistent across the platform.

A common mistake is to share a topic because it seems simpler, then discover that different consumers need different retention or different schemas. Another mistake is to create a separate topic for every event type, which leads to topic sprawl and operational overhead. A third mistake is to use a single topic for all events in a domain, which forces every consumer to filter and can cause performance issues if the volume is high. The trade-off is between the number of topics and the coupling between event types. A good rule of thumb is: if two event types are always consumed together and have the same schema family and retention, they can share a topic. Otherwise, separate topics. The decision should be documented and consistent across the platform. Version note: Kafka's topic count is not a hard limit, but thousands of topics can strain the controller and the metadata. For large platforms, a topic-per-event-type approach can result in tens of thousands of topics, which requires careful capacity planning. Some platforms use a topic-per-domain approach to reduce the count, at the cost of consumer filtering.

javascript
  1. 1

    Five factors: consumers, schemas, retention, partitioning, and ownership.

  2. 2

    If all five are the same, sharing a topic is reasonable.

  3. 3

    If any differ, separate topics are usually better.

  4. 4

    Sharing reduces topic count but couples event types.

  5. 5

    Separate topics decouple but increase topic count and operational overhead.

  6. 6

    Shared topics use a union or envelope schema; consumers filter by type.

  7. 7

    Be consistent across the platform; document the decision.

  8. 8

    Thousands of topics require careful capacity planning.

Share

Share via WhatsApp, X, Facebook, LinkedIn or copy link. Open Graph preview enabled.