How do you implement Kafka consumer groups and what guarantees does partition assignment give you in NestJS?

All instances of a service share the same groupId — Kafka assigns each partition to exactly one consumer in the group. This means messages with the same key are always processed by the same instance, guaranteeing ordering per key. More partitions allow more concurrent consumers up to the partition count.

Consumer group scaling and manual offset management

Consumer group partition assignment rules:

Each partition is assigned to exactly one consumer in the group — no duplicate processing within a group.
More partitions = more consumers can work in parallel — set partition count at least equal to max instance count.
Same-key messages always route to the same partition and therefore the same consumer instance.
When an instance joins or leaves, Kafka rebalances partition assignments across the group.
Manual offset commits allow exactly-once semantics — commit after processing, not before.

Question Loading...

How do you implement Kafka consumer groups and what guarantees does partition assignment give you in NestJS?

Consumer group scaling and manual offset management

Consumer group partition assignment rules:

Each partition is assigned to exactly one consumer in the group — no duplicate processing within a group.
More partitions = more consumers can work in parallel — set partition count at least equal to max instance count.
Same-key messages always route to the same partition and therefore the same consumer instance.
When an instance joins or leaves, Kafka rebalances partition assignments across the group.
Manual offset commits allow exactly-once semantics — commit after processing, not before.