How do you implement Kafka consumer groups and what guarantees does partition assignment give you in NestJS?

Q: How do you implement Kafka consumer groups and what guarantees does partition assignment give you in NestJS?

// Multiple instances with the same groupId — Kafka rebalances partitions // Instance 1 might consume partitions 0, 1 // Instance 2 might consume partitions 2, 3 // All configured identically: NestFactory.createMicroservice(AppModule, { transport: Transport.KAFKA, options: { consumer: { groupId: 'inventory-service' }, // same groupId across instances client: { brokers: ['kafka:9092'] }, }, }); // Messages with the same key go to the same partition // and therefore to the same consumer instance // orderId=42 always processed by the same instance -> ordering guaranteed // Manual offset commit — commit only after successful processing @EventPattern('order-events') async handle( @Payload() data: OrderEvent, @Ctx() ctx: KafkaContext, ): Promise { await this.process(data); const { offset } = ctx.getMessage(); const partition = ctx.getPartition(); const topic = ctx.getTopic(); await this.consumer.commitOffsets([{ topic, partition, offset }]); } Each partition is assigned to exactly one consumer in the group — no duplicate processing within a group. More partitions = more consumers can work in parallel — set partition count at least equal to max instance count. Same-key messages always route to the same partition and therefore the same consumer instance. When an instance joins or leaves, Kafka rebalances partition assignments across the group. Manual offset commits allow exactly-once semantics — commit after processing, not before.

Difficulty: 6/10

Kafka consumer groups, NestJS microservices, partition assignment guarantees

All instances of a service share the same groupId — Kafka assigns each partition to exactly one consumer in the group. This means messages with the same key are always processed by the same instance, guaranteeing ordering per key. More partitions allow more concurrent consumers up to the partition count.

Consumer group scaling and manual offset management

Consumer group partition assignment rules:

1
Each partition is assigned to exactly one consumer in the group — no duplicate processing within a group.
2

How do you implement Kafka consumer groups and what guarantees does partition assignment give you in NestJS?

Scenario Questions

Follow-up Questions