Questions
1 of 5
Basics
Cluster Architecture
Topics, Partitions and Records
Producers
Consumers
Consumer Groups
Offset Management
Replication and Fault Tolerance
Storage Internals
Networking and Request Processing
Performance and Throughput
Delivery Semantics
Transactions
Kafka Streams
Kafka Connect
Schema and Serialization
Security
Error Handling and Retry Patterns
Ordering and Partitioning Design
Testing
Monitoring and Observability
Debugging and Troubleshooting
Deployment and Operations
Admin and CLI Operations
Reliability and Disaster Recovery
Integration Patterns
Modern Kafka Architecture
Advanced Internals
Scalability and Capacity Planning
Architecture and System Design
Production Best Practices
Expert Edge Cases
Versioning and Upgrades
Admin and CLI Operations
Senior End-to-End Scenarios
Difficulty: 2/10
Coordination and rebalancing
A consumer group is a set of consumers sharing a group.id that split a topic's partitions among themselves, so each partition is read by exactly one member of the group
A consumer group is how Kafka scales consumption and lets multiple applications read the same data independently. Every consumer that sets the same group.id belongs to the same group. Kafka assigns each partition of the subscribed topics to exactly one member of the group at a time, so the members divide the work without processing the same record twice. A different group.id is a completely independent group: it gets its own full copy of the stream and tracks its own offsets. That is how billing, analytics and fraud services can all read the orders topic without affecting each other.
Mechanically, one broker acts as the group coordinator for each group. It tracks membership through heartbeats and stores committed offsets per group, topic and partition in the internal __consumer_offsets topic. In the classic protocol, when membership changes, one member (the group leader) computes the assignment using a partition assignor and the coordinator distributes it. The important consequence is that the group, not the individual consumer, is the unit that owns a position in the log. If a member dies, another member takes over its partitions and resumes from the last committed offset.
javascript
- 1
Trade-off: a group gives automatic load sharing and failover, but parallelism is capped by the partition count, and any membership change triggers coordination work (a rebalance).
- 2
Alternative: manual assign() skips group management entirely. You control partitions yourself and get no automatic failover. I use it for tools, replay jobs, or when I need a fixed partition mapping.
- 3
Common mistake: thinking two consumers in the same group both receive every record. Within a group each record goes to one member; to broadcast, use different group IDs.
- 4
Common mistake: thinking a group is tied to a topic. A group can subscribe to many topics, and one topic can be read by many groups.
- 5
Common mistake: reusing a group.id across unrelated applications. They will silently split partitions and offsets between each other.
- 6
Version note: Kafka 4.0 makes the KIP-848 consumer group protocol generally available (group.protocol=consumer), moving assignment to the broker. The group concept is the same, but older clusters and clients use the classic protocol described above, so state which one you mean.
Scenario Questions
0-2 years experience
2-5 years experience
5-8 years experience
- 1Two different services in your company accidentally share a group.id and each sees only about half of the data. How do you detect this from Kafka alone, and how do you fix it safely?
- 2Explain what the group coordinator does for a group and what happens to the group if the broker hosting its coordinator fails.
8+ years experience
- 1You are writing the platform's rules for naming, ownership and lifecycle of consumer group IDs across 200 services. What do you standardize, how do you detect orphaned or colliding groups, and what do you automate?
- 2Compare group-managed consumption with manual partition assignment for a stateful stream-processing application. When would you deliberately give up group management?