Questions
1 of 5
Basics
Cluster Architecture
Topics, Partitions and Records
Producers
Consumers
Consumer Groups
Offset Management
Replication and Fault Tolerance
Storage Internals
Networking and Request Processing
Performance and Throughput
Delivery Semantics
Transactions
Kafka Streams
Kafka Connect
Schema and Serialization
Security
Error Handling and Retry Patterns
Ordering and Partitioning Design
Testing
Monitoring and Observability
Debugging and Troubleshooting
Deployment and Operations
Admin and CLI Operations
Reliability and Disaster Recovery
Integration Patterns
Modern Kafka Architecture
Advanced Internals
Scalability and Capacity Planning
Architecture and System Design
Production Best Practices
Expert Edge Cases
Versioning and Upgrades
Admin and CLI Operations
Senior End-to-End Scenarios
Difficulty: 2/10
Partitioning and ordering
A partition is an ordered, append-only log that is the unit of ordering, parallelism and replication
A topic is a logical name; the data physically lives in partitions. Each partition is an append-only, immutable log stored on a broker as segment files, and every record in it gets a sequential offset starting at 0. Offsets are only meaningful within a partition, so the real identity of a record is topic, partition, offset. Partitions matter because three separate properties hang off them.
- 1
Ordering: Kafka guarantees order only within a partition. Two records in different partitions have no defined relative order.
- 2
Parallelism: within one consumer group, each partition is assigned to at most one consumer at a time. So the partition count is the ceiling on useful consumers in a group. Partitions also spread write and read load across brokers.
- 3
Replication and failure handling: replication is configured per partition. Each partition has one leader that serves reads and writes and follower replicas that stay in sync (the ISR). When a broker dies, leadership moves partition by partition, not topic by topic.
javascript
The trade-off is that more partitions give more parallelism but cost more: more open file handles, more replication traffic, more producer batch memory, longer leader elections and slower rebalances. Common mistakes I see: assuming a topic has a single global order, adding consumers beyond the partition count and expecting more throughput (the extras sit idle), and forgetting that you can increase partitions but never decrease them. Adding partitions later also changes the key-to-partition mapping for keyed data. Version note: very high partition counts per broker were a real problem in ZooKeeper-based clusters; KRaft (the only mode from Kafka 4.0) raises that ceiling, but I would still verify limits against my version and hardware.
Scenario Questions
0-2 years experience
2-5 years experience
- 1A broker crashes and some partitions of a topic keep working while others briefly stall. Explain why the impact is partial and what determines how long each partition is unavailable.
- 2A teammate wants to reduce a topic from 12 partitions to 6 to save resources. What is the problem and what are your options?
5-8 years experience
- 1Brokers in your cluster each host a very different number of partition leaders, and CPU is uneven. How do you confirm this and rebalance without disrupting producers?
- 2Producers report increasing memory use and latency after a topic was grown from 20 to 500 partitions. What is the likely mechanism and how would you verify it?
8+ years experience
- 1You are setting platform limits on partitions per topic and per cluster for a multi-tenant Kafka service. Which resources and failure modes drive the limits, and how do you enforce them?
- 2Explain, from the storage layer up, why a partition is the right unit for both parallelism and replication, and what you give up by making it so.