Questions
4 of 5
Basics
Cluster Architecture
Topics, Partitions and Records
Producers
Consumers
Consumer Groups
Offset Management
Replication and Fault Tolerance
Storage Internals
Networking and Request Processing
Performance and Throughput
Delivery Semantics
Transactions
Kafka Streams
Kafka Connect
Schema and Serialization
Security
Error Handling and Retry Patterns
Ordering and Partitioning Design
Testing
Monitoring and Observability
Debugging and Troubleshooting
Deployment and Operations
Admin and CLI Operations
Reliability and Disaster Recovery
Integration Patterns
Modern Kafka Architecture
Advanced Internals
Scalability and Capacity Planning
Architecture and System Design
Production Best Practices
Expert Edge Cases
Versioning and Upgrades
Admin and CLI Operations
Senior End-to-End Scenarios
Difficulty: 8/10
ISR and leader election
Find the common factor (one broker, one rack, or cluster-wide), then check broker health, disk, network, replication throttles and recent changes
Under-replicated partitions (URP) mean some followers are not in the ISR, so durability and failover safety are reduced. The first step is to find the pattern, because the cause differs. If every URP involves the same broker, that broker is the problem: it may be down, slow or overloaded. If the URPs follow a rack or availability zone, suspect the network or infrastructure there. If they are spread across all brokers, suspect cluster-wide load, a leader-side bottleneck or a recent change such as a deploy, a reassignment or a traffic spike. I also check whether it is just noise: a reassignment in progress, or a recently restarted broker catching up, causes temporary URPs that are expected.
javascript
- 1
Broker down or restarting: check process health, OOM kills and crash loops. Followers on a restarting broker show as URP until they catch up, which can take long after a big outage.
- 2
Disk: high await, a failing drive, full disk or a slow volume on the follower (or on the leader) makes fetches and appends slow. A single slow disk can hold back every partition it hosts.
- 3
Network: saturated NICs, packet loss, or cross-AZ links limiting replication throughput. Compare inbound produce traffic times (RF minus 1) against available inter-broker bandwidth.
- 4
Broker load: low RequestHandlerAvgIdlePercent or NetworkProcessorAvgIdlePercent means the broker is saturated and cannot serve replica fetches promptly. Look for traffic spikes, too many partitions per broker, or leader skew concentrating load.
- 5
JVM: long GC pauses stall fetcher threads and can also delay heartbeats. Correlate pauses with ISR shrink events.
- 6
Replication tuning: num.replica.fetchers too low for the partition count, replica.fetch.max.bytes too small for large batches, or an over-aggressive replication throttle (leader.replication.throttled.rate) left on after a reassignment.
- 7
Recent changes: partition reassignment, a new high-volume producer, a config change, a deploy or a topic created with a very large partition count often line up exactly with the first URP alert.
- 8
Common mistake: restarting brokers one after another to clear URPs. It adds catch-up load, destroys evidence and can turn URPs into UnderMinIsr and write outages.
- 9
Common mistake: raising replica.lag.time.max.ms to silence the alert. It hides slow followers, lets them hold back acks=all writes, and delays detection of real problems.
- 10
Common mistake: treating every URP as an incident. Check whether a reassignment or restart explains it, and alert on duration and UnderMinIsr rather than on any single URP sample.
- 11
Version note: metric names and the KRaft tooling above are current for modern releases, while ZooKeeper-based clusters needed controller and ZooKeeper health checks as well. Under KIP-966 (early access in 4.0) the reported ISR and eligible-replica details may differ, so verify on your version.
Scenario Questions
0-2 years experience
2-5 years experience
- 1URPs appeared right after a partition reassignment began and are not clearing. What would you check about throttles and progress, and how would you decide to adjust them?
- 2A broker was down for six hours and is back, but URPs persist for a long time. Explain why and how you would estimate the catch-up time.
5-8 years experience
- 1URPs are spread across all brokers but only appear during peak hours, and request handler idle percentage drops near zero. Walk through your diagnosis and short- and long-term fixes.
- 2Only partitions with followers in one availability zone are under-replicated. What infrastructure and configuration causes do you investigate, and how do you confirm each?
8+ years experience
- 1Design the detection and runbook system that distinguishes planned URPs (restarts, reassignments) from incidents across 100 brokers, with alerts that page only when data safety is actually at risk.
- 2A cluster hosts 40,000 partitions per broker group and URPs recur after every rolling restart. Analyze the scaling limits involved and propose changes to topology, settings and operational procedure.