Questions
5 of 5
Basics
Cluster Architecture
Topics, Partitions and Records
Producers
Consumers
Consumer Groups
Offset Management
Replication and Fault Tolerance
Storage Internals
Networking and Request Processing
Performance and Throughput
Delivery Semantics
Transactions
Kafka Streams
Kafka Connect
Schema and Serialization
Security
Error Handling and Retry Patterns
Ordering and Partitioning Design
Testing
Monitoring and Observability
Debugging and Troubleshooting
Deployment and Operations
Admin and CLI Operations
Reliability and Disaster Recovery
Integration Patterns
Modern Kafka Architecture
Advanced Internals
Scalability and Capacity Planning
Architecture and System Design
Production Best Practices
Expert Edge Cases
Versioning and Upgrades
Admin and CLI Operations
Senior End-to-End Scenarios
Difficulty: 9/10
Automation, Policy enforcement, Self-service administration
Architecture for policy-governed Kafka self-service administration
I would build a control plane that lets teams request approved Kafka resources without unrestricted broker-admin access. A portal or API authenticates the requester, a policy engine validates the request, and a reconciler applies approved changes through Kafka's Admin API. The platform stores desired state, compares it with actual cluster state, and records every decision and operation. Low-risk standard requests can be self-service; risky changes require explicit review and approval.
javascript
- 1
Identity and security: integrate SSO/OIDC, map identities to teams, enforce least privilege, and protect controller credentials. Do not expose cluster-admin credentials to users.
- 2
Policy engine: validate naming, ownership, replication factors, partition limits, retention bounds, cleanup policy, allowed ACL operations, and quotas. Version policies and explain denials clearly.
- 3
Reconciliation: compare desired and actual state. Use idempotent operations, retries with backoff, bounded concurrency, and explicit handling for partial failure and concurrent requests.
- 4
Prevent privilege escalation through arbitrary ACL requests. Grant the controller only the permissions it needs and enforce authorization at the cluster as well as the portal.
- 5
For quotas, define tenant/client limits, ownership, and change controls. Monitor usage rather than silently changing shared tenant limits.
- 6
Treat shortening retention or changing cleanup policy as potentially destructive. Require elevated approval where appropriate and explain that retained data may be permanently deleted.
- 7
Audit requester, approver, policy version, requested diff, result, timestamps, and correlation ID in access-controlled, tamper-resistant records.
- 8
Expose lifecycle states such as requested, validated, awaiting approval, applying, succeeded, and failed. Add drift detection, alerts, dashboards, and reconciliation safeguards.
- 9
Trade-off: central governance improves consistency but can become a bottleneck or critical dependency. Keep routine low-risk changes self-service and reserve approvals for high-impact operations.
- 10
Common mistake: assuming API success proves the cluster reached desired state, or allowing users to bypass the platform with direct administrative credentials.
- 11
Use Kafka Admin API and authorization features supported by the deployed Kafka version and security configuration; validate ACL semantics and supported settings against the actual cluster.
Scenario Questions
0-2 years experience
2-5 years experience
5-8 years experience
8+ years experience
- 1A topic creation request times out after submission. How would you determine whether it succeeded before retrying? How would you automate and govern this process?
- 2A controller creates a topic and ACL but fails before applying quotas. How would you model state and recover safely? How would you recover safely from a partial failure?