What would you verify before migrating from ZooKeeper mode to KRaft?

Before migrating from ZooKeeper mode to KRaft, you must verify five things: version compatibility, configuration readiness, tooling compatibility, recovery and rollback procedures, and the migration procedure itself. Version compatibility is the first check: the migration tooling is available in Kafka 3.5 and later, and you should be on a version that supports the migration path. If you are on an older version, you must upgrade first. Configuration readiness is the second check: KRaft requires new configurations (process.roles, node.id, controller.quorum.voters) and does not use broker.id or zookeeper.connect. You must prepare the new configuration files for controllers and brokers, and ensure that the node IDs are unique and consistent. Tooling compatibility is the third check: some tools and scripts that work with ZooKeeper-based Kafka may not work with KRaft. For example, scripts that read metadata from ZooKeeper or use ZooKeeper-specific commands must be updated. Check your monitoring, automation, and client tools. Recovery and rollback is the fourth check: you must have a plan for what to do if the migration fails. Can you roll back to ZooKeeper mode? The migration is designed to be reversible during the hybrid phase, but after the final switch to KRaft-only, rollback is not possible without restoring from backup. The migration procedure itself is the fifth check: the migration from ZooKeeper to KRaft runs in a hybrid mode where both systems are active, then migrates metadata, then switches to KRaft-only. You must test this procedure in a staging environment that mirrors production.

The mechanism of the migration is a multi-phase process. In the first phase, you deploy controllers alongside the existing ZooKeeper-based brokers and configure the brokers to use the controllers for metadata while still writing to ZooKeeper. This is the hybrid mode. In the second phase, you migrate the metadata from ZooKeeper to the KRaft metadata log using the migration tooling. In the third phase, you switch the cluster to KRaft-only by removing the ZooKeeper dependency and restarting the brokers with the new configuration. Throughout the process, the cluster remains available, but there are moments of increased risk, particularly during the metadata migration and the final switch. The trade-off is between the benefit of removing ZooKeeper and the risk of the migration. The migration is complex and requires careful planning, testing, and execution. For a mission-critical cluster, you should run the migration in a staging environment first, measure the time and the impact, and have a rollback plan. Version note: the migration tooling is available in Kafka 3.5+; earlier versions do not support the migration. Also, the migration is not supported for all configurations; for example, if you use custom security configurations or custom authorizers, you may need to verify compatibility. Always check the official migration documentation for your version.

A common mistake is to attempt the migration without testing it first. The migration involves changes to the cluster's metadata management, and a mistake can cause an outage or data loss. Another mistake is to assume that the migration is a simple upgrade; it is not, and it requires a maintenance window or at least a period of increased monitoring. A third mistake is to ignore the tooling and automation that depends on ZooKeeper; if you have scripts that read from ZooKeeper, they will break after the migration. The trade-off is between staying on a deprecated architecture and taking on the migration risk. ZooKeeper is removed in Kafka 4.0, so eventually you must migrate. The question is when and how. For most teams, the right approach is to migrate during a planned maintenance window, with a staging test first and a rollback plan. Version note: the migration procedure has evolved with each Kafka release; check the documentation for your specific version. Also, KRaft has different metrics and monitoring; update your dashboards and alerts before the migration.