What role did ZooKeeper play in Kafka, and what replaces it in Kafka 4.x?

Instruction: Explain ZooKeeper-mode coordination as legacy architecture, contrast it with KRaft controller metadata, and state the migration boundary for Kafka 4.x.

Context: Assesses Kafka metadata coordination, the separation of controller and partition replication, and awareness of the ZooKeeper-to-KRaft migration boundary.

Updated

Example answer

ZooKeeper belongs to Kafka's legacy architecture. Kafka 4.0 removed ZooKeeper mode; Kafka 4.x uses KRaft.

In a ZooKeeper-mode cluster, ZooKeeper stored coordination metadata such as broker registrations, topic/partition metadata, and configurations. Broker sessions helped detect membership changes. ZooKeeper participated in selecting the active Kafka controller; that controller managed partition-leader decisions. ZooKeeper did not store the topic's message payloads or independently replace Kafka's replication logic.

In KRaft, Kafka manages metadata through a replicated metadata log and a controller quorum. One controller is active and the others replicate the metadata; brokers use that metadata to serve partition data. Controller-quorum availability and partition replication are related operational concerns but distinct mechanisms.

For a legacy cluster, inventory the exact broker and metadata versions and follow the corresponding migration procedure. A ZooKeeper-mode cluster must migrate to KRaft before upgrading to Kafka 4.0+. Validate metadata, broker health, replication, client compatibility, and the documented rollback boundary in a rehearsal. Do not assume replacing a ZooKeeper connection string performs the migration.

Sources

Kafka 4.0 upgrade requirements; KRaft and ZooKeeper differences.

Related Questions