How does Kafka achieve fault tolerance?

Official answer

Kafka uses replicated partition logs and a controller quorum to recover from supported failures. Partitioning distributes work; replication provides redundant copies. A topic with many partitions but replication factor 1 still loses availability when a partition's only broker is unavailable.

A common data configuration is replication factor 3, min.insync.replicas=2 and producer acks=all, with unclean leader election disabled. Successful acknowledgements require the configured in-sync replication conditions. If too few suitable replicas remain, writes must fail or wait rather than pretend the durability target was met. This trades availability for safety; it is not protection against losing every copy.

When a leader fails, the controller selects an eligible replacement. Current Kafka versions can use Eligible Leader Replicas as well as the ISR under the documented safety rules, so “only an ISR member can ever become leader” is too broad. Do not enable unsafe election merely to hide a recovery problem.

KRaft separately replicates metadata through its controller quorum. Three voting controllers need a majority and can tolerate one controller failure. This count does not mean that any arbitrary two data-broker failures are safe for every partition. Replica placement across failure domains and the actual synchronized copies still matter.

Clients retry and refresh metadata after failures. Idempotent producers help prevent retry duplicates, while consumers must handle reassignment and replay without corrupting external side effects.

Monitor under-replicated/unavailable partitions, storage and quorum health, and rehearse failover and restore. Replication is not a backup against accidental deletion, malicious writes or a bad application update replicated to every copy.

Replication design, eligible replicas, controller quorum