How would you optimize Kafka's Java Virtual Machine (JVM) settings for better performance?

Official answer

First identify the process being tuned: broker, controller, Connect worker or Streams application. Their allocation and state patterns differ. Use a supported, patched Java release for the deployed Kafka version and record the current flags before changing anything.

A broker's heap is not a buffer that must fill before Kafka writes its log. Kafka writes log data through the filesystem and relies heavily on the operating system's page cache. The memory budget must also allow native/direct buffers, thread stacks, runtime metadata, other processes and the operating system. Increasing heap can take memory away from the page cache or exceed a container's memory limit.

Collect GC logs, heap occupancy after collection, allocation rates, pause distributions, process/container memory and request latency under a representative workload. Distinguish heap exhaustion from native-memory pressure, disk stalls and CPU saturation. Include partition count, payloads, compression, replication and recovery in the workload definition.

Start from the Kafka/JDK recommendations and make one justified change at a time. A larger heap can reduce collection frequency while increasing other costs. Collector and pause-target changes are hypotheses to test, not guarantees. Do not enable string deduplication, suppress explicit GC or copy a fixed heap size without evidence that it addresses this workload.

Canary the change, compare throughput, tail latency, GC and total memory, and retain a rollback plan. Include sustained load and recovery tests. Report measured results and limits; business engagement metrics alone cannot attribute a JVM improvement.

Supported Java and example JVM settings, filesystem/page-cache behavior