7 min read
|
Saved February 14, 2026
|
Copied!
Do you care about this?
This article discusses the performance benchmarks of Diskless Kafka (KIP-1150), showcasing significant cost savings and low latency achieved using just six m8g.4xlarge machines. It emphasizes the importance of realistic and open-source testing to validate the effectiveness of Diskless topics in Apache Kafka deployments.
If you do, here's more
Aiven's open-source benchmark for KIP-1150 Diskless Topics highlights significant performance and cost benefits for Apache Kafka users. The benchmark tests a Diskless Kafka setup with a workload of 1 GiB/s in and 3 GiB/s out across three availability zones, using just six m8g.4xlarge machines. This configuration maintained CPU usage below 30% and achieved a P99 end-to-end latency of around 1.6 seconds. The financial impact is substantial, with annual infrastructure costs slashed from over $3.32 million to under $288,000, reflecting more than a 94% reduction.
The benchmark identifies metadata operations as a cost factor, generating approximately 1.4 MiB/s of cross-AZ traffic, costing about $7,803 yearly. In contrast, Diskless topics eliminate over $3 million in cross-AZ replication costs and $222,576 in disk expenses compared to traditional Kafka deployments. Aiven aims to extend Kafka rather than replace it, allowing both Diskless and classic topics to coexist on the same cluster. The benchmarks are designed to be realistic and reproducible, avoiding setups that could skew results.
Testing utilized the OpenMessaging Benchmark (OMB) framework, employing a fully managed Inkless Kafka cluster. The setup consisted of 144 producers and 144 consumers, optimized for throughput. Key configurations included a linger.ms of 100ms for batching, with uncompressed data traffic to avoid bottlenecks. The results showed stability across the workload, with the cluster potentially able to support 2 to 3 times the tested throughput. The article emphasizes the importance of realistic benchmarks to provide a reliable data point for the community in evaluating Diskless topics.
Questions about this article
No questions yet.