Overview
I was recently asked to compare the performance of Kafka with Chronicle Queue. No two products are exactly alike, and performing a fair comparison is not easy. We can try to run similar tests and see what results we get.
This test is based on Apache Kafka Performance Results.
What was the test used?
One area Kafka tests is multi-threaded performance. In tests we have done, it is neither better or worse to use more more threads (up to the number CPUs you have). We didn’t benchmark this here. All tests use one producer.
Another difference, is that we flush to disk periodically by time rather than by count. Being able to say you are never behind by more than X milli-seconds is often more useful than say 600 messages, as you don’t know how long those messages could have been waiting there. For our tests, we look at flush periods of between 1 ms and 10 ms. In Kafka’s tests, they appears to be every 3 ms approximately.
The message size used was 200 bytes in each case, and we explored the difference writing batches of 1, 2, 5 and 10 messages at once made. We also tried 200 messages in a batch and the performance was similar to batches of 10.
We only tested writing to SSD disks for persistence. Note: Chronicle is broker-less.
The results
The result of this test show you the message rate in terms of MB/s. This is a reasonable way to describe the performance as the message size can vary, but you will get a similar amount of bandwidth, especially over 1 KB message sizes.
| device | flush period (ms) | 1 | 2 | 5 | 10 |
| ssd.ext4 | 1 | 236 MB/s | 300 MB/s | 340 MB/s | 363 MB/s |
| ssd.ext4 | 3 | 378 MB/s | 483 MB/s | 556 MB/s | 583 MB/s |
| ssd.ext4 | 10 | 495 MB/s | 595 MB/s | 687 MB/s | 705 MB/s |
| tmpfs | na | 988 MB/s | 1317 MB/s | 1680 MB/s | 1847 MB/s |
We also tested “writing” to a tmpfs file system. This is much faster as there is no actual writes to a device performed.
Conclusions
It isn’t possible to draw a direct comparison with Kafka as it is a broker based system as must send ever message over TCP. Chronicle can replicate over TCP, however it doesn’t have to, and if you want to maximise performance you will use a high speed network, the fastest being the memory bus of your server.
You can run similar tests and get exceptional results. If you need to handle bursts of hundreds of MB/s, Chronicle may be a better solution.
| Reference: | Kafka Benchmark on Chronicle Queue from our JCG partner Peter Lawrey at the Vanilla Java blog. |
