Technologies come and go, but one thing remains constant.
We love the complex components that make our lives easier when designing enterprise solutions and as architects and developers we are constantly searching for ways to make our lives easier.
One way to do this is to keep up on the popular new sites that relate to technologies of interest. Another way is to read as much as we can in the form of books, magazines or blogs on technology topics.
Introduction
A more subtle and deeper dive into the roots of the technologies that interest us is found in the research arena. For example, on this site you will find some of the earlier works I was involved in when I was supporting research into generic information retrieval at the Radboud University Nijmegen, Netherlands. This experience has shown the value of watching the more rigorous and deeper sources that contribute research papers to various scientific based conferences in the areas of technology that interest me.
When Mark Proctor pointed out a new comparative study of Complex Event Processing (CEP) engines that included the JBoss community based Drools project engine, it was time to dive into the paper and examine the results as they pertain to JBoss products. The community components referenced in this paper are part of the Drools project , which can be found in our directly supported JBoss Business Rules Management System (BRMS) and JBoss BPM Suite products. The community version used was 5.5 which was integrated into JBoss BRMS from version 6.0 and beyond.
I do realize not everyone enjoys the rigorous and often mathematical foundations that are used in these papers to prove and support the theoretical results. Therefore, in an attempt to bring the JBoss relevant information to you with regards to the ties we have between community and products, this article will focus on extracting the CEP related results for Drools only.
You are free to download and read the complete original paper that was presented at 10th International Conference on Cyber Warfare and Security (ICCWS-2015) as the authors were so kind as to put the entire paper online.
Overview
This paper takes a look a a class of information systems that gathers data and events together to provide the ability to audit or maintain some form of security in today’s complex information technology environments. They classify these systems in the paper as Software Information and Event Management (SIEM) system, into which the popular open source rule-based Drools Complex Event Processing (CEP) engine fits for the authors evaluations.
The authors see the most important feature of these systems to be “…the correlation engine, which is used to normalize, reduce, filter and aggregate events from a set of heterogeneous inputs.” The paper promises to compare and present performance evaluations of the following correlation engines:
- Simple Event Correlator (SEC)
- Esper
- Nodebrain
- Drools, which is supported by Red Hat in JBoss BRMS & JBoss BPM Suite
The remainder of this article will refer to the results in relation the the supported JBoss BRMS which productizes the Drools CEP engine that the authors consider a correlation engine in this paper. Remember that JBoss BPM Suite is a super set of JBoss BRMS, so therefore we choose to focus on JBoss BRMS for this article.
The testing architecture that pushed a load through the JBoss BRMS CEP component using a set of rules for processing, monitors the progress and then filters out the results into a report. Events were generated to trigger rules and in a predefined distribution.
The paper also states that the CEP component was optimized to produce the best results possible, but the authors do not present any details as to what this might entail. Testing was done on a virtualized Xeon CPU X5660 processor, Linux based operating system, with 4GB of RAM allocated and there were multiple runs of the test suite.
Benchmark
The final numbers were taken as an average over the results as measured over three runs and reflect measurements based on execution time and throughput (events processed per second). The following shows the results for set number of rules with variable number of events and for set number of events with variable number of rules.
1. Execution time and throughput for 500 rule set
The events are scaled up and the rule set remains static in size.
- 1k events
- Throughput – 125 events/sec
- Time – 8 sec
- 10k events
- Throughput – 1111 events/sec
- Time – 9 sec
- 100k events
- Throughput – 6250 events/sec
- Time – 16 sec
- 1 million events
- Throughput – 14286 events/sec
- Time – 70 sec
Compared to the other engines, with medium to larger event sets we see dramatic processing throughput increases, being the fasted correlation engine measured by factor of two or three. The smaller event sets see little change due to the initial cost of indexing and engine setup, which Mark Proctor noted in his article on these results.
2. Execution time and throughput for 1 million event set
The second results offered are based on a single large event set and rules sets that grow in size.
- 20 rules
- Throughput – 21,272 events/sec
- Time – 47 sec
- 200 rules
- Throughput – 14,925 events/sec
- Time – 67 sec
- 500 rules
- Throughput – 14,286 events/sec
- Time – 70
These are dramatic and as the rule sets scale up in size the performance scales quite well. Again, the smaller rule sets feel the effects of engine setup and indexing actions causing a standard time loss that becomes negligible as the workload increases.
We will leave the conclusions presented by the authors as an exercise for you to read, but without a doubt, JBoss BRMS CEP component provides a solid and powerful engine for processing your event streams no matter the size or rule complexity.
| Reference: | JBoss BRMS Complex Event Processing (CEP) Performance Benchmark from our JCG partner Eric Schabell at the Eric Schabell’s blog blog. |
