Processing 10 million messages with Akka

Munish GuptaMay 24th, 2012Last Updated: October 22nd, 2012

1 112 2 minutes read

Akka Actors promise concurrency. What better way to simulate that and see if how much time it takes to process 10 million messages using commodity hardware and software without any low level tunings.I wrote the entire 10 million messages processing in Java and the overall results astonished me.

When I ran the program on my iMac machine with an intel i5 – 4 core, 4 Gb RAM machine and JVM heap at 1024Mb, the program processed 10 million machines in 23 secs. I ran the program multiple times and the average time was in the range of 25 secs. So the through put I received was almost in the range of 400K messages per second which is phenomenal.

The below picture explains the flow used to simulate the load generation scenario.

Caveat: Each message sends a response after 1 second which is not the right simulation for a real world scenario. The message processing will consume some resources on the heap and gc activity which are not accounted for, in this scenario.

The program uses overall direction from the post Akka actors : 10 millions messages processed (1s / message) in 75 seconds ! although without any messages throttling.

The code base for the program is available at the following location – https://github.com/write2munish/Akka-Essentials

The ApplicationManagerSystem creates the actors and pumps in the traffic to the WorkerActor

private ActorSystem system;
private final ActorRef router;
private final static int no_of_msgs = 10 * 1000000;
 
public ApplicationManagerSystem() {
 
 final int no_of_workers = 10;
 
 system = ActorSystem.create('LoadGeneratorApp');
 
 final ActorRef appManager = system.actorOf(
   new Props(new UntypedActorFactory() {
    public UntypedActor create() {
     return new JobControllerActor(no_of_msgs);
    }
   }), 'jobController');
 
 router = system.actorOf(new Props(new UntypedActorFactory() {
  public UntypedActor create() {
   return new WorkerActor(appManager);
  }
 }).withRouter(new RoundRobinRouter(no_of_workers)));
}
 
private void generateLoad() {
 for (int i = no_of_msgs; i >= 0; i--) {
  router.tell('Job Id ' + i + '# send');
 }
 System.out.println('All jobs sent successfully');
}

Once the messages are received by the WorkerActor, the responses are scheduled to be send after 1000 milli secs

public class WorkerActor extends UntypedActor {
 
 private ActorRef jobController;
 
 @Override
 public void onReceive(Object message) throws Exception {
   using scheduler to send the reply after 1000 milliseconds
  getContext()
    .system()
    .scheduler()
    .scheduleOnce(Duration.create(1000, TimeUnit.MILLISECONDS),
      jobController, 'Done');
 }
 
 public WorkerActor(ActorRef inJobController) {
  jobController = inJobController;
 }
}

The response messages from the WorkerActor are send across to the JobControllerActor, that collects all the responses.

public class JobControllerActor extends UntypedActor {
 
 int count = 0;
 long startedTime = System.currentTimeMillis();
 int no_of_msgs = 0;
 
 @Override
 public void onReceive(Object message) throws Exception {
 
  if (message instanceof String) {
   if (((String) message).compareTo('Done') == 0) {
    count++;
    if (count == no_of_msgs) {
     long now = System.currentTimeMillis();
     System.out.println('All messages processed in '
       + (now - startedTime)  1000 + ' seconds');
 
     System.out.println('Total Number of messages processed '
       + count);
     getContext().system().shutdown();
    }
   }
  }
 
 }
}