In the past years, we’ve often heard the request (both from community and our customers) on how to swap the persistence logic of Activiti from relational database to something else. When we announced Activiti 6, one of the promises we made was that we would make exactly this possible.
People that have dived into the code of the Activiti engine will know that this is a serious refactoring, as the persistence code is tightly coupled with the regular logic. Basically, in Activiti v5, there were:
- Entity classes: these contain the data from the database. Typically one database row is one Entity instance
- EntityManager: these classes group operations related to entities (find, delete,… methods)
- DbSqlSession: low-level operations (CRUD) using MyBatis. Also contains command-duration caches and manages the flush of the data to the database.
The problems in version 5 were the following:
- No interfaces. Everything is a class, so replacing logic gets really hard.
- The low-level DbSqlSession was used everywhere across the code base.
- a lot of the logic for entities was contained within the entity classes. For example look at the TaskEntity complete method. You don’t have to be an Activiti expert to understand that this is not a nice design:
- It fires an event
- it involves users
- It calls a method to delete the task
- It continues the process instance by calling signal
Now don’t get me wrong. The v5 code has brought us very far and powers many awesome stuff all over the world. But when it comes to swapping out the persistence layer … it isn’t something to be proud of.
And surely, we could hack our way in into the version 5 code (for example by swapping out the DbSqlSession with something custom that responds to the methods/query names being used there), but it still would be not quite nice design-wise and quite relational-database-like. And that doesn’t necessarily match the data store technology you might using.
No, for version 6 we wanted to do it properly. And oh boy … we knew it was going to be a lot of work … but it was even more work than we could imagine (just look at the commits on the v6 branch for the last couple of weeks). But we made it … and the end result is just beautiful (I’m biased, true). So let’s look at the new architecture in v6 (forgive me my powerpoint pictures. I’m a coder not a designer!):
So, where in v5 there were no interfaces, there are interfaces everywhere in v6. The structure above is applied for all of the Entity types in the engine (currently around 25). So for example for the TaskEntity, there is a TaskEntityImpl, a TaskEntityManager, a TaskEntityManagerImpl, a TaskDataManager and a TaskDataManagerImpl class (and yes I know, they still need javadoc). The same applies for all entities.
Let me explain the diagram above:
- EntityManager: this is the interface that all the other code talks to whenever it comes to anything around data. It is the only entrypoint when it comes to data for a specific entity type.
- EntityManagerImpl: implementation of the EntityManager class.The operations are often high level and do multiple things at the same time. For example an Execution delete might also delete tasks, jobs, identityLinks, etc and fire relevant events. Every EntityManager implementation has a DataManager. Whenever it needs data from the persistence store, it uses this DataManager instance to get or write the relevant data.
- DataManager: this interface contains the ‘low level’ operations. Typically contains CRUD methods for the Entity type it manages and specific find methods when data for a particular use case is needed
- DataManagerImpl: implementation of the DataManager interface. Contains the actual persistence code. In v6, this is the only class that now uses the DbSqlSession classes to communicate with the database using MyBatis. This is typically the class you will want to swap out.
- Entity: interface for the data. Contains only getters and setters.
- EntityImpl: implementation of the above interface. In Activiti v6, this is a regular pojo, but the interface allows you to switch to different technologies such as Neo4 with spring-dataj, JPA, … (which use annotations). Without it, you would need to wrap/unwrap the entities if the default implementation would not work on your persistence technology.
Consolidation
Moving all of the operations into interfaces gave us a clear overview of what methods were spread across the codebase. Did you know for example there were at least five different methods to delete an Execution (named ‘delete’, ‘remove’, ‘destroy’, etc…)? They did almost the same, but with subtle differences. Or sometimes not subtle at all.
A lot of the work over the past weeks included consolidating all of this logic into one method. Now, in the current codebase, there is but one way to do something. Which is quite important for people that want to use different persistence technologies. Making them implement all varieties and subtleties would be madness.
In-memory implementation
To prove the pluggability of the persistence layer, I’ve made a small ‘in-memory’ prototype. Meaning that, instead of a relational database, we use plain old HashMaps to store our entities as {entityId, entities}. The queries then become if-clauses.
- The code can be found on Github: https://github.com/jbarrez/activiti-in-mem-prototype
(people have sometimes on the forum asked how hard it was to run Activiti purely in memory, for simple use cases that don’t mandate the use of a database. Well, now it’s not hard at all anymore! Who knows … this little prototype might become something if people like it!)
- As expected, we swap out the DataManager implementations with our in-memory version, see InMemoryProcessEngineConfiguration
@Override
protected void initDataManagers() {
this.deploymentDataManager = new InMemoryDeploymentDataManager(this);
this.resourceDataManager = new InMemoryResourceDataManager(this);
this.processDefinitionDataManager = new InMemoryProcessDefinitionDataManager(this);
this.jobDataManager = new InMemoryJobDataManager(this);
this.executionDataManager = new InMemoryExecutionDataManager(this);
this.historicProcessInstanceDataManager = new InMemoryHistoricProcessInstanceDataManager(this);
this.historicActivityInstanceDataManager = new InMemoryHistoricActivityInstanceDataManager(this);
this.taskDataManager = new InMemoryTaskDataManager(this);
this.historicTaskInstanceDataManager = new InMemoryHistoricTaskInstanceDataManager(this);
this.identityLinkDataManager = new InMemoryIdentityLinkDataManager(this);
this.variableInstanceDataManager = new InMemoryVariableInstanceDataManager(this);
this.eventSubscriptionDataManager = new InMemoryEventSubscriptionDataManager(this);
}Such DataManager implementations are quite simple. See for example the InMemoryTaskDataManager who needs to implement the data retrieval/write methods for a TaskEntity:
public List<TaskEntity> findTasksByExecutionId(String executionId) {
List<TaskEntity> results = new ArrayList<TaskEntity>();
for (TaskEntity taskEntity : entities.values()) {
if (taskEntity.getExecutionId() != null && taskEntity.getExecutionId().equals(executionId)) {
results.add(taskEntity);
}
}
return results;
}To prove it works, let’s deploy, start a simple process instance, do a little task query and check some history. This code is exactly the same as the ‘regular’ Activiti usage.
public class Main {
public static void main(String[] args) {
InMemoryProcessEngineConfiguration config = new InMemoryProcessEngineConfiguration();
ProcessEngine processEngine = config.buildProcessEngine();
RepositoryService repositoryService = processEngine.getRepositoryService();
RuntimeService runtimeService = processEngine.getRuntimeService();
TaskService taskService = processEngine.getTaskService();
HistoryService historyService = processEngine.getHistoryService();
Deployment deployment = repositoryService.createDeployment().addClasspathResource("oneTaskProcess.bpmn20.xml").deploy();
System.out.println("Process deployed! Deployment id is " + deployment.getId());
ProcessInstance processInstance = runtimeService.startProcessInstanceByKey("oneTaskProcess");
List<Task> tasks = taskService.createTaskQuery().processInstanceId(processInstance.getId()).list();
System.out.println("Got " + tasks.size() + " tasks!");
taskService.complete(tasks.get(0).getId());
System.out.println("Number of process instances = " + historyService.createHistoricProcessInstanceQuery().count());
System.out.println("Number of active process instances = " + historyService.createHistoricProcessInstanceQuery().finished().count());
System.out.println("Number of finished process instances = " + historyService.createHistoricProcessInstanceQuery().unfinished().count());
}
}Which, if you run it gives you this (blazingly fast as it’s all in memory!):
Process deployed! Deployment id is 27073df8-5d54-11e5-973b-a8206642f7c5
Got 1 tasks!
Number of process instances = 1
Number of active process instances = 0
Number of finished process instances = 1
In this prototype, I didn’t add transactional semantics. Meaning that if two users would complete the same user task at the very same time the outcome would be indeterminable. It’s of course possible to have in-memory transaction-like logic which you expect from the Activiti API, but I didn’t implement that yet. Basically you would need to keep all objects you touch in a little cache until flush/commit time and do some locking/synchronisation at that point. And of course, I do accept pull requests :)
What’s next?
Well, that’s pretty much up to you. Let us know what you think about it, try it out!
We’re in close contact with one of our community members/customers who plan to try it out very soon. But we want to play with it ourselves too of course and we’re looking at what would be a cool first choice (I myself still have a special place in my heart for Neo4j … which would be a great fit as it’s transactional).
But the most important bit is: in Activiti v6 it is now possible to cleanly swap out the persistence layer. We’re very proud of how it looks now. And we hope you like it too!
| Reference: | Pluggable persistence in Activiti 6 from our JCG partner Joram Barrez at the Small steps with big feet blog. |
