Systems integration has been a tough problem from the beginning of the use of software systems in the enterprises. This had taken many faces in the last coupe of decades and is still evolving as other factors of the computing.
At the very beginning people used to design heterogeneous monolithic systems to automate certain parts of their business and the integration back then was mostly a manual process where there was a lot of human intervention in the process. This came in the form of data entry from one system to the other or the use of data export and import. With the invention of the internet people came up with point-to-point connections where each system has to be modified to be integrated with the other system. These interactions were mostly file based however more serious integration tasks used the queue based messaging which has provided reliability over the file based integrations approach.
Then came the web era and people wanted to expose at least a tiny piece of their systems to be available over the web introducing public user interaction into the picture. Which resulted in the introduction of J2EE like technologies to facilitate most of these systems to be exposed over the web and the systems tend to be following the client-server architecture. However the systems integration at the server side was still having the same set of problems.
The next step was a major step towards successful integration of the systems. With the advancement of distributed computing, the concept of services came into the picture and a new paradigm was introduced. This is named the “Service Oriented Architecture“, commonly known as SOA. While this software paradigm theoretically solved most of the issues, the legacy systems that have been in use was causing problems to bottom up implementation of the SOA. While these services (mostly called web-services) are then connected to a hub like architecture, it also tried to resolve the gap between the web-services and the legacy systems by doing protocol conversion and format conversion. This is the “Enterprise Service Bus” (commonly known as ESB) which is still being used to solve the integration issues in between systems. It provides the location transparency and many other quality of service improvements to the integration. We (AdroitLogic) has a product offering for this which is named the UltraESB.
Even though the web services technologies seem to give a promising solution, the weight associated with the SOAP infoset based on XML used by web services and the WS-* stack lead people to think of alternative solutions. REST had later used as a framework for developing a major share of services and the JAX-RS like specifications helped its way for the REST to reach the enterprise over protocols like JSON, TEXT and POX, though there were constraints of this technology such as the transport used and lack of application level security and reliability. The technologies such as Protocol Buffers, Thrift, Hessian, AS2 and other forms are also used within specific domains and they had been standardized. While AdroitLogic has a separate product offering for the AS2 communication named the AS2 Gateway the rest of the technologies are generically supported by the UltraESB.
The same way enterprises wanted a piece of their system functionalities to be exposed as web form front ends to the users, now the enterprise are interested in sharing some of their services over the internet. This brought the concerns of security to these exposed services. These are identified as APIs that an organization wants to expose out with security, governance and many other centrally available features. API Management solutions addressed this and the product that we came up with for this domain is API Director.
We are in the phase of introduction of an another new software paradigm and transition of the services integration, which is known as Micro Services. On a separate note cloud computing was something that evolved hand-in-hand with systems integration and the services exposure and in the mean time the basic component of cloud computing the virtualization has taken a revolutionary step to provide process oriented containers emulating an O/S yet utilizes the resources more effectively compared to a virtual machine. The integration had to evolve to facilitate this movement and the requirement of a container driven lean, distributed, highly scalable solution with the ability to easily provision new integration flows has to invented. This concept spans from integration of services, APIs and partners to front-end, users and devices including mobile devices, and is named the “Integration Platform”
Yes, we are now working on getting a product offering out to address this, which runs on top of a set of Docker containers, utilizing the Kubernetes container management platform.
Stay tuned to hear more about this amazing piece of software that will change the way you look at the problem of systems integration!!
| Reference: | Evolution of Systems Integration from our JCG partner Ruwan Linton at the Blind Vision – of Software Engineering and Life blog. |
