IMDG or In memory data grid is not an in-memory relational database, a NOSQL database or a relational database. It is a different breed of software datastore. The data model is distributed across many servers in a single location or across multiple locations. This distribution is known as a data fabric. This distributed model is known as a ‘shared nothing’ architecture. IMDG has following characteristics:
- All servers can be active in each site.
- All data is stored in the RAM of the servers.
- Servers can be added or removed non-disruptively, to increase the amount of RAM available.
- The data model is non-relational and is object-based.
- Distributed applications written on the platform independent language.
- The data fabric is resilient, allowing non-disruptive automated detection and recovery of a single server or multiple servers.
Most of all time we uses IMDG for web session management of Application server and as a distributed cache or L2 cache. Hazelcast community addition was our all time favourite IMDG tools, but from the last few realses of hazelcast community edition, it’s performance not happy us at all. As a quick alternative of HazelCast, we decided to make a try with
Apache ignite. This post is dedicated to apache ignite and be used for quick startup guide. For installation i will used 2 virtual machine of Redhat operating system with following configurations:
- CPU: 2
- RAM: 4
- HDD: 25 GB
- OS: Redhat Santiago
From a lot of features of Apache ignite6 we will only examines following features:
- Prepare operating system
- Using Spring for using DataGrid
- MyBatis Cache configuration
- Spring Caching
Instaling apache ignite
Pre require-ties:
- Java 1.7 and above
- open ports: 47500..47509, 8080 (for Rest interface), 47400, 47100:47101, 48100:48101, 31100:31101After installing JDK in operating system, we have to open ports that mentioned above. By following commands we can manipulates iptables.
vi /etc/sysconfig/iptables -A INPUT -m state --state NEW -m tcp -p tcp --dport 47500:47509 -j ACCEPT -A INPUT -m state --state NEW -m tcp -p tcp --dport 47400 -j ACCEPT -A INPUT -m state --state NEW -m tcp -p tcp --dport 47100 -j ACCEPT -A INPUT -m state --state NEW -m tcp -p tcp --dport 47101 -j ACCEPT -A INPUT -m state --state NEW -m tcp -p tcp --dport 48100 -j ACCEPT -A INPUT -m state --state NEW -m tcp -p tcp --dport 48101 -j ACCEPT -A INPUT -m state --state NEW -m tcp -p tcp --dport 31100 -j ACCEPT -A INPUT -m state --state NEW -m tcp -p tcp --dport 31101 -j ACCEPT /etc/init.d/iptables restart
Installation of Apache ignite in few machines
- Lets download the ignite 1.5.0final version from the following links.
- Unzip the archive any where in os such as /opt/apache-ignite
- Add environmental path IGNITE_HOME to the home directory of the apache ignite.
- copy the folder $IGNITE_HOME/libs/optional/ignite-rest-http to /home/user/apache-ignite-fabric-1.5.0/libs, it will enable the apache ignite through rest interface.
- Run the command ignite.sh examples/config/example-cache.xml to start the apache ignite.
If every thing goes fine you should see the following log in your console:
[12:32:01] Ignite node started OK (id=ceb614ca) [12:32:01] Topology snapshot [ver=4, servers=2, clients=0, CPUs=3, heap=2.0GB]
and ignite will also available through http by URL http://host:port/ignite?cmd=version
Using Spring for using DataGrid
First of all we have to build maven project to write down bunch of code to examine features of apache Ignite.
- Add the following dependencies to the pom.xml
<dependency> <groupId>org.apache.ignite</groupId> <artifactId>ignite-core</artifactId> <version>${ignite.version}</version> </dependency> <dependency> <groupId>org.apache.ignite</groupId> <artifactId>ignite-spring</artifactId> <version>${ignite.version}</version> </dependency> <dependency> <groupId>org.apache.ignite</groupId> <artifactId>ignite-indexing</artifactId> <version>${ignite.version}</version> </dependency> <!-- myBatis --> <dependency> <groupId>org.mybatis.caches</groupId> <artifactId>mybatis-ignite</artifactId> <version>1.0.0-beta1</version> </dependency> <dependency> <groupId>org.mybatis</groupId> <artifactId>mybatis-spring</artifactId> <version>1.2.4</version> </dependency> <dependency> <groupId>org.mybatis</groupId> <artifactId>mybatis</artifactId> <version>3.3.1</version> </dependency> <!-- Oracle 12--> <dependency> <groupId>com.oracle</groupId> <artifactId>ojdbc6</artifactId> <version>11.2.0.3</version> </dependency>Please, note that Oracle JDBC client jar should be in local maven repositories. In my case i use Oracle 11.2.02 client.
- Add spring-context.xml file in resources directory with following contexts:
<beans xmlns="http://www.springframework.org/schema/beans" xmlns:context="http://www.springframework.org/schema/context" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:mvc="http://www.springframework.org/schema/mvc" xmlns:cache="http://www.springframework.org/schema/cache" xsi:schemaLocation=" http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd http://www.springframework.org/schema/cache http://www.springframework.org/schema/cache/spring-cache-3.1.xsd http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd "> <!-- Enable annotation-driven caching. --> <cache:annotation-driven/> <context:property-placeholder location="classpath:jdbc.properties"/> <!-- beans --> <bean id="ignite.cfg" class="org.apache.ignite.configuration.IgniteConfiguration"> <property name="gridName" value="TestGrid"/> <!-- Enable client mode. --> <property name="clientMode" value="true"/> <property name="cacheConfiguration"> <list> <!-- Partitioned cache example configuration (Atomic mode). --> <bean class="org.apache.ignite.configuration.CacheConfiguration"> <!--<property name="atomicityMode" value="ATOMIC"/>--> <!-- Set cache mode. --> <property name="cacheMode" value="PARTITIONED"/> <property name="backups" value="1"/> <property name="statisticsEnabled" value="true" /> </bean> </list> </property> <!-- Explicitly configure TCP discovery SPI to provide list of initial nodes. --> <property name="discoverySpi"> <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi"> <property name="ipFinder"> <!-- Uncomment static IP finder to enable static-based discovery of initial nodes. --> <!--<bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">--> <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.multicast.TcpDiscoveryMulticastIpFinder"> <property name="addresses"> <list> <!-- In distributed environment, replace with actual host IP address. --> <value>Add your node ip address</value> <value>add your node ip address</value> </list> </property> </bean> </property> </bean> </property> </bean> <bean id="sqlSessionFactory" class="org.mybatis.spring.SqlSessionFactoryBean"> <property name="dataSource" ref="dataSource" /> <property name="mapperLocations" value="classpath*:com/blu/ignite/dao/*Mapper.xml"/> </bean> <bean id="dataSource" class="oracle.jdbc.pool.OracleDataSource" destroy-method="close"> <property name="URL" value="${jdbc.url}" /> <property name="user" value="${jdbc.username}"/> <property name="password" value="${jdbc.password}"/> <property name="connectionCachingEnabled" value="true"/> </bean> </beans>Lets examine a few configuration properties:
- property name=”clientMode” value=”true” – this property will force the current application to run as client.
- property name=”cacheMode” value=”PARTITIONED” – cache mode will be partitioned, cache mode can be replicated also.
- property name=”backups” value=”1″ – always there will be one redundant element of cache in another node.
- property name=”statisticsEnabled” value=”true” – this property will activate the cache statistics.
- now lets write some:
public class SpringIgniteRun { public static void main(String[] args) throws Exception{ System.out.println("Run Spring example!!"); ApplicationContext ctx = new ClassPathXmlApplicationContext("spring-core.xml"); IgniteConfiguration igniteConfiguration = (IgniteConfiguration) ctx.getBean("ignite.cfg"); Ignite ignite = Ignition.start(igniteConfiguration); // get or create cache IgniteCache cache = ignite.getOrCreateCache("myCacheName"); for(int i = 1; i < 1000; i++){ cache.put(i, Integer.toString(i)); } for(int i =1; i<1000;i++){ System.out.println("Cache get:"+ cache.get(i)); } Thread.sleep(20000); // sleep for 20 seconds // statistics System.out.println("Cache Hits:"+ cache.metrics(ignite.cluster()).getCacheHits()); ignite.close(); } }above code is self explained, we just create a cache named “myCacheName” and add 1000 String value of Integer. After inserting the value to cache, we also read the elements from cache and check the statistics. through ignitevisorcmd you can also monitor the data grid, follows you can find screen shot of the statistics of the grid
MyBatis Cache configuration
Now lets add MyBatis ORM l2 cache and examine how it’s works.
<bean id="servicesBean" class="com.blu.ignite.WebServices">
<property name="dao" ref="userServicesBean"/>
</bean>
<bean id="userServicesBean" class="com.blu.ignite.dao.UserServices">
<property name="userMapper" ref="userMapper"/>
</bean>
<bean id="sqlSessionFactory" class="org.mybatis.spring.SqlSessionFactoryBean">
<property name="dataSource" ref="dataSource" />
<property name="mapperLocations" value="classpath*:com/blu/ignite/dao/*Mapper.xml"/>
</bean>
<bean id="dataSource" class="oracle.jdbc.pool.OracleDataSource" destroy-method="close">
<property name="URL" value="${jdbc.url}" />
<property name="user" value="${jdbc.username}"/>
<property name="password" value="${jdbc.password}"/>
<property name="connectionCachingEnabled" value="true"/>
</bean>
<bean id="userMapper" autowire="byName" class="org.mybatis.spring.mapper.MapperFactoryBean">
<property name="mapperInterface" value="com.blu.ignite.mapper.UserMapper" />
<property name="sqlSessionFactory" ref="sqlSessionFactory" />
</bean>
<bean class="org.mybatis.spring.mapper.MapperScannerConfigurer">
<property name="basePackage" value="com.blu.ignite.mapper" />
</bean>We add SQLsessionFactory, MyBatis mapper and Service Bean. Now lets add the *.Mapper.xml
<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE mapper
PUBLIC "-//mybatis.org//DTD Mapper 3.0//EN" "http://mybatis.org/dtd/mybatis-3-mapper.dtd">
<mapper namespace="com.blu.ignite.mapper.UserMapper">
<cache type="org.mybatis.caches.ignite.IgniteCacheAdapter" />
<select id="getUser" parameterType="String" resultType="com.blu.ignite.dto.User" useCache="true">
SELECT * FROM users WHERE id = #{id}
</select>
<select id="getUniqueJob" parameterType="String" resultType="String" useCache="false">
select unique job from emp order by job desc
</select>
</mapper>Full sql (DDL/DML) scripts of emp and dept tables will be find in the directory com/blu/ignite/scripts I have created a simple web service to get the users and the unique jobs for employees. Here is the code for the web service as follows:
@WebService(name = "BusinessRulesServices",
serviceName="BusinessRulesServices",
targetNamespace = "http://com.blu.rules/services")
public class WebServices {
private UserServices userServices;
@WebMethod(operationName = "getUserName")
public String getUserName(String userId){
User user = userServices.getUser(userId);
return user.getuName();
}
@WebMethod(operationName = "getUniqueJobs")
public List getUniqueJobs(){
return userServices.getUniqueJobs();
}
@WebMethod(exclude = true)
public void setDao(UserServices userServices){
this.userServices = userServices;
}
}Invoke of the web method getUserName will query the database and cache the query result in ignite cache.
Spring Caching
With spring caching you could achieve caching of the return value of any spring bean method. Apache ignite will create the cache by the name of the cache which you will provide by the annotation @Cacheable(“returnHello”) For example, if i have a such method as follows:
@Cacheable("returnHello")
public String sayhello(String str){
System.out.println("Client says:"+ str);
return "hello"+str;
}The first time when the method will be invoked, a replicated cache with argument name will create in ignite, next time every invocation of the above method will return the value from the cache.
- For now it’s enough. Soon i will return back with some new features of apache ignite. Full source code of the project will found in the github.
| Reference: | Quick start with In memory Data Grid, Apache Ignite from our JCG partner Shamim Bhuiyan at the My workspace blog. |
Nice post, Shamim. Users on IT Central Station also often compare Apache Ignite (formerly known as GridGain) to Oracle Coherence. Your readers might find a direct comparison between these two solutions from real users to be helpful: https://goo.gl/8KSR9M.
Good, I finally get the ideal about Ignite in data grid. It is really good. Could you write some service grid?