Enterprise Java

Head first elastic search on java with spring boot and data features

In this article I’ll try to give you an easy introduction on how to use Elastic Search in a Java project. As Spring Boot is the easiest and fastest way to begin our project I choose to use it. Futhermore, we will heavily use Repository goodies of beloved Spring Data.

Let’s begin by installing Elastic Search on our machine and run our elastic server for the first time.

I go to elastic-folder\bin and run elasticsearch.bat (yeah I’m using Windows) but no luck. I get this:
 
 
elastic-vm

“Error occurred during initialization of VM
Could not reserve enough space for object heap
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.”

What a great start!

In my bin folder there’s a “elasticsearch.in.bat” file. I set ES_MAX_MEM=1g to ES_MAX_MEM=512mb and voila it is fixed.

I start a new server without problem after that.

Now it is time to define the document we will index in elastic search. Assume we have movie information to index. Our model is quite straightforward. Movie has a name, rating and a genre in it. I chose “elastic_sample” as index name which sounds good as a database name and “movie” as type which is good for a table name if we think in relational database terms. Nothing fancy in the model as you can see.

@Document(indexName = "elastic_sample", type = "movie")
public class Movie {

    @Id
    private String id;

    private String name;

    @Field(type = FieldType.Nested)
    private List < Genre >  genre;

    private Double rating;

    public Double getRating() {
        return rating;
    }

    public void setRating(Double rating) {
        this.rating = rating;
    }

    public void setId(String id) {
        this.id = id;
    }

    public List < Genre >  getGenre() {
        return genre;
    }

    public void setGenre(List < Genre >  genre) {
        this.genre = genre;
    }

    public String getId() {
        return id;
    }

    public String getName() {
        return name;

    }

    public void setName(String name) {
        this.name = name;
    }

    @Override
    public String toString() {
        return "Movie{" +
                "id=" + id +
                ", name='" + name + '\'' +
                ", genre=" + genre +
                ", rating=" + rating +
                '}';
    }
}

For those who wonder what Genre is here is it. Just a POJO.

public class Genre {
    private String name;

    public Genre() {
    }

    public Genre(String name) {
        this.name = name;
    }

    public String getName() {
        return name;
    }

    @Override
    public String toString() {
        return "Genre{" +
                "name='" + name + '\'' +
                '}';
    }

    public void setName(String name) {
        this.name = name;
    }
}

Not it is time to create DAO layer so we can save and load our document to/from our elastic search server. Our Repository extends the classic ElasticserchRepository (no idea why it is search and not Search). As you probably know Spring Data can query one or more fields with these predefined methods where we use our field names. findByName will search in the name field, findByRating will search in the rating field so on so forth. Furthermore thanks to Spring Data we don’t need to write implementation for it, we just put method names in the interface and that’s finished.

public interface MovieRepository extends ElasticsearchRepository < Movie, Long > {
    public List < Movie >  findByName(String name);

    public List < Movie>  findByRatingBetween(Double beginning, Double end);
}

Our DAO layer will be called by a Service layer:

@Service
public class MovieService {

    @Autowired
    private MovieRepository repository;

    public List < Movie >  getByName(String name) {
        return repository.findByName(name);
    }

    public List < Movie >  getByRatingInterval(Double beginning, Double end) {
        return repository.findByRatingBetween(beginning, end);
    }

    public void addMovie(Movie movie) {
        repository.save(movie);
    }
}

Here is the main Class we will use to run our application. EnableAutoConfiguration will auto-configure everything it recognizes under our classpath. ComponentScan will scan for Spring annotations under the main Class’ directory.

@Configuration
@EnableAutoConfiguration
@ComponentScan
public class BootElastic implements CommandLineRunner {

    @Autowired
    private MovieService movieService;

    private static final Logger logger = LoggerFactory.getLogger(BootElastic.class);

// add star wars and
// princess bride as a movie
// to elastic search
    private void addSomeMovies() {
        Movie starWars = getFirstMovie();
        movieService.addMovie(starWars);

        Movie princessBride = getSecondMovie();
        movieService.addMovie(princessBride);
    }

    private Movie getSecondMovie() {
        Movie secondMovie = new Movie();
        secondMovie.setId("2");
        secondMovie.setRating(8.4d);
        secondMovie.setName("The Princess Bride");

        List < Genre >  princessPrideGenre = new ArrayList < Genre >();
        princessPrideGenre.add(new Genre("ACTION"));
        princessPrideGenre.add(new Genre("ROMANCE"));
        secondMovie.setGenre(princessPrideGenre);

        return secondMovie;
    }


    private Movie getFirstMovie() {
        Movie firstMovie = new Movie();
        firstMovie.setId("1");
        firstMovie.setRating(9.6d);
        firstMovie.setName("Star Wars");

        List < Genre >  starWarsGenre = new ArrayList < Genre >();
        starWarsGenre.add(new Genre("ACTION"));
        starWarsGenre.add(new Genre("SCI_FI"));
        firstMovie.setGenre(starWarsGenre);

        return firstMovie;
    }

    public void run(String... args) throws Exception {
        addSomeMovies();
        // We indexed star wars and pricess bride to our movie
        // listing in elastic search

        //Lets query if we have a movie with Star Wars as name
        List < Movie > starWarsNameQuery = movieService.getByName("Star Wars");
        logger.info("Content of star wars name query is {}", starWarsNameQuery);

        //Lets query if we have a movie with The Princess Bride as name
        List < Movie >  brideQuery = movieService.getByName("The Princess Bride");
        logger.info("Content of princess bride name query is {}", brideQuery);


        //Lets query if we have a movie with rating between 6 and 9
        List < Movie >  byRatingInterval = movieService.getByRatingInterval(6d, 9d);
        logger.info("Content of Rating Interval query is {}", byRatingInterval);
    }

    public static void main(String[] args) throws Exception {
        SpringApplication.run(BootElastic.class, args);
    }
}

If we run it the result is:

015-02-28 18:26:12.368  INFO 3616 --- [           main] main.BootElastic: Content of star wars name query is [Movie{id=1, name='Star Wars', genre=[Genre{name='ACTION'}, Genre{name='SCI_FI'}], rating=9.6}]
2015-02-28 18:26:12.373  INFO 3616 --- [           main] main.BootElastic: Content of princess bride name query is [Movie{id=2, name='The Princess Bride', genre=[Genre{name='ACTION'}, Genre{name='ROMANCE'}], rating=8.4}]
2015-02-28 18:26:12.384  INFO 3616 --- [           main] main.BootElastic: Content of Rating Interval query is [Movie{id=2, name='The Princess Bride', genre=[Genre{name='ACTION'}, Genre{name='ROMANCE'}], rating=8.4}]

As you can see the interval query only retrieved Princess Bride. We did not do any configuration right? It is unusual. I have to share the huge configuration file with you:

spring.data.elasticsearch.cluster-nodes=localhost:9300
 # if spring data repository support is enabled
spring.data.elasticsearch.repositories.enabled=true

Normally you would use port 9200 when you query your elastic server. But when we programmatically reach it we are using 9300. If you have more than one node you would separate them with a comma and use 9301, 9302 etc as port numbers. Our pom file is no surprise either. Just elastic starter pom and we are set to go.

<project xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://maven.apache.org/POM/4.0.0" xsi:schemalocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelversion>4.0.0</modelversion>

    <groupid>caught.co.nr</groupid>
    <artifactid>boot-elastic-sample</artifactid>
    <version>1.0-SNAPSHOT</version>
    <packaging>war</packaging>

    <!-- Inherit defaults from Spring Boot -->
    <parent>
        <groupid>org.springframework.boot</groupid>
        <artifactid>spring-boot-starter-parent</artifactid>
        <version>1.2.2.RELEASE</version>
    </parent>

    <dependencies>
        <dependency>
            <groupid>org.springframework.boot</groupid>
            <artifactid>spring-boot-starter-data-elasticsearch</artifactid>
        </dependency>

    </dependencies>

    <!-- Needed for fat jar -->
    <build>
        <plugins>
            <plugin>
                <groupid>org.springframework.boot</groupid>
                <artifactid>spring-boot-maven-plugin</artifactid>
            </plugin>
        </plugins>
    </build>
</project>

As you can see thanks to Spring Boot and Data it is quite easy to work with elastic search. Lets check what we indexed from the server api as well. I’ll use Sense -a chrome plug-in for elastic commands-.

elastic-sense-movie-search

Here’s the result json:

{
   "took": 2,
   "timed_out": false,
   "_shards": {
      "total": 1,
      "successful": 1,
      "failed": 0
   },
   "hits": {
      "total": 2,
      "max_score": 1,
      "hits": [
         {
            "_index": "elastic_sample",
            "_type": "movie",
            "_id": "1",
            "_score": 1,
            "_source": {
               "id": 1,
               "name": "Star Wars",
               "genre": [
                  {
                     "name": "ACTION"
                  },
                  {
                     "name": "SCI_FI"
                  }
               ]
            }
         },
         {
            "_index": "elastic_sample",
            "_type": "movie",
            "_id": "2",
            "_score": 1,
            "_source": {
               "id": 2,
               "name": "The Princess Bride",
               "genre": [
                  {
                     "name": "ACTION"
                  },
                  {
                     "name": "ROMANCE"
                  }
               ]
            }
         }
      ]
   }
}
  • You can check out the whole project in the github.

Sezin Karli

Mathematics Engineer & Computer Scientist with a passion for software development. Avid learner for new technologies. Currently working as Senior Software Engineer at Sahibinden.com.
Subscribe
Notify of
guest

This site uses Akismet to reduce spam. Learn how your comment data is processed.

20 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
kyriosli
kyriosli
9 years ago

great help. thanks!

Alc
Alc
9 years ago

Thanks a lot!, this was very helpful

Bhavik Shah
Bhavik Shah
9 years ago

Where is save() method in DAO layer in this project?

Sezin Karli
9 years ago
Reply to  Bhavik Shah

addMovie(Movie movie) in MovieService is where you save the data

inayat
inayat
9 years ago

i am getting

org.elasticsearch.transport.ReceiveTimeoutTransportException: [][inet[localhost/127.0.0.1:9200]][cluster/state] request_id [0] timed out after [5001ms]

error

Sezin Karli
9 years ago
Reply to  inayat

Make sure you have this in your properties

spring.data.elasticsearch.cluster-nodes=localhost:9300

You must use 9300 as port not 9200.

Furthermore, try to fetch data directly from your server with Sense plugin or some other elastic client. I doubt you’ll get something from there as well. Root cause can be a firewall issue.

Achraf
Achraf
9 years ago

the movie is indexed but is not saved on dataBase is that normal?

Sezin Karli
9 years ago
Reply to  Achraf

No it is not Achraf. Do you get some kind of exception?

Achraf
Achraf
9 years ago
Reply to  Sezin Karli

No exception, the difference is that i’m using a transport client just like that

cluster-nodes=”localhost:9300″ cluster-name=”elasticsearch” />

Vishnu Awasthi
Vishnu Awasthi
9 years ago

Nice artical, Thank you so much !!!!

Abhishek
Abhishek
8 years ago

I see that @Document,@Id and @Filed annotations are not being recognized by the IDE. What dependency do i need to add for this. Can you please help me with this.

Sezin Karli
8 years ago
Reply to  Abhishek

The pom xml I provided should be enough. I suggest that you build the project again and reload it in your ide.
If you use intellij idea, do maven projects>reimport all maven projects.

jemlifathi
jemlifathi
8 years ago

Hi, thank you for this tutorial, actually, i’m learning spring data elasticsearch and i have a very urgent use case on ES Fuzzy queries, especially on how to set the fuzziness level to be a maximum level to match a very large number of possible documents. Do you have any idea about how to set the fuzziness level to a max level?

Thank you very much.

aashii
aashii
8 years ago

I am using elastic search 2.3.1 and running the above code. It’s not working.

armandogonz
armandogonz
8 years ago

I haven’t even been able to run the above code, neither after building the project myself nor downloading your version straight from your github. One of the problems I’m facing is that my IDE doesn’t recognize any of the Elasticsearch-related imports and cannot seem to resolve any of these terms. Additionally, when I built the project myself I tried to run the command: mvn package But it returned over a dozen errors related to the POM being malformed. Can you please clear up either of these issues? I’m new to Maven projects and Spring Data, but thought this project would… Read more »

armandogonz
armandogonz
8 years ago

Update:

I was able to run this project after importing it into IntelliJ with the correct Maven settings. However, it’s failing later on during run-time primarily because the node at port 9300. Please help.

armandogonzal
armandogonzal
8 years ago

I downloaded the project from your github but it cannot seem to find the configured nodes to be available. In fact, it’s not even finding the values that reside in the application.properties file. Is there something I’m missing in the path where this file has to be enabled, or something similar?

Chris Nolan
Chris Nolan
7 years ago

Can you add code for search by Genre?
can you please post the code
Same for getting all data…i.e getAll()

MH
MH
7 years ago

With elasticsearch-5.5.1, the application fails with the following errors:

[ERROR] Failed to execute goal org.springframework.boot:spring-boot-maven-plugin:1.2.2.RELEASE:run (default-cli) on project boot-elastic-sample: An exception occured while running. null: InvocationTargetException: Failed to execute CommandLineRunner: None of the configured nodes were available: [[#transport#-1][mh-Lenovo][inet[localhost/127.0.0.1:9300]]]: [][inet[localhost/127.0.0.1:9300]][index]: [][inet[localhost/127.0.0.1:9300]] Node not connected -> [Help 1]

In the elasticsearch console I got this:

[2017-08-15T15:24:15,235][WARN ][o.e.t.n.Netty4Transport ] [7CahJCt] exception caught on transport layer [[id: 0xcf7cc05d, L:/127.0.0.1:9300 – R:/127.0.0.1:48994]], closing connection
java.lang.IllegalStateException: Received message from unsupported version: [1.3.2] minimal compatible version is: [5.0.0]
at org.elasticsearch.transport.TcpTransport.messageReceived(TcpTransport.java:1379) ~[elasticsearch-5.5.1.jar:5.5.1]
at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:74) ~[transport-netty4-5.5.1.jar:5.5.1]

Looks like spring-boot-starter-data-elasticsearch is not compatible with latest version of elasticsearch.

Arun A
Arun A
7 years ago

Hi,

I use spring-boot-starter-data-elasticsearch dependency in my Java spring project to query ES 2.4.0

My json’s look like this,

{
“event”:{
“eventName”:”Man U vs Arsenal”,
“eventDateTime”:”Mon 24 Oct 12:00″
}
}
To avoid having to declare an empty “event” POJO wrapper, I use @JsonRootName(value=”event”) to mark that there’s a root element in my POJO and I have also specified the below property in my application.properties,
spring.jackson.deserialization.UNWRAP_ROOT_VALUE=true

The search is happening fine but the root element declaration does not take effect and results in objects with just nulls.

Is there a way to achieve this at all?

Thanks,
Arun

Back to top button