Getting Started With Gradle: Integration Testing

Petri KainulainenMay 5th, 2015Last Updated: May 5th, 2015

1 107 7 minutes read

Because the standard project layout of a Java project defines only one test directory (src/test), we have no standard way to add integration tests to our Gradle build.

If we want to use the standard project layout, we can add integration tests to our Gradle build by using one of the following options:

We can add our integration tests to the same directory than our unit tests. This is an awful idea because integration tests are typically a lot slower than unit tests. If we decide to use this approach, the length of our feedback loop is a lot longer than it should be.
We can create a new project and add our integration tests to that project. This makes no sense because it forces us to transform our project into a multi-project build. Also, if our project is already a multi-project build, we are screwed. We can of course add all integration tests to the same project or create new integration test project for each tested project, but it would be less painful to shoot ourselves in the foot.

It is clear that we need a better way. This blog post describes how we create a Gradle build that fulfils the following requirements:

Integration and unit tests must have different source directories. The src/integration-test/java directory must contain the source code of our integration tests and the src/test/java directory must contain the source code of our unit tests.
Integration and unit tests must have separate resource directories. The src/integration-test/resources directory must contain the resources of our integration tests. The src/test/resources directory must contain the resources of our unit tests.
We must be able to configure compile time and runtime dependencies for our integration tests.
We must be able to run either our unit tests or integration tests.
We must be able to run all tests.
If an integration test fails, our build must fail as well.
Integration and unit tests must have separate HTML reports.

Let’s start by configuring the source and resource directories of our integration tests.

Configuring the Source and Resource Directories of Our Integration Tests

We can add new source and resource directories to our Gradle build by using the sourceSets build script block. Armed with this information, we can configure the source and resource directories of our integration tests by following these steps:

Create a new source set called integrationTest.
Ensure that the output of the main and test source sets is added to the compile time classpath.
Ensure that the output of the main and test source sets is added to the runtime classpath.
Set the source directory of our integration tests to src/integration-test/java.
Set the resource directory of our integration tests to src/integration-test/resources.

When we are done, our build.gradle file should have the following sourceSets build script block right after the repositories build script block:

sourceSets {
    integrationTest {
        java {
            compileClasspath += main.output + test.output
            runtimeClasspath += main.output + test.output
            srcDir file('src/integration-test/java')
        }
        resources.srcDir file('src/integration-test/resources')
    }
}

Additional Reading:
Section 23.2 Java Plugin – Source sets of Gradle User Guide
The DSL Reference of the sourceSets build script block
The DSL Reference of the SourceSetOutput

When we run the command: gradle properties at the command prompt, we will see a long list of the project’s properties. The properties that are relevant for this blog posts are shown in the following:

> gradle properties
:properties

------------------------------------------------------------
Root project
------------------------------------------------------------
configurations: [configuration ':archives', configuration ':compile', configuration ':default', configuration ':integrationTestCompile', configuration ':integrationTestRuntime', configuration ':runtime', configuration ':testCompile', configuration ':testRuntime']

sourceSets: 1
sources: [Java source 'main:java', JVM resources 'main:resources', Java source 'test:java', JVM resources 'test:resources', Java source 'integrationTest:java', JVM resources 'integrationTest:resources']

BUILD SUCCESSFUL

Total time: 3.34 secs

As we can see, we added a new source and resource directories to our Gradle build. The interesting this is that when we created a new source set, the Java plugin added two new dependency configurations to our build:

The integrationTestCompile configuration is used to declare the dependencies that are required when our integration tests are compiled.
The integrationTestRuntime configuration is used to declare the dependencies that are required to run our integration tests. This configuration contains all dependencies that are added to the integrationTestCompile configuration.

Additional Reading:
Section 23.5 Java Plugin – Dependency Management of Gradle User Guide

Let’s move and find out what kind of configuration changes we have to make before these dependency configurations are useful to us.

Configuring the Dependency Configurations of Our Integration Tests

When we configured the source and resource directories of our integration tests, we created a source set that created two new dependency configurations: integrationTestCompile and integrationTestRuntime. The problem is that these configurations do not contain the dependencies of our unit tests.

We could solve this problem by adding the required dependencies to these configurations, but we won’t do that because adding duplicate configuration is an awful idea. Instead we will configure these dependency configurations by following these steps:

Ensure that the integrationTestCompile configuration contains the dependencies that are required to compile our unit tests.
Ensure that the integrationTestRuntime configuration contains the dependencies that are required to run our unit tests.

We can make these changes by using the configurations build script block. In other words, we must add the following code to our build.gradle file between the sourceSets and the dependencies build script blocks:

configurations {
    integrationTestCompile.extendsFrom testCompile
    integrationTestRuntime.extendsFrom testRuntime
}

Additional Reading:
Section 8.3 Dependency configurations of Gradle User Guide
The DSL Reference of the configurations build script block
The DSL Reference of the ConfigurationContainer
The DSL Reference of the Configuration

We can now add dependencies to these configurations. For example, if we want to use AssertJ 3.0 in our integration tests, we have to add the assertj-core dependency to the integrationTestCompile configuration. After we have done this, the dependencies build script block found from our build.gradle file looks as follows:

dependencies {
    compile 'log4j:log4j:1.2.17'
    testCompile 'junit:junit:4.11'
    integrationTestCompile 'org.assertj:assertj-core:3.0.0'
}

Additional Reading:
Getting Started With Gradle: Dependency Management

Our next step is to create the task that runs our integration tests. Let’s find out how we can do that.

Creating the Task That Runs Our Integration Tests

We can create the task that runs our integration tests by following these steps:

Create a new task called integrationTest and set its type to Test.
Configure the location of the compiled test classes.
Configure the classpath that is used when our integration tests are run.

We can create and configure the integrationTest task by adding the following code to our build.gradle file:

task integrationTest(type: Test) {
    testClassesDir = sourceSets.integrationTest.output.classesDir
    classpath = sourceSets.integrationTest.runtimeClasspath
}

Additional Reading:
Section 15.1 Defining tasks of Gradle User Guide
The DSL Reference of Task

We have created the task that runs our integration tests, but the problem is this task is not invoked during our build. Because want to include it in our build, we have to follow these steps:

Ensure that our integration tests are run before the check task and that the check task fails the build if there are failing integration tests.
Ensure that our unit tests are run before our integration tests. This guarantees that our unit tests are run even if our integration tests fails.

We can do these configuration changes by adding the following lines to our build.gradle file:

check.dependsOn integrationTest
integrationTest.mustRunAfter test

Additional Reading:
Section 23.3 Java Plugin – Tasks of Gradle User Guide
Section 15.4 Adding dependencies to a task of Gradle User Guide
Section 15.5 Order tasks of Gradle User Guide

We are almost done, but there is still one problem left. Our unit and integration tests create their HTML reports to the same report directory. This means that if we run both unit and integration tests, we can see only the HTML report that contains the test results of our integration tests.

We can ensure that the HTML reports of unit and integration tests are created to different report directories by adding the following snippet to our build.gradle file:

tasks.withType(Test) {
    reports.html.destination = file("${reporting.baseDir}/${name}")
}

This is not my own idea. I borrowed it from this Stackoverflow answer that explains how you can create separate HTML reports for integration and unit tests.

After we have added this snippet to our build.gradle file, our unit and integration tests use their own report directories that are described in the following:

The build/reports/integrationTest directory contains the HTML report that contains the test results of our integration tests.
The build/reports/test directory contains the HTML report that contains the test results of our unit tests.

We are done! Let’s move on and find out how we can run our tests.

Running Our Tests

We have now created a new task that runs our integration tests and integrated that task with our Gradle build. We are finally ready to run our unit and integration tests. The requirements of our Gradle build states that:

We must be able to run our only unit tests.
We must be able to run only integration tests.
We must be able to run all tests.

Let’s go through these requirements one by one.

First, if we want to run only unit tests, we can use one of these two options:

We can run our unit tests by running the command: gradle clean test at the command prompt.
We can run our build and exclude integration tests by running the command: gradle clean build -x integrationTest at the command prompt.

Second, if we want to run only integration tests, we can choose one of the following options:

We can run our integration tests by running the command: gradle clean integrationTest -x test at the command prompt.
We can run our build and exclude unit tests by running the command: gradle clean build -x test at the command prompt.

Third, if we want to run all tests, we can choose one of these two options:

We can run unit and integration tests by running the command: gradle clean integrationTest at the command prompt.
We can run our build by running the command: gradle clean build at the command prompt.

Additional Reading:
Section 11.2 Excluding tasks of Gradle User User Guide

Let’s summarize what we learned from this blog post.

Summary

This blog post has taught us the following things:

If we add a new source set to our build, the Java plugin creates the compile time and runtime dependency configurations for it.
We can include the dependencies of an another dependency configuration by using the extendsFrom property of the Configuration.
We can create a task that run our integration tests by creating a new Test task, and configuring the location of the integration test classes and the used classpath.
We can add dependencies to a task and configure the order in which our tasks are invoked.
We can exclude tasks by using the -x command-line option.