TestNG or JUnit

Continuing the same battle, yesterday I started to look into Spring’s testing support. It is also built on top of JUnit. However, in a few minutes of using the same, I was searching for a feature in JUnit that I have always found missing. TestNG provides Parameterized Testing using DataProviders. Given that I was once again asking myself a familiar question – TestNG or JUnit, I decided to document this so that next time I am sure which one and why.

Essentially the same

If you are just going to do some basic Unit Testing, both the frameworks are basically the same. Both the frameworks allow you to test the code in a quick and effective manner. They have had tool support in Eclipse and other IDE. They have also had support in the build frameworks like Ant and Maven. For starters JUnit has always been the choice because it was the first framework for Unit Testing and has always been available. Many people I talk about have not heard about TestNG till we talk about it.

Flexibility

Let us look at a very simple test case for each of the two.

package com.kapil.itrader;
import java.util.Arrays;
import java.util.List;
import junit.framework.Assert;
import org.junit.BeforeClass;
import org.junit.Test;

public class FibonacciTest
{
    private Integer input;
    private Integer expected;

    @BeforeClass
    public static void beforeClass()
    {
        // do some initialization
    }

    @Test
    public void FibonacciTest()
    {
        System.out.println("Input: " + input + ". Expected: " + expected);
        Assert.assertEquals(expected, Fibonacci.compute(input));
        assertEquals(expected, Fibonacci.compute(input));
    }
}

Well, this is example showcases I am using a version 4.x+ and am making use of annotations. Priori to release 4.0; JUnit did not support annotations and that was a major advantage that TestNG had over its competitor; but JUnit had quickly adapted. You can notice that JUnit also supports static imports and we can do away with more cumbersome code as in previous versions.

package com.kapil.framework.core;
import junit.framework.Assert;
import org.springframework.context.support.ClassPathXmlApplicationContext;
import org.testng.annotations.BeforeSuite;
import org.testng.annotations.Test;

public class BaseTestCase
{
    protected static final ClassPathXmlApplicationContext context;

    static
    {
        context = new ClassPathXmlApplicationContext("rootTestContext.xml");
        context.registerShutdownHook();
    }

    @BeforeSuite
    private void beforeSetup()
    {
       // Do initialization
    }

    @Test
    public void testTrue()
    {
        Assert.assertTrue(false);
    }
}

A first look at the two code, would infer that both are pretty much the same. However, for those who have done enough unit testing, will agree with me that TestNG allows for more flexibility. JUnit requires me to declare my initialization method as static; and consequently anything that I will write in that method has to be static too. JUnit also requires me to have my initialization method as public; but TestNG does not. I can use best practices from OOP in my testing classes as well. TestNG also allows me to declare Test Suite, Groups, Methods and use annotations like @BeforeSuite, @BeforeMethod, @BeforeGroups in addition to @BeforeClass. This is very helpful when it comes to writing any level of integration testing or unit test cases that need to access common data sets.

Test Isolations and Dependency Testing

Junit is very effective when it comes to testing in isolation. It essentially means that there is you can not control the order of execution of tests. And, hence if you have two tests that you want to run in a specific order because of any kind of dependency, you can not do that using JUnit. However, TestNG allows you to do this very effectively. In Junit you can make workaround this problem, but it is not neat and that easy.

Parameter based Testing

A very powerful feature that TestNG offers is “Parameterized Testing”. JUnit has added some support for this in 4.5+ versions, but it is not as effective as TestNG. You may have worked with FIT you would know what I am talking about. However, the support added in JUnit is very basic and not that effective. I have modified my previous test case to include parameterized testing.

package com.kapil.itrader;

import static org.junit.Assert.assertEquals;

import java.util.Arrays;
import java.util.List;

import junit.framework.Assert;

import org.junit.BeforeClass;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.junit.runners.Parameterized;
import org.junit.runners.Parameterized.Parameters;

@RunWith(Parameterized.class)
public class FibonacciTest
{
    private Integer input;
    private Integer expected;

    @Parameters
    public static List data()
    {
        return Arrays.asList(new Integer[][] { { 0, 0 }, { 1, 1 }, { 2, 1 }, { 3, 2 }, { 4, 3 }, { 5, 5 }, { 6, 8 } });
    }

    @BeforeClass
    public static void beforeClass()
    {
        System.out.println("Before");
    }

    public FibonacciTest(Integer input, Integer expected)
    {
        this.input = input;
        this.expected = expected;
    }

    @Test
    public void FibonacciTest()
    {
        System.out.println("Input: " + input + ". Expected: " + expected);
        Assert.assertEquals(expected, Fibonacci.compute(input));
        assertEquals(expected, Fibonacci.compute(input));
    }

}

You will notice that I have used @RunWith annotation to allow my test case to be parameterized. In this case, the inline method – data() which has been annotated with @Parameters will be used to provide data to the class. However, the biggest issue is that the data is passed to class constructor. This allows me to code only logically bound test cases in this class. And, I will end up having multiple test cases for one service because all the various methods in the Service wil require different data sets. The good thing is that there are various open source frameworks which have extended this approach and added their own “RunWith” implementations to allow integration with external entities like CSV, HTML or Excel files.

TestNG provides this support out of the box. Not support for reading from CSV or external files, but from Data Providers.

package com.kapil.itrader.core.managers.admin;

import org.testng.Assert;
import org.testng.annotations.Test;

import com.uhc.simple.common.BaseTestCase;
import com.uhc.simple.core.admin.manager.ILookupManager;
import com.uhc.simple.core.admin.service.ILookupService;
import com.uhc.simple.dataprovider.admin.LookupValueDataProvider;
import com.uhc.simple.dto.admin.LookupValueRequest;
import com.uhc.simple.dto.admin.LookupValueResponse;

/**
 * Test cases to test {@link ILookupService}.
 */
public class LookupServiceTests extends BaseTestCase
{

    @Test(dataProvider = "LookupValueProvider", dataProviderClass = LookupValueDataProvider.class)
    public void testGetAllLookupValues(String row, LookupValueRequest request, LookupValueResponse expectedResponse)
    {
        ILookupManager manager = super.getLookupManager();
        LookupValueResponse actualResponse = manager.getLookupValues(request);
        Assert.assertEquals(actualResponse.getStatus(), expectedResponse.getStatus());
    }
}

The code snippet above showcases that I have used dataProvider as a value to the annotations and then I have provided a class which is responsible for creating the data that is supplied to the method at the time of invocation. Using this mechanism, I can easily write test cases and its data providers in a de-coupled fashion and use it very effectively.

Why I choose TestNG

For me the Parameterized Testing is the biggest reason why I choose TestNG over Junit. However, everything that I have listed above is the reason why I always want to spend a few minutes in setting up TestNG in a new Eclipse setup or maven project. TestNG is very useful when it comes to running big test suites. For a small project or a training exercise JUnit is fine; because anyone can start with it very quickly; but not for projects where we need 1000s of test cases and in most of those test cases you will have various scenarios to cover.

Reference: TestNG or JUnit from our JCG partner Kapil Viren Ahuja at the Scratch Pad blog.