How to Use JUnit5 to Test Kotlin

JUnit is a popular framework used for testing code that runs in the Java virtual machine (JVM). Junit version 5 has support for functional paradigms like lambdas. Kotlin is a popular JVM based language and is the official language for Android development. In this tutorial you'll see how to test Kotlin code using the Junit5 framework.

Terminology

• Test: A particular scenario to be tested (e.g., calling clear() removes all elements in a collection). You implement it with a method, marked with the Test annotation.
• Test Case: A group of related tests, that can use a common set of resources, and has methods to initialize and destroy those resources. You implement it with a class, which can share instance variables among the methods.
• Test Suite: A group of test cases(e.g., test suites for sanity checks or regression, or test suites based on functional areas). This makes it possible to run a particular selection of test cases instead of running all of them. For example, you might want to run the smoke-test suite first, and only if it passes, then run the other tests. A suite is implemented as a static method suite(), which returns a suite containing its test cases.

Setup

For the purposes of this tutorial, we'll test the LinkedHashMap, a Map implementation that retains the order of insertion, and hence, can also be used somewhat like a List. You’ll need the Junit5 dependencies junit-jupiter-api, junit-jupiter-engine.   

Test Case Class

JUnit does not require the test class to implement or extend any particular interface or class. Rather, we use annotations to mark the test methods. Let’s name our test case class as LinkedMapTest.kt. An initial version of our class, with two test methods looks like this: 

import org.junit.jupiter.api.Assertions
import org.junit.jupiter.api.Test

class LinkedMapTest {
    private var lmap = LinkedHashMap();

    @Test
    fun whenPut2_thenGet2() {
        lmap.put(1,2);
        Assertions.assertEquals(2, lmap.get(1), "The value for key 1 should be 2")
    }

    @Test
    fun whenAdd2_thenSizeIs2() {
        lmap.put(1,2);
        lmap.put(2,3);
        Assertions.assertEquals(2, lmap.size)
        // Check that values are in insertion order
        val values = lmap.values
        Assertions.assertEquals( values.elementAt(0), 2);
        Assertions.assertEquals( values.elementAt(1), 3);
    }
}

‍

IDEs like Intellij and Eclipse provide tools to run Junit test cases. Or you can also run them via the command line. We initialized the map instance where we declared it. The @Test annotation marks the methods that should be run as tests. They have access to the instance variables. Each test method should have a descriptive name as per best practices. (You can use backticks to enclose a method name that has special characters, like spaces). We used the assertEquals method here. Note that we used our custom error message for the first assertEquals. If the message is expensive to compute, say because it concatenates all elements in an array, then you wouldn’t want to build it unless it’s actually required (i.e., when the assertion fails). To do this, you can provide a Supplier<String> as an argument instead of the string message. The Supplier will be called on to produce the message when required. 

Other Assert Methods

There are other assert methods like assertNull, assertTrue, assertArrayEquals, assertLinesMatch, assertSame, assertThrows, assertTimeout, assertInstanceOf, and so on. These are variations of assertTrue. Let’s look at assertTimeout, which you can use to test delays, or to ensure that a piece of code executes within a certain time limit. For example, the test below would fail if it takes more than 500 ms to load the Google home page. 

    @Test
    fun checkTimeout(){
        Assertions.assertTimeout(Duration.ofMillis(500)) {
            URL("http://google.com").readText();
        }
    }

You can also specify a timeout for a test using the @Timeout tag. 

Test Instances

If we look at the methods in our test case, the first one adds an element. The second one adds two more. So shouldn't the size be three? It would be if we used the same instance of the test class for testing all the methods. But that is usually not a good thing to do, because each test case would have to keep track of which data was updated by the test cases that came before it. If we changed some data in one test method, it could break the test methods after it. This is undesirable. It’s easier to write each test method as if it starts with a clean slate. So, by default, JUnit creates a new instance of the test class to test each method.

However, in some scenarios, you have to perform some expensive operation, like loading data at the start of the test case, and all the test methods use that data. In that case, you wouldn’t want to run the expensive initialize for each test method. Instead, you go for the single test case instance. You can control this with the @TestInstance annotation, which accepts a Lifecyle type argument. LifeCycle.PER_METHOD is the default, wherein each method executes on a new test case instance. We can specify LifeCycle.PER_CLASS to use the same test case instance. You can also use the BeforeAll/AfterAll callbacks to initialize or release the resources needed for a test case. When using the PER_CLASS execution, the order in which methods execute may be important. You can use the @TestMethodOrder annotation to control the order of test-method execution. 

Multiple Assertions in a Go

When a test has multiple assertions one after another, the test will abort at the first failure. The assertions coming later could fail too, but they won't even be tested. If you want a set of assertions to be tested, all in one go, the assertAll method can come in handy. So for the above example, we could write: 

Assertions.assertAll(
    Executable { Assertions.assertEquals( values.elementAt(0), 2) }, 
    Executable { Assertions.assertEquals( values.elementAt(1), 3)}
)

‍

Executable is a functional interface that has an execute() method. Here, we provide the execute() method using the code within braces. The assertAll method will run all the executables passed to it, even if one or more fail. 

Testing for Exceptions

Some tests will throw particular exceptions. For example, if you try to access an element from a collection at an index that’s not present, you’ll get an IndexOutOfBounds exception. You can test this on the values collection, which is empty when lmap is empty: 

@Test
    fun whenIndexOutofBounds_thenThrow(){
        val iobe = Assertions.assertThrows(IndexOutOfBoundsException::class.java) {
            lmap.values.elementAt(0)
        };
        println( iobe.message);
    }

‍

Here, assertThrows checks that the enclosing block throws the expected type of exception and also returns it for further processing, if any. Note that the expected class can also be a superclass of the actual exception class, like RuntimeException, and the test would still pass. If no exception or the incorrect type of exception is thrown, the test would fail. Note how you specify the class; you need a Java class object as the argument, not a Kotlin class. 

Disabling Tests

You can disable some tests (that are breaking but can't be fixed right away) with the @Disabled annotation. These tests still appear in the test report so you can keep track of them. 

Conditional Execution

You can execute tests based on certain conditions, which you provide in annotations. You can also disable conditions for a test run using the junit.jupiter.conditions.deactivate system property. 

Os

Sometimes you want to run tests on certain OSes or platforms and not on others. The EnableOnOs and DisableOnOs annotations can help here. 

@EnabledOnOs({ LINUX, WINDOWS})

A test method with the above annotation runs only if the OS is Linux or Windows. 

JRE

Another criteria is JRE versions. You may only want to run a test on certain versions, or on a range of versions: 

@EnabledForJreRange(min = JAVA_9, max = JAVA_13)

A test method with the annotation above runs only on JRE versions between 9 and 13. 

Native Image

@EnabledInNativeImage

A test method with this annotation runs only in a native image using the GraalVM. A native image is one compiled to native executable code like .exe, .dll, and .so files, rather than Java byte code. This leads to a smaller footprint and faster startup, which is important for running the application as a containerized service. 

System Properties

You can use Java system properties to conditionally enable tests. 

@DisabledIfSystemProperty(named = "server-type", matches = "standalone")

A test method with annotation as above won’t run if the Java system property server-type has the value standalone. 

Environment Variables

Quite similar to system properties, you can also use environment variables in conditions. For example: 

@EnabledIfEnvironmentVariable(named = "MODE", matches = "dev")

A test method with the above annotation runs only if the environment variable MODE has the value dev. 

Custom

You can use a custom method that returns a boolean to provide a condition. For example, a test method with the annotation below executes only if the isTomcat() method in the my.SystemUtils class returns as true. 

@EnabledIf("my.SystemUtils#isTomcat")

Tags

You can also annotate test methods with tags, and then run only those tests with (without) particular tags. You can use any string as a tag. Below is an example of tagging a test method with the tag smoke-test: 

@Tag("smoke-test")

Constructor and Method Parameters

Up to version 4, with the default test runner, Junit did not allow parameters to be passed to test case methods or constructors. Since we achieve dependency injection using such parameters, it was a sorely needed feature. Junit5 now allows this. It supports two classes, TestInfo and TestReporter, as parameters by default. You can use TestInfo to get information about the test you’re running, and TestReporter to query or manipulate the test report you’re generating. You need to use another mechanism to resolve other types of parameters. See parameter resolver for more details.   

Grouping Annotations

You can create a custom annotation that groups other annotations. When using a custom annotation, Junit applies the other annotations associated with it. Below, we declare an annotation named LinuxSmokeTest: 

import org.junit.jupiter.api.Tag
import org.junit.jupiter.api.Test
import org.junit.jupiter.api.condition.EnabledOnOs
import org.junit.jupiter.api.condition.OS

@Tag("smoke-test")
@EnabledOnOs( OS.LINUX)
@Test
annotation class LinuxSmokeTest()

Now, if you annotate a test method with LinuxSmokeTest, Junit internally applies the annotations associated with it. So it will know this is a test method, with tag smoke-test, and it should run it only on the Linux OS. 

LifeCycle Methods

You can use the following life-cycle methods in the test class to add more functionality. 

• @BeforeAll — Junit calls a method with this annotation only once per test class, before any tests and hooks. The method needs to be static if the TestInstance is PER_METHOD, which is the default. Since Kotlin does not have static methods, you need to define a companion object that has an @JvmStatic method, and use that.
• @BeforeEach — Junit calls a method with this annotation once before it runs each test in a class. If you pass TestInfo as parameter, you can get information about the test, like name, tags, and so on.
• @AfterEach — Junit calls a method with this annotation once after it runs each test in a class. If you pass TestInfo as parameter, you can get information about the test, like name, tags, and so on.
• @AfterAll — Junit calls a method with this annotation only once per test class, after it runs all tests and hooks. The method needs to be static if the TestInstance is PER_METHOD, which is the default. Since Kotlin does not have static methods, you need to define a companion object that has a @JvmStatic method, and use that.

class DAOTestCase {
    private val conn : Connection = DBUtils.getConn();

    @BeforeEach
    fun beforeEachTest(testInfo: TestInfo){
        if( testInfo.tags.contains("dbconn")){
            conn.autoCommit = false;
        }
    }

    @AfterEach
    fun afterEachTest(testInfo: TestInfo){ 
       if( testInfo.tags.contains("dbconn")){
           conn.rollback()
       }
    }

    @Test
    @Tag("dbconn")
    fun addUser(){
        // some code
    }

}

In the code above, before and after each test that has a tag dbconn, you’re manipulating the instance variable conn for transaction control. You can define more than one method for a life-cycle event, in which case Junit orders them in a consistent, but not obvious, order. Also, if you define such methods in the superclasses of the test class, Junit executes those too. You can also implement these methods using the extensions mechanism, which we’ll show you next. It may make sense to use the class level life-cycle methods for processing related to that specific class, and use the extensions mechanism for common processing like logging, tracing, and exception handling. 

Extensions

JUnit provides a powerful extension mechanism to provide common functionalities, like logging and tracing, across tests. You write the extension code in separate classes and incorporate it into the test classes, declaratively with @ExtendWith, programmatically with @RegisterExtension, or automatically using the service-loader mechanism in Java. Let’s look at an example using @ExtendWith, which is similar to implementing interfaces that have default methods. Here, we’ll try to provide common exception handling through the extension ExceptionHandlerExtension. 

import org.junit.jupiter.api.extension.ExtensionContext
import org.junit.jupiter.api.extension.TestExecutionExceptionHandler

class ExceptionHandlerExtension : TestExecutionExceptionHandler {
    override fun handleTestExecutionException(context : ExtensionContext, throwable: Throwable ) {
        println( "In exception handler")
        if( throwable is OutOfMemoryError){
            // Send mail to devops team
        }
        throw throwable;
    }
}

This extension extends TestExecutionExceptionHandler and overrides the handleTestExecutionException method. You can choose whether to suppress or rethrow the exception. Now, if you want to apply this to a test class, you would include it before the class definition: 

@ExtendWith(ExceptionHandlerExtension::class)
class SomeTestCase {
}

If any tests within the extended test class throw an exception, Junit sends it to the extension class for handling. Extensions can do many other things in addition to exception handling, like life-cycle callbacks, resolving parameters, intercepting invocations, and using test templates, to name a few. 

Conclusion

Junit is a feature rich and extensible framework for testing. It serves quite well for testing Kotlin applications. Rather than having to write and manage tests manually, tools like Waldo make it easier for developers to test mobile applications.