JUnit 5: Dynamic Tests

JUnit 5 was formally released in July 2016 and is quite a major evolution to JUnit 4 which has been the standard unit testing framework since Android first appeared on the scene. There are some quite significant changes and getting things set up, and then getting the best out of JUnit 5 can require a little effort, but the results are well worth it. In this loosely-coupled series we’ll explore a few distinct aspects of JUnit 5 and look at how we can get the best out of it. In this article we’ll look at some of the new features in JUnit 5 and see how they can simplify our tests.

Previously we’ve looked at how we can use @DisplayName and @Nested inner classes to improve the readability of both our test reports and the test code itself, and also to reduce some of the boilerplate. However, when we have tests which are largely identical, as is the case with the test suite we’re looking at, we can further simplify things and reduce boilerplate.

Let’s take a look at our existing tests to see quite how similar they really are. Consider these two:

public class CalculatorJUnit5Test {

    private Calculator calculator;
    final float input1 = 1;

    @BeforeEach
    void setup() {
        calculator = new Calculator();
    }

    @Nested
    @DisplayName("Given inputs of One and Two")
    class oneAndTwo {
        final float input2 = 2;

        @Test
        @DisplayName("When we add them Then the result is Three")
        void plus() {
            float result = calculator.calculate(input1, input2, Operator.PLUS);

            assertThat(result).isEqualTo(3f);
        }

        @Test
        @DisplayName("When we subtract them Then the result is Minus One")
        void minus() {
            float result = calculator.calculate(input1, input2, Operator.MINUS);

            assertThat(result).isEqualTo(-1f);
        }
        .
        .
        .
    }
    .
    .
    .
}

Each test contains two lines:

The first represents the “When...” clause and the only difference is the Operator which gets applied – in the first test it is Operator.PLUS, and in the second it is Operator.MINUS. The rest of the line is identical.

The second represents the “Then…” clause and the only difference is the expected value which is the argument to the isEqualTo() method call. The rest of the line is identical.

The obvious optimisation that we can do here is to extract the test to a separate method and pass in these two variable components as arguments:

public class CalculatorJUnit5Test {

    private Calculator calculator;
    final float input1 = 1;

    @BeforeEach
    void setup() {
        calculator = new Calculator();
    }

    @Nested
    @DisplayName("Given inputs of One and Two")
    class oneAndTwo {
        final float input2 = 2;

        @Test
        @DisplayName("When we add them Then the result is Three")
        void plus() {
            performTest(input1, input2, Operator.PLUS, 3f);
        }

        @Test
        @DisplayName("When we subtract them Then the result is Minus One")
        void minus() {
            performTest(input1, input2, Operator.MINUS, -1f);
        }
        .
        .
        .
    }
    .
    .
    .

    void performTest(float value1, float value2, Operator operator, float expected) {
        float result = calculator.calculate(value1, value2, operator);

        assertThat(result).isEqualTo(expected);
    }
}

That actuall is beginning to look pretty good. We’ve taken out much of the repetition and now the full JUnit 5 test suite has been reduced to 110 lines.

But we’re not going to stop there. There is actually another new feature of JUnit 5 which can further simplify things: Dynamic Tests. Before we proceed, it is worth pointing out that Dynamic Tests are still an experimental feature of JUnit 5, but I’ve found them to be quite reliable whenever I’ve had cause to use them.

There are two key components when it comes to using Dynamic Tests – the first is a TestFactory which is responsible for creating a set of tests, and the second is the DynamicTest itself which essentially creates a mapping between a block of code to execute for the test and the DisplayName to use for it.

To access the full power of DynamicTests requires some APIs which are only available in Java 8 onwards – and we don’t have access to them in our Android projects. We’ll cover a bit more on Java 8 later in the series. However they are still pretty useful even with the older APIs.

Let’s begin by looking at the DynamicTest itself:

public class CalculatorJUnit5Test {
    .
    .
    .

    private DynamicTest createTest(final float value1, final float value2, final Operator operator, final float expected) {
        String displayName = String.format(Locale.getDefault(), "When we %s them Then the result is %.2f", operator.name(), expected);
        return dynamicTest(displayName, new Executable() {
            @Override
            public void execute() throws Throwable {
                float result = calculator.calculate(value1, value2, operator);

                assertThat(result).isEqualTo(expected);
            }
        });
    }
}

First we construct the displayName string based on the operator and expected value. Next we create a DynamicTest passing in this string and an Executable object. Executable is an interface defined within the JUnit5 Jupiter API, and requires us to implement a single method named execute(). The contents of this are almost identical to the performTest)_ method we created earlier – the only difference being that performTest() had parameters passed as arguments whereas execute accesses final arguments passed to the parent method.

Essentially this enables us to create both the test code and the appropriate DisplayName dynamically based on the parameters passed in.

Now we need to create the Factories which will construct these tests. A Factory simply returns a Collection of DynamicTests:

public class CalculatorJUnit5Test {

    private Calculator calculator;
    private final float input1 = 1;

    @BeforeEach
    void setup() {
        calculator = new Calculator();
    }

    @TestFactory
    @DisplayName("Given inputs of One and Two")
    Collection<DynamicTest> oneAndTwo() {
        final float input2 = 2;
        Set<DynamicTest> tests = new LinkedHashSet<>(6);
        tests.add(createTest(input1, input2, Operator.PLUS, 3f));
        tests.add(createTest(input1, input2, Operator.MINUS, -1f));
        tests.add(createTest(input1, input2, Operator.MULTIPLY, 2f));
        tests.add(createTest(input1, input2, Operator.DIVIDE, 0.5f));
        tests.add(createTest(input1, input2, Operator.MODULO, 1f));
        tests.add(createTest(input1, input2, Operator.UNKNOWN, 0f));
        return tests;
    }
    .
    .
    .
}

The TestFactory replaces the @Nested inner class that we used previously, whilst maintaining the same @DisplayName. It does nothing more than create a set of DynamicTest instances and the test platform will iterate through this Collection and run each test in turn, and create a report using the appropriate DisplayName.

If we run this we can see that the report is much the same as before except for the case of the operator name resulting from generating this dynamically:

For the first time I’m now going to include the entire JUnit 5 test suite because this set of changes finally reduces the same set of twelve test down to a really terse 67 lines:

package com.stylingandroid.junit5;

import java.util.Collection;
import java.util.LinkedHashSet;
import java.util.Locale;
import java.util.Set;

import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.DynamicTest;
import org.junit.jupiter.api.Executable;
import org.junit.jupiter.api.TestFactory;

import static org.assertj.core.api.Java6Assertions.assertThat;
import static org.junit.jupiter.api.DynamicTest.dynamicTest;

@SuppressWarnings("unused")
public class CalculatorJUnit5Test {

    private Calculator calculator;
    private final float input1 = 1;

    @BeforeEach
    void setup() {
        calculator = new Calculator();
    }

    @TestFactory
    @DisplayName("Given inputs of One and Two")
    Collection<DynamicTest> oneAndTwo() {
        final float input2 = 2;
        Set<DynamicTest> tests = new LinkedHashSet<>(6);
        tests.add(createTest(input1, input2, Operator.PLUS, 3f));
        tests.add(createTest(input1, input2, Operator.MINUS, -1f));
        tests.add(createTest(input1, input2, Operator.MULTIPLY, 2f));
        tests.add(createTest(input1, input2, Operator.DIVIDE, 0.5f));
        tests.add(createTest(input1, input2, Operator.MODULO, 1f));
        tests.add(createTest(input1, input2, Operator.UNKNOWN, 0f));
        return tests;
    }

    @TestFactory
    @DisplayName("Given inputs of One and Zero")
    Collection<DynamicTest> oneAndZero() {
        final float input2 = 0;
        Set<DynamicTest> tests = new LinkedHashSet<>(6);
        tests.add(createTest(input1, input2, Operator.PLUS, 1f));
        tests.add(createTest(input1, input2, Operator.MINUS, 1f));
        tests.add(createTest(input1, input2, Operator.MULTIPLY, 0f));
        tests.add(createTest(input1, input2, Operator.DIVIDE, Float.POSITIVE_INFINITY));
        tests.add(createTest(input1, input2, Operator.MODULO, Float.NaN));
        tests.add(createTest(input1, input2, Operator.UNKNOWN, 0f));
        return tests;
    }

    private DynamicTest createTest(final float value1, final float value2, final Operator operator, final float expected) {
        String displayName = String.format(Locale.getDefault(), "When we %s them Then the result is %.2f", operator.name(), expected);
        return dynamicTest(displayName, new Executable() {
            @Override
            public void execute() throws Throwable {
                float result = calculator.calculate(value1, value2, operator);

                assertThat(result).isEqualTo(expected);
            }
        });
    }
}

This is finally getting to a point where we have quite a full set of tests which are nicely compact. As a result of this I now feel that they are much easier to read and understand, so are much more fit for purpose as documentation of the expected behaviour of the system under test.

However, we’re not going to stop there. Those familiar with lambda expressions will have noticed that the Executable that we created as part of our DynamicTest is a perfect candidate for replacement with a lambda. In the next article in this series we’ll look at the various options for using lambdas in our tests.

The source code for this article is available here.

© 2017, Mark Allison. All rights reserved.

Copyright © 2017 Styling Android. All Rights Reserved.
Information about how to reuse or republish this work may be available at http://blog.stylingandroid.com/license-information.