JUnit 5 – How to Write Parameterized Tests

By using parameterized tests, we can reuse the single test configuration between multiple test cases. It will allow us to reduce the code base and easily verify multiple test cases without the need to create a separate test method for each one. This article will describe how we can develop parameterized tests using the JUnit 5 framework. We will go through the following steps:

1. Setup the required dependencies with Maven and Gradle build tools
2. Create the first parameterized test
3. Discover arguments sources for simple data types

Before we start, please make sure that you’re familiar with the following topics:

1. Creating software projects using either Maven or Gradle build tools
2. Writing unit tests using Java and the JUnit 5 framework 

Dependency setup

JUnit 5 doesn’t provide support for running parameterized tests out of the box. To enable this feature, we have to include the junit-jupiter-params dependency into our project. In case you’re using Maven, you need to include such a code snippet in your pom.xml file:

XML

<dependency>
    <groupId>org.junit.jupiter</groupId>
    <artifactId>junit-jupiter-params</artifactId>
    <version>5.9.0</version>
    <scope>test</scope>
</dependency>

The full example of the pom.xml file:

XML

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
                             https://maven.apache.org/xsd/maven-4.0.0.xsd">

    <modelVersion>4.0.0</modelVersion>

    <groupId>com.example</groupId>
    <artifactId>parameterized-tests-example</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <name>parameterized-tests-example</name>
    <description>parameterized-tests-example</description>

    <properties>
        <java.version>11</java.version>
        <maven.compiler.source>11</maven.compiler.source>
        <maven.compiler.target>11</maven.compiler.target>
    </properties>

    <dependencies>
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <version>1.18.24</version>
            <optional>true</optional>
        </dependency>

        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter-engine</artifactId>
            <version>5.9.0</version>
        </dependency>
        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter-params</artifactId>
            <version>5.9.0</version>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>2.22.0</version>
            </plugin>
        </plugins>
    </build>
</project>

Gradle users need to add this dependency to the test implementation dependency section. It can be done by adding the following code block to the build.gradle file:

Kotlin

testImplementation(
        'org.junit.jupiter:junit-jupiter-params:5.9.0'
)

Creating a parameterized test

Let’s create a simple test service for use in our first parameterized test. We’ll need something primitive, and I will suggest going with the phone validation service, which accepts a single phone number as a String and validates it against some regex. Please refer to the following example below:

Java

import java.util.regex.Pattern;

public interface PhoneValidationService {
    boolean validatePhone(String phone);
}

public class TestPhoneValidationService implements PhoneValidationService {

    private final Pattern phoneRegex = Pattern.compile("^\\+?(?:[0-9] ?){6,14}[0-9]$");

    @Override
    public boolean validatePhone(String phone) {
        return phone != null && phoneRegex.matcher(phone).matches();
    }

}

Our method for validating phones should return true in the case the provided phone number is not null and matches the regex, and false otherwise. To create simple unit tests with JUnit we use the @Test annotation. In the case of parameterized tests, we need to use the @ParameterizedTest annotation instead. Besides that, we need to provide an argument source — this is the structure that will hold our test arguments (phone numbers in our case). Please refer to the whole test class below, and then we will go through it in detail.

Java

import com.example.phone.PhoneValidationService;
import com.example.phone.TestPhoneValidationService;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.ValueSource;

import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertTrue;

class ValueSourceExampleParameterizedTest {

    private final PhoneValidationService phoneValidationService = new TestPhoneValidationService();

    @ParameterizedTest
    @ValueSource(strings = {"555 555 55 55", "5555555555", "+15555555555"})
    void testProcessValidPhones(String phone) {
        assertTrue(phoneValidationService.validatePhone(phone));
    }

    @ParameterizedTest
    @ValueSource(strings = {"555", "@+15555555555", "test"})
    void testProcessInvalidPhones(String phone) {
        assertFalse(phoneValidationService.validatePhone(phone));
    }

}

As was mentioned above the @ParameterizedTest simply replaces the @Test annotation. But since it’s a parameterized test, we have also added another annotation @ValueSource. It has an array of strings with valid phone numbers in the first method and nonvalid ones in the second one. Each string is a separate test case with which our test will run. Also, you may have noticed one last obvious difference with non-parameterized tests — our test method has a parameter in its signature, the phone number of type String in our case. Running this test class will give us the following execution results:

Execution results of the ValueSourceExampleParameterizedTest class

We may specify arrays of strings, classes, shorts, bytes, ints, longs, floats, doubles, chars, and booleans in the @ValueSource annotation. However, it supports only one type of input. Because of this, we’re not able to specify several test arguments using this annotation, so it makes sense to use it only for simple test cases. This article focuses only on simple data types and argument sources for parameterized tests. All available argument sources may be found in the org.junit.jupiter.params.provider package. Let’s review a few more simple argument source annotations, which may come in handy.

Using @NullSource and @EmptySource

We often need to test our code with null and empty values to be assured that we can process them correctly and avoid the most loved `NullPointerException`. Parameterized test argument sources provide us the built-in support for such test data with  @NullSource and @EmptySource annotations. Let’s modify our parameterized tests for the phone validation services and see how it works:

Java

import com.example.phone.PhoneValidationService;
import com.example.phone.TestPhoneValidationService;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.EmptySource;
import org.junit.jupiter.params.provider.NullSource;
import org.junit.jupiter.params.provider.ValueSource;

import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertTrue;

class ValueNullAndEmptySourceExampleParameterizedTest {

    private final PhoneValidationService phoneValidationService = new TestPhoneValidationService();

    @ParameterizedTest
    @ValueSource(strings = {"555 555 55 55", "5555555555", "+15555555555"})
    void testProcessValidPhones(String phone) {
        assertTrue(phoneValidationService.validatePhone(phone));
    }

    @ParameterizedTest
    @NullSource
    @EmptySource
    @ValueSource(strings = {"555", "@+15555555555", "test"})
    void testProcessInvalidPhones(String phone) {
        assertFalse(phoneValidationService.validatePhone(phone));
    }

}

As you can see, we have simply added these additional annotations besides @ValueSource. This change will add two additional test cases (null and empty values) to our method for testing invalid phone arguments. This how the execution results will look like for our updated test class:

Execution results of the ValueNullAndEmptySourceExampleParameterizedTest class

Also, we can replace  @NullSource and @EmptySource annotations with a single one @NullAndEmptySource, which combines these two for our convenience. Next, we will review one more useful argument source, which may be used for simple test cases.

Using @EnumSource

Let’s assume that we have a method that receives some enum value as a parameter and performs some operations based on the value of this parameter. For example, the method for sending messages through different channels:

Java

import java.util.UUID;

public interface MessageService {
    Message sendMessage(Message message, Channel channel);
}

public class TestMessageService implements MessageService {
    @Override
    public Message sendMessage(Message message, Channel channel) {
        // send a message based on the channel value,
          // and return the message object with a generated id
        message.setId(UUID.randomUUID().toString());
        return message;
    }
}

The enum class with possible channel values and the Message class will look like this:

Java

import lombok.AllArgsConstructor;
import lombok.Builder;
import lombok.Data;
import lombok.NoArgsConstructor;

@Data
@Builder
@NoArgsConstructor
@AllArgsConstructor
public class Message {
    private String id;
    private String message;
}

public enum Channel {
    SMS,
    EMAIL,
    WHATSAPP,
    SLACK
}

It will make sense to test this method’s behavior with all possible channel types. And with parameterized tests, we can do this with a single test method, instead of creating a separate one for each channel type. This will be the case for using our next possible arguments source — @EnumSource. Let’s firstly look at the code example and then analyze it:

Java

import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.EnumSource;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertNotNull;

class EnumSourceExampleParameterizedTest {

    private final MessageService messageService = new TestMessageService();

    @ParameterizedTest
    @EnumSource(Channel.class)
    void testSendMessage(Channel channel) {
        Message message = messageService.sendMessage(createMessage(channel), channel);
        assertNotNull(message);
        assertNotNull(message.getId());
        assertFalse(message.getId().isEmpty());
    }

    private Message createMessage(Channel channel) {
        // create a message based on the channel value
        return Message.builder()
                .message(String.format("Test %s message", channel))
                .build();
    }
}

To make it more like a real-world test class, we have initiated the ‘MessageService’ class and created a separate method called ‘createMessage’ which will create a test message for us based on the channel type. We’re using the builder pattern implementation provided by Lombok to create a message, please check it here if you’re not familiar with it. The @ValueSource annotation from the previous example was replaced with @EnumSource. Everything else should look familiar. As a value for @EnumSource, we have added our Channel enum class. It will make the parameterized test run with every value from the enum class. Running this test class will give us the following execution results:

Execution results of the EnumSourceExampleParameterizedTest class

As we can see, our test method was executed four times with every value from the Channel enum class. But let’s imagine the case when a couple of our channels have different behavior, and we need to implement a separate test method for it. Would it be possible to use a parameterized test from this case? Definitely, please refer to the example below to see how it works:

Java

import com.example.message.Channel;
import com.example.message.Message;
import com.example.message.MessageService;
import com.example.message.TestMessageService;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.EnumSource;

import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertNotNull;

class EnumSourceExampleParameterizedTest {

    private final MessageService messageService = new TestMessageService();

    @ParameterizedTest
    @EnumSource(value = Channel.class, names = {"WHATSAPP", "SLACK"})
    void testSendMessage(Channel channel) {
        Message message = messageService.sendMessage(createMessage(channel), channel);
        assertNotNull(message);
        assertNotNull(message.getId());
        assertFalse(message.getId().isEmpty());
    }

    @ParameterizedTest
    @EnumSource(value = Channel.class, names = {"SMS", "EMAIL"})
    void testSendMessageThroughEmailAndSmsChannels(Channel channel) {
        Message message = messageService.sendMessage(createMessage(channel), channel);
        assertNotNull(message);
        assertNotNull(message.getId());
        assertFalse(message.getId().isEmpty());
        // check other custom behavior
    }

    private Message createMessage(Channel channel) {
        // create a message based on the channel value
        return Message.builder()
                .message(String.format("Test %s message", channel))
                .build();
    }

}

We have decomposed the test method into two: the first will test sending messages through WhatsApp and Slack, and the second will test SMS and Email channels. To achieve this, we have modified our @EnumSource annotation. The value property stayed the same — it’s the enum class from which we want to load channel parameters. However, now we have specified the exact list of enum properties for each test method in the ‘names’ property of the enum argument source. Execution results for the updated test class:

Execution results of the EnumSourceExampleParameterizedTest class with decomposed test methods

Customizing names of test cases

Before finishing our experiments with parameterized tests, let’s review one more nice feature, which may make our tests more detailed and understandable for others. We will continue working on our methods for testing the message service. As we have seen on 5.2 execution results, each test case is displayed as a simple enum String representation. Let’s try to improve it by adding a custom detailed message, which may help better understand our test results. To do this, we will modify the @ParameterizedTest annotation by specifying the optional name property with a pattern for the displayed test case title. Please review the code snippet below, and then we will review it in detail:

Java

import com.example.message.Channel;
import com.example.message.Message;
import com.example.message.MessageService;
import com.example.message.TestMessageService;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.EnumSource;

import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertNotNull;

class EnumSourceExampleParameterizedTest {

    private final MessageService messageService = new TestMessageService();

    @ParameterizedTest(name = "[{index}] Send a message through the {0} channel")
    @EnumSource(value = Channel.class, names = {"WHATSAPP", "SLACK"})
    void testSendMessage(Channel channel) {
        Message message = messageService.sendMessage(createMessage(channel), channel);
        assertNotNull(message);
        assertNotNull(message.getId());
        assertFalse(message.getId().isEmpty());
    }

    @ParameterizedTest(name = "[{index}] Send a message through the {0} channel")
    @EnumSource(value = Channel.class, names = {"SMS", "EMAIL"})
    void testSendMessageThroughEmailAndSmsChannels(Channel channel) {
        Message message = messageService.sendMessage(createMessage(channel), channel);
        assertNotNull(message);
        assertNotNull(message.getId());
        assertFalse(message.getId().isEmpty());
        // check other custom behavior
    }

    private Message createMessage(Channel channel) {
        // create a message based on the channel value
        return Message.builder()
                .message(String.format("Test %s message", channel))
                .build();
    }

}

Let’s review the text pattern from the name property of the @ParameterizedTest annotation step by step:

1. All dynamic variables should be placed in curly brackets. Like {index}, for example.
2. The {index} variable is the index of the test method run. Like ‘[1] SMS’ and ‘[2] EMAIL’. 
3. The {0} variable is the reference to the first method argument from our test method. It’s the ‘Channel’ in our case. In case we would have more than one argument, we can reference the second one as {1}, the third one as {2}, and so on.

This how the execution results will look like after this change:

Execution results of the EnumSourceExampleParameterizedTest with customized names

Such customizations may be helpful when we have complex test cases which aren’t intuitively understandable.

Conclusion

In this article, we have reviewed the process of creating parameterized tests with the JUnit 5 framework. We have learned how to configure necessary dependencies with Maven and Gradle build tools. We have discovered how to create parameterized tests using value, null, empty and enum argument sources, and learned the differences in configuration between parameterized and regular tests. Besides this, we have discovered how to configure and customize the displayed names of individual test cases. Complete source code and examples used in this article can be found here.