Advanced unit testing with JUnit 5, Mockito, and Hamcrest

In this second half of a two-part introduction to JUnit 5, we’ll move beyond the basics and learn how to test more complicated scenarios. In the previous article, you learned how to write tests using the @Test and @ParameterizedTest annotations, validate test results using JUnit 5’s built-in assertions, and work with JUnit 5 lifecycle annotations and tags. In this article, we’ll focus more on integrating external tools with JUnit 5.

You’ll learn:

How to use Hamcrest to write more flexible and readable test cases.

How to use Mockito to create mock dependencies that let you simulate any scenario you want to test.

How to use Mockito spies to ensure that method calls return the correct values, as well as verify their behavior.

Using JUnit 5 with an assertions library

For most circumstances, the default assertions methods in JUnit 5 will meet your needs. But if you would like to use a more robust assertions library, such as AssertJ, Hamcrest, or Truth, JUnit 5 provides support for doing so. In this section, you’ll learn how to integrate Hamcrest with JUnit 5.

Hamcrest with JUnit 5

Hamcrest is based on the concept of a matcher, which can be a very natural way of asserting whether or not the result of a test is in a desired state. If you have not used Hamcrest, examples in this section should give you a good sense of what it does and how it works.

The first thing we need to do is add the following additional dependency to our Maven POM file (see the previous article for a refresher on including JUnit 5 dependencies in the POM):

org.hamcrest
hamcrest
3.0
test

When we want to use Hamcrest in our test classes, we need to leverage the org.hamcrest.MatcherAssert.assertThat method, which works in combination with one or more of its matchers. For example, a test for String equality might look like this:

assertThat(name, is('Steve'));

Or, if you prefer:

assertThat(name, equalTo('Steve'));

Both matchers do the same thing—the is() method in the first example is just syntactic sugar for equalTo().

Hamcrest defines the following common matchers:

Objects: equalTo, hasToString, instanceOf, isCompatibleType, notNullValue, nullValue, sameInstance

Text: equalToIgnoringCase, equalToIgnoringWhiteSpace, containsString, endsWith, startsWith

Numbers: closeTo, greaterThan, greaterThanOrEqualTo, lessThan, lessThanOrEqualTo

Logical: allOf, anyOf, not

Collections: array (compare an array to an array of matchers), hasEntry, hasKey, hasValue, hasItem, hasItems, hasItemInArray

The following code sample shows a few examples of using Hamcrest in a JUnit 5 test class.

Listing 1. Using Hamcrest in a JUnit 5 test class (HamcrestDemoTest.java)

package com.javaworld.geekcap.hamcrest;

import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;

import java.util.ArrayList;
import java.util.List;

import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.Matchers.*;

class HamcrestDemoTest {
@Test
@DisplayName('String Examples')
void stringExamples() {
String s1 = 'Hello';
String s2 = 'Hello';

assertThat('Comparing Strings', s1, is(s2));
assertThat(s1, equalTo(s2));
assertThat('ABCDE', containsString('BC'));
assertThat('ABCDE', not(containsString('EF')));
}

@Test
@DisplayName('List Examples')
void listExamples() {
// Create an empty list
List list = new ArrayList();
assertThat(list, isA(List.class));
assertThat(list, empty());

// Add a couple items
list.add('One');
list.add('Two');
assertThat(list, not(empty()));
assertThat(list, hasSize(2));
assertThat(list, contains('One', 'Two'));
assertThat(list, containsInAnyOrder('Two', 'One'));
assertThat(list, hasItem('Two'));
}

@Test
@DisplayName('Number Examples')
void numberExamples() {
assertThat(5, lessThan(10));
assertThat(5, lessThanOrEqualTo(5));
assertThat(5.01, closeTo(5.0, 0.01));
}
}

One thing I like about Hamcrest is that it is very easy to read. For example, “assert that name is Steve,” “assert that list has size 2,” and “assert that list has item Two” all read like regular sentences in the English language. In Listing 1, the stringExamples test first compares two Strings for equality and then checks for substrings using the containsString() method. An optional first argument to assertThat() is the “reason” for the test, which is the same as the message in a JUnit assertion and will be displayed if the test fails. For example, if we added the following test, we would see the assertion error below it:

assertThat('Comparing Strings', s1, is('Goodbye'));

java.lang.AssertionError: Comparing Strings
Expected: is 'Goodbye'
but: was 'Hello'

Also note that we can combine the not() logical method with a condition to verify that a condition is not true. In Listing 1, we check that the ABCDE String does not contain substring EF using the not() method combined with containsString().

The listExamples creates a new list and validates that it is a List.class, and that it’s empty. Next, it adds two items, then validates that it is not empty and contains the two elements. Finally, it validates that it contains the two Strings, 'One' and 'Two', that it contains those Strings in any order, and that it has the item 'Two'.

Finally, the numberExamples checks to see that 5 is less than 10, that 5 is less than or equal to 5, and that the double 5.01 is close to 5.0 with a delta of 0.01, which is similar to the assertEquals method using a delta, but with a cleaner syntax.

If you’re new to Hamcrest, I encourage you to learn more about it from the Hamcrest website.

Get the code
Download the source code for all examples in this article.

Introduction to Mock objects with Mockito

Thus far, we’ve only reviewed testing simple methods that do not rely on external dependencies, but this is far from typical for large applications. For example, a business service probably relies on either a database or web service call to retrieve the data that it operates on. So, how would we test a method in such a class? And how would we simulate problematic conditions, such as a database connection error or timeout?

The strategy of mock objects is to analyze the code behind the class under test and create mock versions of all its dependencies, creating the scenarios that we want to test. You can do this manually—which is a lot of work—or you could leverage a tool like Mockito, which simplifies the creation and injection of mock objects into your classes. Mockito provides a simple API to create mock implementations of your dependent classes, inject the mocks into your classes, and control the behavior of the mocks.

The example below shows the source code for a simple repository.

Listing 2. Example repository (Repository.java)

package com.javaworld.geekcap.mockito;

import java.sql.SQLException;
import java.util.Arrays;
import java.util.List;

public class Repository {
public List getStuff() throws SQLException {
// Execute Query

// Return results
return Arrays.asList('One', 'Two', 'Three');
}
}

This next listing shows the source code for a service that uses this repository.

Listing 3. Example service (Service.java)

package com.javaworld.geekcap.mockito;

import java.sql.SQLException;
import java.util.Arrays;
import java.util.List;
import java.util.stream.Collectors;

public class Service {
private Repository repository;

public Service(Repository repository) {
this.repository = repository;
}

public List getStuffWithLengthLessThanFive() {
try {
return repository.getStuff().stream().filter(stuff -> stuff.length()

The Repository class in Listing 2 has a single method, getStuff, that would presumably connect to a database, execute a query, and return the results. In this example, it simply returns a list of three Strings. The Service class in Listing 3 receives the Repository through its constructor and defines a single method, getStuffWithLengthLessThanFive, which returns all Strings with a length less than 5. If the repository throws an SQLException, then it returns an empty list.

Unit testing with JUnit 5 and Mockito

Now let’s look at how we can test our service using JUnit 5 and Mockito. Listing 4 shows the source code for a ServiceTest class.

Listing 4. Testing the service (ServiceTest.java)

package com.javaworld.geekcap.mockito;

import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.InjectMocks;
import org.mockito.Mock;
import org.mockito.Mockito;
import org.mockito.junit.jupiter.MockitoExtension;

import java.sql.SQLException;
import java.util.Arrays;
import java.util.List;

@ExtendWith(MockitoExtension.class)
class ServiceTest {
@Mock
Repository repository;

@InjectMocks
Service service;

@Test
void testSuccess() {
// Setup mock scenario
try {
Mockito.when(repository.getStuff()).thenReturn(Arrays.asList('A', 'B', 'CDEFGHIJK', '12345', '1234'));
} catch (SQLException e) {
e.printStackTrace();
}

// Execute the service that uses the mocked repository
List stuff = service.getStuffWithLengthLessThanFive();

// Validate the response
Assertions.assertNotNull(stuff);
Assertions.assertEquals(3, stuff.size());
}

@Test
void testException() {
// Setup mock scenario
try {
Mockito.when(repository.getStuff()).thenThrow(new SQLException('Connection Exception'));
} catch (SQLException e) {
e.printStackTrace();
}

// Execute the service that uses the mocked repository
List stuff = service.getStuffWithLengthLessThanFive();

// Validate the response
Assertions.assertNotNull(stuff);
Assertions.assertEquals(0, stuff.size());
}
}

The first thing to notice about this test class is that it is annotated with @ExtendWith(MockitoExtension.class). The @ExtendWith annotation is used to load a JUnit 5 extension. JUnit defines an extension API, which allows third-party vendors like Mockito to hook into the lifecycle of running test classes and add additional functionality. The MockitoExtension looks at the test class, finds member variables annotated with the @Mock annotation, and creates a mock implementation of those variables. It then finds member variables annotated with the @InjectMocks annotation and attempts to inject its mocks into those classes, using either construction injection or setter injection.

In this example, MockitoExtension finds the @Mock annotation on the Repository member variable, so it creates a mock implementation and assigns it to the repository variable. When it discovers the @InjectMocks annotation on the Service member variable, it creates an instance of the Service class, passing the mock Repository to its constructor. This allows us to control the behavior of the mock Repository class using Mockito’s APIs.

In the testSuccess method, we use the Mockito API to return a specific result set when its getStuff method is called. The API works as follows:

First, the Mockito::when method defines the condition, which in this case is the invocation of the repository.getStuff() method.

Then, the when() method returns an org.mockito.stubbing.OngoingStubbing instance, which defines a set of methods that determine what to do when the specified method is called.

Finally, in this case, we invoke the thenReturn() method to tell the stub to return a specific List of Strings.

At this point, we have a Service instance with a mock Repository. When the Repository’s getStuff method is called, it returns a list of five known strings. We invoke the Service’s getStuffWithLengthLessThanFive() method, which will invoke the Repository’s getStuff() method, and return a filtered list of Strings whose length is less than five. We can then assert that the returned list is not null and that the size of it is three. This process allows us to test the logic in the specific Service method, with a known response from the Repository.

The testException method configures Mockito so that when the Repository’s getStuff() method is called, it throws an SQLException. It does this by invoking the OngoingStubbing object’s thenThrow() method, passing it a new SQLException instance. When this happens, the Service should catch the exception and return an empty list.

Mocking is powerful because it allows us to simulate scenarios that would otherwise be difficult to replicate. For example, you may invoke a method that throws a network or I/O error and write code to handle it. But unless you turn off your WiFi or disconnect an external drive at the exact right moment, how do you know the code works? With mock objects, you can throw those exceptions and prove that your code handles them properly. With mocking, you can simulate rare edge cases of any type.

Introduction to Mockito spies

In addition to mocking the behavior of classes, Mockito allows you to verify their behavior. Mockito provides “spies” that watch an object so you can ensure specific methods are called with specific values. For example, you may want to ensure that, when you call a service, it makes a specific call to a repository. Or you might want to ensure that it does not call the repository but rather loads the item from a cache. Using Mockito spies lets you not only validate the response of a method call but ensure the method does what you expect.

This may seem a little abstract, so let’s start with a simple example that works with a list of Strings. Listing 5 shows a test method that adds two Strings to a list and then checks the size of the list after each addition. We’ll then verify that the list’s add() method is called for the two Strings, and that the size() method is called twice.

Listing 5. Testing a List with spies (SimpleSpyTest.java)

package com.javaworld.geekcap.mockito;

import static org.mockito.Mockito.atLeastOnce;
import static org.mockito.Mockito.never;
import static org.mockito.Mockito.times;
import static org.mockito.Mockito.verify;

import java.util.ArrayList;
import java.util.List;

import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.Spy;
import org.mockito.junit.jupiter.MockitoExtension;

@ExtendWith(MockitoExtension.class)
public class SimpleSpyTest {
@Spy
List stringList = new ArrayList();

@Test
public void testStringListAdd() {
// Add an item to the list and verify that it has one element
stringList.add('One');
Assertions.assertEquals(1, stringList.size());

// Add another item to the list and verify that it has two elements
stringList.add('Two');
Assertions.assertEquals(2, stringList.size());

// Verify that add was called with arguments 'One' and 'Two'
verify(stringList).add('One');
verify(stringList).add('Two');

// Verify that add was never called with an argument of 'Three'
verify(stringList, never()).add('Three');

// Verify that the size() method was called twice
verify(stringList, times(2)).size();

// Verify that the size() method was called at least once
verify(stringList, atLeastOnce()).size();
}
}

Listing 5 starts by defining an ArrayList of Strings and annotates it with Mockito’s @Spy annotation. The @Spy annotation tells Mockito to watch and record every method called on the annotated object. We add the String 'One' to the list, assert that its size is 1, and then add the String 'Two' and assert that its size is 2.

After we do this, we can use Mockito to verify everything we did. The org.mockito.Mockito.verify() method accepts a spied object and returns a version of the object that we can use to verify that specific method calls were made. If those methods were called, then the test continues, and if those methods were not called, the test case fails. For example, we can verify that add('One') was called as follows:

If the String list’s add() method is called with an argument of 'One' then the test continues to the next line, and if it’s not the test fails. After verifying 'One' and 'Two' are added to the list, we verify that add('Three') was never called by passing an org.mockito.verification.VerificationMode to the verify() method. VerificationModes validate the number of times that a method is invoked, with whatever arguments are specified, and include the following:

times(n): specifies that you expect the method to be called n times.

never(): specifies that you do not expect the method to be called.

atLeastOnce(): specifies that you expect the method to be called at least once.

atLeast(n): specifies that you expect the method to be called at least n times.

atMost(n): specifies that you expect the method to be called at most n times.

Knowing this, we can verify that add('Three') is not called by executing the following:

verify(stringList, never()).add('Three')

It’s worth noting that when we do not specify a VerificationMode, it defaults to times(1), so the earlier calls were verifying that, for example, the add('One') was called once. Likewise, we verify that size() was invoked twice. Then, just to show how it works, we also verify that it was invoked at least once.

Now let’s test our service and repository from Listings 2 and 3 by spying on the repository and verifying the service calls the repository’s getStuff() method. This test is shown in Listing 6.

Listing 6. Testing a spied mock object (SpyAndMockTest.java)

package com.javaworld.geekcap.mockito;

import static org.junit.jupiter.api.Assertions.fail;
import static org.mockito.Mockito.spy;
import static org.mockito.Mockito.verify;

import java.sql.SQLException;
import java.util.Arrays;
import java.util.List;

import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.InjectMocks;
import org.mockito.Mock;
import org.mockito.Mockito;
import org.mockito.junit.jupiter.MockitoExtension;

@ExtendWith(MockitoExtension.class)
public class SpyAndMockTest {
// Create a Mock of a spied Repository
@Mock
Repository repository = spy(new Repository());

// Inject the respository into the service
@InjectMocks
Service service;

@Test
public void verifyRepositoryGetStuffIsCalled() {
try {
// Setup mock scenario
Mockito.when(repository.getStuff()).thenReturn(Arrays.asList('A', 'B', 'CDEFGHIJK', '12345', '1234'));
} catch (SQLException e) {
fail(e.getMessage());
}

// Execute the service that uses the mocked repository
List stuff = service.getStuffWithLengthLessThanFive();

// Validate the response
Assertions.assertNotNull(stuff);
Assertions.assertEquals(3, stuff.size());

try {
// Verify that the repository getStuff() method was called
verify(repository).getStuff();
} catch (SQLException e) {
fail(e.getMessage());
}
}
}

Most of the code in Listing 6 is the same as the test we wrote in Listing 4, but with two changes to support spying. First, when we want to spy on a mocked object, we cannot add both the @Mock and @Spy annotations to the object because Mockito only supports one of those annotations at a time. Instead, we can create a new repository, pass it to the org.mockito.Mockito.spy() method, and then annotate that with the @Mock annotation. The @Spy annotation is shorthand for invoking the spy() method, so the effect is the same. Now we have a mock object we can use to control the behavior, but Mockito will spy on all its method calls.

Next, we use the same verify() method we used to verify that add('One') was called to now verify the getStuff() method is called:

We need to wrap the method call in a try-catch block because the method signature defines that it can throw an SQLException, but since it doesn’t actually call the method, we would never expect the exception to be thrown.

You can extend this test with any of the VerificationMode variations I listed earlier. As a practical example, your service may maintain a cache of values and only query the repository when the requested value is not in the cache. If you mocked the repository and cache and then invoked a service method, Mockito assertions would allow you to validate that you got the correct response. With the proper cache values, you could infer that the service was getting the value from the cache, but you couldn’t know for sure. Spies, on the other hand, allow you to verify absolutely that the cache is checked and that the repository call is never made. So, combining mocks with spies allows you to more fully test your classes.

Mockito is a powerful tool, and we’ve only scratched the surface of what it can do. If you’ve ever wondered how you can test abhorrent conditions—such as network, database, timeout, or other I/O error conditions—Mockito is the tool for you. And, as you’ve seen here, it works elegantly with JUnit 5.

Conclusion

This article was a deeper exploration of JUnit 5’s unit testing capabilities, involving its integration with external tools. You saw how to integrate and use JUnit 5 with Hamcrest, a more advanced assertions framework, and Mockito, which you can use to mock and control a class’s dependencies. We also looked at Mockito’s spying capabilities, which you can use to not only test the return value of a method but also verify its behavior.

At this point, I hope you feel comfortable writing unit tests that not only test happy-path scenarios but also leverage tools like Mockito, which let you simulate edge cases and ensure your applications handle them correctly.