Radu Ghiorghisor logo
Radu Ghiorghisor

When mocks break the build

Published on

You’re running your Maven build, all your tests pass successfully, but suddenly the build fails with a cryptic error:

org.apache.maven.surefire.booter.SurefireBooterForkException: 
There was an error in the forked process

Looking at the dump file, you find:

java.lang.NullPointerException
	at java.lang.Throwable.printEnclosedStackTrace(Throwable.java:703)
	at org.testng.reporters.TestHTMLReporter.generateTable(TestHTMLReporter.java:168)
	at org.testng.reporters.TestHTMLReporter.onFinish(TestHTMLReporter.java:37)
    ...

Your tests didn’t fail, the test reporter did. Welcome to just one of the many frustrating edge cases in Java testing: mocked exceptions breaking the reporting phase.

TL;DR

If you mock exceptions (e.g. mock(MyException.class)), TestNG’s HTML reporter may throw an NPE after all tests finish, causing Maven to fail the build.

Fix: Never mock exceptions. Always create real ones.

Why the build fails after tests succeed

When a test throws an exception, expected or not, TestNG stores it in order to generate the HTML reports afterward. During this reporting phase, to print the exception it calls printStackTrace(). And that’s where things break.

Mockito generated exception mocks are incomplete Throwable objects:

  • Missing or null stacktraces;
  • Null suppressed-exception lists;
  • Incomplete cause chains;
  • Internal state Throwable assumes must be initialized.

When TestNG tries to walk through these structures in printEnclosedStackTrace(), it hits a null and throws an NPE, killing the forked JVM.

How exceptions work internally

Java’s java.lang.Throwable class maintains a complex internal structure:

public class Throwable {
    private Throwable cause = this;
    private StackTraceElement[] stackTrace;
    private List<Throwable> suppressedExceptions;
    ...
    
    private void printEnclosedStackTrace(...) { 
        ... 
    }
}

When printing a stacktrace, Java recursively walks through:

  1. The exception itself;
  2. Its cause (if any);
  3. Any suppressed exceptions;
  4. The causes of those suppressed exceptions;
  5. And so on…

The mock problem

A typical problematic test:

@Test(expectedExceptions = MockedExceptionClass.class)
public void testErrorHandling() {
    MockedExceptionClass mockException =  mock(MockedExceptionClass.class);
    when(mockedDelegateService.doSomething()).thenThrow(mockException);
    
    service.callMethod();
}

The mocked exception often has:

  • Null or incomplete stacktraces;
  • Null suppressed exceptions array (instead of empty list);
  • Improperly initialized cause chain;
  • Missing internal state** that Throwable expects.

Why it breaks during reporting

The HTML reporter calls printStackTrace() on caught exceptions to generate pretty HTML reports. Here’s what happens:

When printEnclosedStackTrace() tries to iterate over a mocked exception’s internal structures, it encounters null references where it expects initialized (even if empty) collections or objects.

The insidious part: all your tests pass. The failure happens in the onFinish() phase:

Test Execution Phase:
  ✓ Test 1 passes
  ✓ Test 2 passes  
  ✓ Test 102 passes
  
Reporting Phase:
  ✗ TestNG tries to generate HTML report
  ✗ NPE when printing mock exception stack trace
  ✗ Forked JVM crashes
  ✗ Maven reports build failure

This is particularly frustrating because:

  • Your tests are correct;
  • Your code is correct;
  • The failure is in infrastructure, not logic;
  • It only happens in specific test environments (CI/CD with certain reporters enabled);
  • It often surfaces only in CI/CD pipelines, not locally, because many IDEs run tests without HTML reporting enabled by default.

Solutions

The best fix is simple: don’t mock exceptions. They’re data objects, not collaborators.

@Test(expectedExceptions = MockedExceptionClass.class)
public void testErrorHandling() {
    MockedExceptionClass mockException =  new MockedExceptionClass();
    when(mockedDelegateService.doSomething()).thenThrow(mockException);
    
    service.callMethod();
}

Why this works:

  • Real exceptions have properly initialized internal structures;
  • Stack traces are valid (even if short in tests);
  • No NPE during reporting;
  • More realistic test behavior.

Solution 2: Disable TestNG HTML reporter

If you can’t fix all mocked exceptions immediately, disable the problematic reporter:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>3.0.0-M7</version>
    <configuration>
        <properties>
            <property>
                <name>usedefaultlisteners</name>
                <value>false</value>
            </property>    
        </properties>
    </configuration>
</plugin>

Trade-offs:

  • Build won’t fail;
  • Quick fix;
  • Lose HTML reports;
  • Doesn’t address root cause.

Solution 3: Upgrade TestNG

Later versions of TestNG (7.8+) have better null-safety in reporters:

<dependency>
    <groupId>org.testng</groupId>
    <artifactId>testng</artifactId>
    <version>7.10.2</version>
    <scope>test</scope>
</dependency>

Considerations:

  • Better long-term solution;
  • Many other bug fixes;
  • May require test compatibility changes;
  • Still better to use real exceptions.

Solution 4: Custom exception builder (Advanced)

For complex exception scenarios where you need specific control over exception creation (note: Solutions 1-3 cover probably 99% of cases):

public class TestExceptionBuilder {
    public static <T extends Throwable> T buildTestException(
            Class<T> exceptionClass, 
            String message) {
        try {
            T exception = exceptionClass
                .getConstructor(String.class)
                .newInstance(message);
            
            exception.fillInStackTrace();
            return exception;
            
        } catch (Exception e) {
            throw new RuntimeException("Cannot create test exception", e);
        }
    }
}

// Usage
@Test
public void testWithProperException() {
    MockedExceptionClass mockException = TestExceptionBuilder.buildTestException(
            MockedExceptionClass.class,
            "Test error"
        );
    when(mockedDelegateService.doSomething()).thenThrow(mockException);
    
    service.callMethod();
}

Best Practices

1. Never mock value objects

Exceptions, like DTOs and entities, are value objects. Mock collaborators, not data:

// Don't
mock(UserNotFoundException.class)
mock(ValidationException.class)
mock(IllegalArgumentException.class)

// Do
new UserNotFoundException("User not found")
new ValidationException("Invalid input")
new IllegalArgumentException("Null parameter")

2. Use exception factories in tests

Create a test utility for consistent exception creation:

public class TestExceptions {
    public static UserNotFoundException 
            userNotFoundException(String message) {
        return new UserNotFoundException(message);
    }
    
    public static ValidationException 
            validationException(String message) {
        return new ValidationException(message);
    }
}

3. Test exception chains explicitly

If your code wraps exceptions, test that explicitly:

@Test
public void testExceptionChaining() {
    UserNotFoundException cause = new UserNotFoundException("Root cause");
    
    when(service.doSomething()).thenThrow(cause);
    
    ValidationException thrown = assertThrows(
        ValidationException.class,
        () -> service.updateUser("testUser")
    );
    
   
    assertThat(thrown.getCause())
        .isInstanceOf(UserNotFoundException.class)
        .hasMessage("Root cause");
}

The Deeper Lessons

This issue teaches us important principles:

1. Infrastructure matters

Your tests can be logically correct but still break the build due to infrastructure assumptions. Test frameworks make assumptions about object structure and we should try to limit those assumptions.

2. Mocking has limits

Mocking frameworks are powerful, but mocking everything creates artificial test environments that can behave differently from production. Value objects should be real.

3. Fail-fast is better

A better design would have TestNG fail immediately when encountering an improperly initialized exception, rather than during reporting.

4. Test your test infrastructure

Consider testing that your test setup produces valid objects:

@Test
public void verifyTestExceptionsAreValid() {
    Exception e = TestExceptions.userNotFoundException("test");
    
    assertDoesNotThrow(() -> {
        StringWriter sw = new StringWriter();
        PrintWriter pw = new PrintWriter(sw);
        e.printStackTrace(pw);
    });
}

Conclusion

When you see error in the forked process combined with NPE in exception stack trace printing, suspect mocked exceptions. The fix is usually straightforward: use real exception instances instead of mocks.

This seemingly obscure bug represents a broader principle: mock behavior, not data. As exceptions are data, they are meant to be created, thrown, caught, and inspected. Mocking them creates incomplete objects that can pass your tests but fail during cleanup or reporting phases.

The next time your build fails after all tests pass, check your exception mocking practices. Your future self (and your CI/CD pipeline) will thank you.

Creative Commons License

This work is licensed under a Creative Commons Attribuition-ShareAlike 4.0 International License .