Skip to content

Support multithreaded test execution within test runner #6994

@bherw

Description

@bherw

Overview

I have approximately 5000 unit tests across ~100 submodules, running using JUnit Jupiter 6. The tests are configured for JUnit to run them in parallel, which it does using the ForkJoinPool. The tests are well-behaved, only writing to a RAM-backed tmpfs on per-test temporary directories in /tmp, and use dependency injection rather than static mocks, so they don't benefit much from the isolation provided by forking.

The service modules use Mockito for mocking. Mockito takes approximately 2 seconds to initialize, per test fork. With threading, this is manageable, but testForked is likely paying that cost multiple times.

Surprisingly, the default testParallelism provided by Mill actually slows down my test suite even on a domain model package.

Looking at the profiling results, it looks like with testParallelism=true, Mill parallelizes across forked JVMs rather than threads. While this is great for isolation, my tests don't need that level of isolation, so I seem to be paying a startup cost for no gain.

If this is not already the case, I'd love to be able to parallelize my tests using threads rather than forks.

JUnit Jupiter Configuration

All unit test modules have the following configuration, which enables JUnit to run both methods and classes in parallel, using the same number of threads as CPU cores:

junit.jupiter.execution.parallel.enabled = true
junit.jupiter.execution.parallel.config.dynamic.factor = 1
junit.jupiter.execution.parallel.mode.default = concurrent
junit.jupiter.execution.parallel.mode.classes.default = concurrent

Benchmarks

I ran some benchmarks to compare build systems, following the same protocol (5 runs to warm; median time of 3 runs) as https://mill-build.org/mill/comparisons/performance.html.

Tools

Processor: Intel Core i7-12800HX (24 cores)
Java: OpenJDK 25

Maven

Version: 3.9.9
Command: mvn test -T24

Mill

Version: 1.1.5
Command: ./mill <submodule>.test or ./mill __.test

Gradle

Version: 9.4.0
Command ./gradlew :<submodule>:test --rerun and ./gradlew test --rerun

Mill takes 5+ runs to warm the JVM and get full speed with testParallelism enabled, and is about half as fast until then. Gradle reaches full speed after 1 run, and is only about 50% slower on the first run.

Modules

Domain model module: 1266 unit tests, no Mockito usage.

Service module: 474 tests, Mockito used

Results

Build target Maven Mill (-j24, testParallelism=false) Mill (-j24 testParallelism=true Gradle
Domain model module 4.690s 7.493s 8.879s 1.652s
One module (uses Mockito) 6.326s 6.431s 10.413s 3.686s
Full project 93s 33.535s 44.748s 24.943s

While most of the test samples were quite close, Mill with testParallelism=true on the full project test run varied widely, from 41.924s to 61s.

The gradle numbers are similar to what I see in IntelliJ, running all tests in a submodule.

I also tried using testLocal with testParallelism=true. Unfortunately, my domain model module ran into an error with this mode (I'll raise another issue once I can isolate it), so I only have numbers for the service module, which ran in 6.731s.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions