Race condition with warnings issued once directly and once inside pytest.warns

[lesteve]

I bumped into a slighty surprising behaviour in scikit-learn and it took me a bit of time to realize that the root cause is that the same warning is issued twice, once directly in the test function and once inside a pytest.warns context. There appears to be some race condition that makes the test fail when run with enough iterations. Not sure pytest-run-parallel can do much about it but I thought I would report it here first.

Wild-guessing, maybe per-module __warningregistry__ is still a global (in contrary the per-thread warnings.filters since Python 3.14 and python/cpython#130010)? My understanding is that per-module __warningregistry__ is used on top of the warnings.filter to decide whether to issue repeated warnings.

Here is a reproducer:

# test.py
import warnings
import pytest

import my_module


def test_pytest_warns():
    my_module.function_that_warns()

    with pytest.warns(UserWarning):
        my_module.function_that_warns()

# my_module.py
import warnings

def function_that_warns():
    warnings.warn('bla')

pytest test.py --parallel-threads 4 --iterations 100

Output:

============================================================================================================= test session starts ==============================================================================================================
platform linux -- Python 3.14.2, pytest-9.0.1, pluggy-1.6.0
rootdir: /tmp
plugins: run-parallel-0.8.0, cov-7.0.0, repeat-0.9.4, xdist-3.8.0
collected 1 item                                                                                                                                                                                                                               
Collected 1 items to run in parallel

test.py e                                                                                                                                                                                                                                [100%]

==================================================================================================================== ERRORS ====================================================================================================================
______________________________________________________________________________________________________ ERROR at call of test_pytest_warns ______________________________________________________________________________________________________

    def test_pytest_warns():
        my_module.function_that_warns()
    
>       with pytest.warns(UserWarning):
             ^^^^^^^^^^^^^^^^^^^^^^^^^
E       Failed: DID NOT WARN. No warnings of type (<class 'UserWarning'>,) were emitted.
E        Emitted warnings: [].

test.py:10: Failed
=============================================================================================================== warnings summary ===============================================================================================================
test.py: 286 warnings
  /tmp/my_module.py:4: UserWarning: bla
    warnings.warn('bla')

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
********************************************************************************************************** pytest-run-parallel report **********************************************************************************************************
All tests were run in parallel! 🎉
=========================================================================================================== short test summary info ============================================================================================================
PARALLEL FAILED test.py::test_pytest_warns - Failed: DID NOT WARN. No warnings of type (<class 'UserWarning'>,) were emitted.
======================================================================================================== 286 warnings, 1 error in 0.07s ========================================================================================================

Possible work-arounds:

• make sure to use with pytest.warns to capture the first warning as well
• mark test as thread-unsafe
• use --mark-warnings-as-unsafe

[ngoldbaum]

Ping @nascheme - do you happen to have some context about __warningregistry__?

To give a little extra context - the test in the issue passes every time on my machine with --iterations=2, but starts failing randomly with --iterations=4 and then around --iterations=8 will fail every time. So this is a race condition - I think in CPython?

[nascheme]

Wild-guessing, maybe per-module __warningregistry__ is still a global

Yes, it's still a per-module global.

The registry is supposed to get cleared every time the filters change. Code in the Python implementation:

    key = (text, category, lineno)
    with _wm._lock:
        if registry is None:
            registry = {}
        if registry.get('version', 0) != _wm._filters_version:
            registry.clear()
            registry['version'] = _wm._filters_version
        # Quick test for common case
        if registry.get(key):
            return

The global warnings lock is held when this code runs. However, I think that doesn't prevent a race like the following:

1. thread 1: update filters (and filters version)
2. thread 2: implicitly call warn_explicit(). Since the filters version changed, clear registry.
3. thread 2: generate warning, it will be stored in registry to avoid re-warning
4. thread 1: generate warning that is expected to be emitted

If 3 and 4 steps are swapped, you either get or don't get a warning.