Skip to content

Add pytest main CI smoke test#1431

Open
Herrtian wants to merge 2 commits intopytest-dev:mainfrom
Herrtian:add-pytest-dev-ci
Open

Add pytest main CI smoke test#1431
Herrtian wants to merge 2 commits intopytest-dev:mainfrom
Herrtian:add-pytest-dev-ci

Conversation

@Herrtian
Copy link
Copy Markdown

@Herrtian Herrtian commented May 1, 2026

Fixes #1132.

This adds a separate non-blocking CI job that installs pytest from the pytest main branch and runs the existing test target against it. The regular tox matrix stays unchanged.

@Herrtian
Copy link
Copy Markdown
Author

Herrtian commented May 1, 2026

The failing timeline check is for the missing changelog fragment. Since this is only adding a CI smoke test and doesn't change runtime behavior or docs, I think this should probably use the skip-changelog label rather than adding a release note.

Copy link
Copy Markdown
Contributor

@seifertm seifertm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the initiative @Herrtian !

It's desirable to have parity between the CI/CD workflow and the local development environment. That means, it should be possible to run the tests against pytest dev in the local development environment.

There was another Pull Request which seems like it had this figured out. Can you take a look and extend the local development environment so that there's a tox environment that runs all tests against the development version of pytest?

Comment thread .github/workflows/main.yml Outdated
if-no-files-found: error

test-pytest-dev:
name: ubuntu - Python ${{ env.PYTHON_LATEST }} - pytest main
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The workflow run fails and complains about env not being found. I assume the reason is that env just isn't available here. We can just use "latest", instead of the variable value.

@codecov-commenter
Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.50%. Comparing base (31772d2) to head (09a6a37).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1431   +/-   ##
=======================================
  Coverage   94.50%   94.50%           
=======================================
  Files           2        2           
  Lines         510      510           
  Branches       62       62           
=======================================
  Hits          482      482           
  Misses         22       22           
  Partials        6        6           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@Herrtian
Copy link
Copy Markdown
Author

Herrtian commented May 2, 2026

Thanks, updated this in 09a6a37. The job name now uses a literal latest, and the CI job runs a new local tox env, pytest-dev, so the same pytest-main check can be run outside CI too. I checked the tox config locally and ran the env against pytest main. It currently surfaces a few strict-mode behavior failures from pytest main, so I kept the CI job non-blocking as originally intended.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CI: add job that that uses pytest-dev version

3 participants