Flaky Frontend Tests?

[ZeroWiggliness]

Hi,

I wasn't sure where to ask or if I should open a bug. Are the tests flaky? We get a lot of test timeouts and we see similar sometimes in the failed actions. We are struggling to get all the tests to pass because of numerous timeouts. Some of these tests we see also failed in the actions here in the repo.

We have tried npm run tests and also similar to the github build machines with:

docker buildx build -t test01042026 --cache-from=type=registry,ref=apache/superset-cache:3.10-slim-bookworm --target superset-node-ci .

mkdir -p superset-frontend/coverage

docker run -v ./superset-frontend/coverage:/app/superset-frontend/coverage --rm test01042026:latest bash -c "npm run test -- --coverage --coverageReporters=json-summary"
`
and we get very similar issues but not always the same. Sometimes they all pass if we use sharding but not always, without we always get failures.

For reference we are using the 6.0 branch.

Example:

FAIL src/dashboard/components/EmbeddedModal/EmbeddedModal.test.tsx (101.488 s)
● enables embedding

thrown: "Exceeded timeout of 20000 ms for a test.
Add a timeout value to this test to increase the timeout, if this is a long-running test. See https://jestjs.io/docs/api#testname-fn-timeout."

  105 | });
  106 |
> 107 | test('enables embedding', async () => {
      | ^
  108 |   setMockApiNotFound();
  109 |   setup();
  110 |

  at Object.test (src/dashboard/components/EmbeddedModal/EmbeddedModal.test.tsx:107:1)

Any solution for this would be great.

[dosubot[bot]]

Hi @ZeroWiggliness! I'm Dosu and I’m helping the superset team.

Yes, frontend test flakiness (especially timeouts) is a well-documented problem in Superset. The team has been actively fixing these across multiple PRs. Here's what's going on and what you can try:

Root cause for most timeouts: The most common culprit is unmocked API endpoints — when a component makes an API call on mount but the test doesn't mock that endpoint, Jest hangs waiting for a response until it hits the 20-second timeout [1]. The global timeout is set to 20000ms in jest.config.js [2].

Specifically for EmbeddedModal.test.tsx: There's a race condition in the test setup where resetMockApi() is called after render(), but the component's useEffect fires during render — meaning API calls can execute before mocks are configured [3]. Moving resetMockApi() before render() in the setup function should help.

Why sharding helps: The CI uses 8-way Jest sharding to parallelize tests, which reduces memory pressure and timer pollution between tests — both known causes of intermittent failures.

What the team has been fixing recently:

• PR #36037: Fixed React Testing Library anti-patterns (redundant double-awaits, userEvent inside waitFor callbacks)
• PR #35993: Fixed 30+ unawaited userEvent calls and missing API mocks
• PR #38430: Fixed OOM crashes in tests by mocking heavy child components (reduced heap from 615MB to 287MB)

Quick things you can try:

1. Use sharding like CI does: npm run test -- --shard=1/8, --shard=2/8, etc.
2. Increase --maxWorkers or decrease it (e.g., --maxWorkers=2) to reduce memory contention
3. For specific failing tests, increase the timeout as a workaround: test('name', async () => { ... }, 60000)
4. Ensure proper cleanup in afterEach — call jest.runOnlyPendingTimers() before jest.useRealTimers() to avoid timer pollution [1]

Modal components in particular are prone to this kind of flakiness — FiltersConfigModal.test.tsx was actually skipped after 11 fix attempts due to complex Redux + API mock coordination issues.

Since you're on the 6.0 branch, you may not have all the recent flakiness fixes. It might be worth cherry-picking or checking if the fixes from the PRs above apply to your branch.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}
   

[rsmihir389]

Flaky timeout failures in CI are really common in large frontend test suites — you're definitely not alone. The root cause is usually one of two things: runner resource contention (slower runner = more timeouts) or test isolation issues where shared state makes tests order-dependent.

One thing that helps is analyzing your CI history to see which specific tests are flaky vs. consistently slow. If a test fails 30% of runs but passes on retry, that's a different problem than a test that consistently times out.

I built Kleore specifically to surface this — it connects to GitHub, analyzes your CI run history, and ranks tests by failure rate and cost. Free tier shows you the full picture. Might help identify which tests are genuinely flaky vs. just slow.