Skip to content

Fix e2e test flakiness: wait for the preview to be ready before pressing Play - #4400

Open
clairep94 wants to merge 2 commits into
developfrom
clairep/fix_e2e_test_flake
Open

clairep94 wants to merge 2 commits into
developfrom
clairep/fix_e2e_test_flake

Conversation

@clairep94

@clairep94 clairep94 commented Oct 31, 2026 •

Copy link
Copy Markdown
Collaborator

Issue:

Fixes #4284 P2

The e2e test sketches_management.spec.ts › "can rename a sketch and the update persists" fails on CI now and then:

Expected substring: "updated log"
Received string:    ""

The sketch's code is saved and loaded correctly, and Play is clicked, but the sketch never runs. The console stays empty for the whole timeout.

Cause: a race between clicking Play and the preview iframe loading. Play sends the sketch to the preview iframe with a one-way postMessage. There's no acknowledgement and no retry. The preview is a separate app that only starts handling messages after it mounts. This test does a full page reload, opens the sketch, and clicks Play straight away. If Play lands before the preview is listening, the message is dropped and nothing runs.

The test's existing check, toHaveAttribute('src', /9002/), doesn't prevent this. The iframe's src is set immediately, so the check passes before the preview has loaded.

This is also why #4343 doesn't fix the flake. Adding not.toBeEmpty() only waits longer. The sketch was never started, so the console never fills.

Demo:

Changes:

  • New e2e/helpers/preview.ts with a waitForPreviewReady(page) helper. It uses the same handshake FullView.jsx already relies on: it posts REGISTER to the preview iframe every 100ms until the preview echoes REGISTER back. The preview only echoes once its message listener is attached, so after this wait a Play click will be received. The IDE console ignores these messages, so they have no side effects.
  • sketches_management.spec.ts and signup.spec.ts: call waitForPreviewReady(page) before every #play-sketch click, and remove the src checks, which didn't prove anything.

NOTE:

Follow-up (separate issue): this PR only fixes the test. The race behind it is a real bug for users: on a slow connection, clicking Play right after opening a sketch can silently do nothing, because the preview iframe hasn't started listening yet. The follow-up PR will have the preview send a READY message once it is listening. If the editor is in a playing state when that message arrives, it re-sends the sketch. With that change, Play works even when it's clicked before the preview loads, and waitForPreviewReady becomes a safeguard rather than a requirement.

I have verified that this pull request:

  • has no linting errors (npm run lint)
  • has no test errors (npm run test)
  • has no typecheck errors (npm run typecheck)
  • is from a uniquely-named feature branch and is up to date with the develop branch.
  • is descriptively named and links to an issue number, i.e. Fixes #123
  • meets the standards outlined in the accessibility guidelines

@clairep94 clairep94 changed the title wait for the preview to be ready before pressing Play Fix e2e test flakiness: wait for the preview to be ready before pressing Play Oct 31, 2026
@clairep94
clairep94 marked this pull request as ready for review October 31, 2026 11:21
@clairep94
clairep94 requested a balanced review from Copilot October 31, 2026 11:21

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues remain for the change to wait for preview acknowledgement before clicking Play.

No open issues 🎉

What changed in this PR

This PR addresses intermittent end-to-end test failures by waiting for the preview app to acknowledge messages before clicking Play, rather than relying on the iframe’s src attribute.

Changes:

  • Adds a reusable waitForPreviewReady helper that retries a REGISTER handshake until acknowledged or timed out.
  • Uses the helper before Play clicks in sketch-management and signup tests.
  • Removes iframe URL checks and redundant console-not-empty assertions while retaining expected-log assertions.
File Description
e2e/​specs/​sketches_management.spec.ts Waits for preview readiness before running initial and renamed sketches.
e2e/​specs/​signup.spec.ts Waits for preview readiness before running sketches before and after signup.
e2e/​helpers/​preview.ts Adds the bounded readiness handshake with timer and listener cleanup.

Review effort: Balanced

@Vaivaswat2244 Vaivaswat2244 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi! This looks good to me.
Although, I saw something similar in editor.spec.ts
https://github.com/processing/p5.js-web-editor/blob/develop/e2e/specs/editor.spec.ts#L62 , L260, L474. which also the same races, I think maybe we should apply waitForPreviewReady there as well...

This branch was successfully deployed

1 active deployment
e2e-tests — 181bd1f0 Deployed Oct 31, 2026 by clairep94 via test-e2e #448
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Make e2e tests mandatory for PR merging

3 participants