[Bug]: NetworkIdle is permanently lost when later requests finish before DOMContentLoaded #42598

Summary

After Playwright has reported an initial networkidle during parsing, requests that start and finish before DOMContentLoaded can leave the frame with neither a recorded networkidle lifecycle state nor a running idle timer. A subsequent WaitForLoadStateAsync(LoadState.NetworkIdle) times out even though there are no outstanding requests.

This reproduces on genuine bundled Chromium, not just a third-party CDP implementation. All four responses are HTTP 200, all four response bodies remain readable, every Network.requestWillBeSent ID has exactly one terminal event, and the current frame/loader emits CDP Page.lifecycleEvent(networkIdle).

Related: #19835 was closed without a reproducible case, with maintainers requesting a new report and logs. This investigation originated in sebastienros/jint#3883; the Chromium control rules out a Jint-specific protocol-loss explanation.

Versions / environment

• Microsoft.Playwright NuGet and its bundled Node driver: 1.62.0
• Playwright v1.62.0 source commit: e3950d9
• Bundled Chromium revision 1234, browser 151.0.7922.34
• macOS arm64; .NET 8.0.30 and 10.0.11
• Independent second engine: Jint at 43f6f8931af867f3686d507a5c2d1c6ec328e6ea, without source changes

Complete standalone reproduction

Runnable two-engine .NET project, instructions, and Chromium/Jint logs

The gist includes Program.cs, NetworkIdleRepro.csproj, README.md, and logs for the failing case and its negative control. Chromium mode has no Jint dependency. It starts the browser from playwright.Chromium.ExecutablePath with an ephemeral CDP port and attaches using the public ConnectOverCDPAsync API. The optional Jint mode runs exactly the same client sequence.

Save the two source files together, then:

dotnet build -c Release -f net8.0
# Only if the matching browser is not installed:
pwsh bin/Release/net8.0/playwright.ps1 install chromium

DEBUG=pw:api dotnet run -c Release -f net8.0 -- chromium
DEBUG=pw:api dotnet run -c Release -f net10.0 -- chromium

The program prints BUG REPRODUCED only after asserting the complete request correlation, readable bodies, rendered result, no page errors, and the final timeout. Exit code zero means the bug was reproduced, not that the wait succeeded.

Deterministic ordering, without sleeps

1. /page loads /app.js. A debugger statement holds the HTML parser after both bodies have finished. Await Playwright's real first NetworkIdle; it resolves using the unchanged 500 ms timer.
2. Resume. The script starts /schema.json and uses document.write to insert parser-blocking /finish.js. The origin releases /finish.js only after the public Playwright RequestFinished event for the schema.
3. A second debugger statement holds the parser until all four public request-completion events have arrived. Resume parsing; DOMContentLoaded and load fire, and the fetched GetEndpoint value is rendered.
4. Read all four response bodies, correlate the independent CDP trace, and await the current document's CDP networkIdle. A new public WaitForLoadStateAsync(NetworkIdle) still times out.

The debugger pauses replace a naturally slow parser-blocking script with explicit gates. No lifecycle event is simulated, no request is suppressed, and no idle interval is changed. The reproduction uses only a local origin. There are no redirects, disk-cache hits, service-worker responses, or failed requests; all requests name the committed frame and loader.

Negative control

Append --post-dcl:

DEBUG=pw:api dotnet run -c Release -f net8.0 -- chromium --post-dcl

This releases /finish.js immediately and instead holds the schema response until the public DOMContentLoaded event. Thus the last request finishes after DOMContentLoaded. Everything else is unchanged, and NetworkIdle resolves (CONTROL PASSED).

All runs were sequential. The Chromium-only project was also built and run without the optional Jint project reference.

Representative Chromium timeline

Times are elapsed milliseconds from the standalone run; full logs and IDs are in the gist.

  90  document request finished
  97  app.js request finished; parser paused
 599  FIRST public Playwright NetworkIdle resolved
 601  schema.json and finish.js requests started
 603  schema.json request finished
 605  finish.js request finished; parser paused
 606  resume -> DOMContentLoaded and load
1478  current document's CDP Page.lifecycleEvent(networkIdle)
1481  all four CDP IDs paired; all four bodies already read successfully
4484  final public Playwright NetworkIdle timed out

Owning driver state transition

The relevant owner is packages/playwright-core/src/server/frames.ts, at the pinned source commit above:

• _inflightRequestStarted / _inflightRequestFinished
• onLifecycleEvent
• _recalculateNetworkIdle
• _startNetworkIdleTimer / _stopNetworkIdleTimer

The sequence is:

1. The first idle sets _firedNetworkIdleSelf = true and adds networkidle to _firedLifecycleEvents.
2. A new request calls _stopNetworkIdleTimer(), clearing _firedNetworkIdleSelf but leaving the lifecycle entry present.
3. The last request finishes. _startNetworkIdleTimer() returns early because _firedLifecycleEvents.has('networkidle') is still true. No replacement timer is armed.
4. DOMContentLoaded calls onLifecycleEvent(), which calls _recalculateNetworkIdle() without an argument. Its removal condition frameThatAllowsRemovingNetworkIdle !== this is true; self-idle is false, so it removes the networkidle lifecycle entry.
5. The frame now has an empty inflight set, no recorded idle, and no timer. With no subsequent request or navigation, the wait cannot recover.

The negative control changes step 4 to happen before step 3, so the final request completion can arm the timer.

Chromium's _onLifecycleEvent forwards only load and DOMContentLoaded. Receiving CDP networkIdle does not repair this driver-maintained state, explaining why both browsers can truthfully report idle while the public wait remains stuck.

Expected behavior

Once all tracked requests have completed, the frame must not remain permanently unable to report NetworkIdle solely because another lifecycle event arrived after the last completion. Whichever lifecycle semantics are intended, removing a previously reported idle must not leave an idle frame without a way to reach the state again.

No compensating changes were made to Jint's protocol or lifecycle events.