Repository navigation
Test processes no longer pay 0.3 s per WPF STA thread at exit, which failed green Darling runs (#5596) - #5598
Merged
Merged
Conversation
Each STA test thread that touches a WPF weak-event manager leaves a ProcessExit handler that waits out a 300 ms dispatcher invoke on a dead thread. The WPF switch that makes the handler purge directly is now set in both test projects, with a test that times the handler and a pin on the project files. The diag workflow and module initializer stay for one more run.
erikdarlingdata
marked this pull request as ready for review
October 8, 2026 22:27
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #5596. A CI-only fix: no product code changed.
Why
The "Darling whole-tree guards" job runs the whole Darling.Tests suite in one process. After every test passed it sometimes printed "Waiting 10 seconds for foreground threads to exit..." and then "[FATAL ERROR] Foreground threads were left running, forcing process exit", exit code 1, 0 failed tests. Three of four Lite.Tests shards printed the wait line in green runs too and exited 1 to 3.6 seconds later.
No thread was left running. The runner only reports "the process was still alive 1 second after the run returned" and forces exit 10 seconds after that, so anything slow in the exit stage trips it.
What held the process
WPF's own weak-event exit handlers.
WeakEventManager(a boundDataGrid, say) makes WPF create aWeakEventTablefor that thread's dispatcher. The table registers aProcessExithandler (ShutDownListener.HandleShutDown, targetWeakEventTableShutDownListener) that nothing removes when the thread ends.WeakEventTable.OnShutDownsees that the table's own thread is gone, and callsDispatcher.Invoke(..., TimeSpan.FromMilliseconds(300))on a dispatcher nobody will ever run. It waits the full 300 ms, once per table.Evidence (the diagnostic workflow and module initializer were temporary and are deleted):
ColumnFilterPersistenceTests, 38 tests, 1.1 s) left 13 weak-event-table handlers. Timing each handler showed 13 x ~300 ms (about 4.0 s) inWeakEventTableShutDownListener, nothing else slow (the data-binding listeners, the column-filter flush and the module uninitializer each took 0 to 1 ms). The process printed the wait line and exited about 4 s after the summary.ProcessExithandler list (read through the runtime's event field) names the same handlers: 13 pairs ofWeakEventTableandDataBindEnginelisteners for that one class.ConsoleLifetime(.NET 10 uses signal handlers, not aProcessExithook). AlsoColumnFilterStore's exit flush (0 ms),ScratchPostgres's exit drain (no database in the guards job), and everynew Thread((all background or joined).ProcessExithandler does not work: the runtime has already taken its snapshot of the list. The fix has to be in place before the process starts, which is why it is a runtime configuration switch.What changes
Darling.Tests.csprojandLite.Tests.csprojset the WPF runtime switchSwitch.MS.Internal.DoNotInvokeInWeakEventTableShutdownListenertotrue. The handler then purges the table directly. Both test projects start WPF STA threads, so both get it (the same cost showed in the Lite shards).WpfExitSwitchTests(new, in Darling.Tests, linked into Lite.Tests so each process checks itself) has three checks. The switch is on in this process. The installedWindowsBase.dllstill contains the switch name, because a renamed switch would be a silent no-op. A table's own exit handler, picked out by the table it holds, returns in under 250 ms for a table whose STA thread ended. The assertion takes the best of three runs, and the old shape is 300 ms or more every time.WpfExitSwitchPinTests(new, guard stage): both project files declare the switch. Messages name The Darling whole-tree guards job fails when a foreground thread outlives the suite #5596 and the rule.build.yml: the new linked file is added to thelite_linked_shardfilter, asDarlingReadsTraitGuardTestsrequires for every file Lite.Tests compiles from Darling.Tests.Test plan
runtimeconfig.json,WpfExitSwitchTestsfails 2 of 3 (the switch check and the timing check). With the project files as onorigin/dev,WpfExitSwitchPinTestsfails 2 of 2.WpfExitSwitch*in Darling.Tests: Total 5 (3 + 2), Failed 0.WpfStaGateCensusTestswith them: Total 7, Failed 0.WpfExitSwitch*,DarlingReadsTraitGuardTests,GuardStage*,WpfStaGateCensus*: Total 20, Failed 0.*Census*and*Ratchet*class in Darling.Tests, plusWpfExitSwitch*: Total 505, Failed 0, Skipped 1.-trait Stage=Guard -failSkipson Darling.Tests: Total 1434, Failed 0, Skipped 0.ProcessExithandler ran 0.17 s before the process was gone.runtimeconfig.json(jobdiag (WPF switch off)of the temporaryDiag 5596workflow): Total 29398, Failed 2 (the two switch checks inWpfExitSwitchTests, the expected RED), the runner printed "Waiting 10 seconds for foreground threads to exit..." and then "[FATAL ERROR] Foreground threads were left running, forcing process exit", exit code 1. The 118ProcessExithandlers took 17.9 s from the first to the last. With the switch on (jobdiag (WPF switch on)) the same 118 handlers took 0.03 s, exit code 0, Failed 0, no wait line.Handoff
Done: cause found, fix, tests, diagnostic workflow and module initializer deleted, squash body written. Left: the "Darling whole-tree guards" job result on this PR now that it is ready.
CHANGELOG entry (CI-only fix):
SECTION: Fixed
ENTRY: - The Darling whole-tree guards CI job no longer fails after every test passed ([#5598]) - WPF left a process-exit handler for every test thread that used a bound control, each waiting out a 300 millisecond timeout, so the test process outlived the runner's 11 second exit window and the runner forced exit code 1 with no failed test; both test projects now set the WPF switch that skips the wait. This is a CI-only fix: no product code changed.