Repository navigation
Conversation
…hub via timer ExecutionContext MeshHubDisposalLeakTest flaked on main run 30068597014 (shard 1) and PR #624's run 30049945115 with the IDENTICAL ClrMD retention path in both artifacts: TimerQueueTimer -> ExecutionContext -> AsyncLocalValueMap -> MeshWeaver.Kernel.Hub.LoggerTextWriter -> ActivityLogLogger -> MessageHub [RunLevel=6, Address=kernelExec/Roland/_Activity/...] Root cause: CapturingTextWriter.Capture stored the script's stdout/stderr pipe (LoggerTextWriter -> ActivityLogLogger -> kernel activity hub -> whole mesh) DIRECTLY in the AsyncLocal. Anything that snapshots the ExecutionContext while a capture scope is active - a lazily-created long-lived TimerQueueTimer, a pending Rx delay timer - freezes that AsyncLocal map into the snapshot. The scope's Restorer only rewrites the AsyncLocal on the current flow; it can never reach the frozen snapshots, so one such timer pinned the DISPOSED kernelExec activity hub (created by an earlier script-running test class in the same process, e.g. LinkedInPullActionsTest) and through it the whole disposed mesh for the timer's lifetime. Intermittent because it needs an infrastructure timer's first creation (or a pending delay) to land inside a capture window. Fix (product, not test): the AsyncLocal now holds a mutable CaptureCell indirection; disposing the capture scope nulls CaptureCell.Target FIRST, which severs the writer graph inside every already-captured ExecutionContext - a stray snapshot keeps only the empty ~16-byte cell. Late writes from stray continuations now fall back to the real console instead of a disposed pipe. Deterministic repro: ConsoleCaptureExecutionContextLeakTest creates a timer inside a capture scope, keeps it alive, and asserts the capture target is collectible after scope disposal - RED before this fix, GREEN after. Verified: repro + MeshHubDisposalLeakTest + KernelScriptMemoryLeakTest + KernelReplChainingTest (asserts Console.WriteLine output still reaches the ActivityLog) all green; dotnet build --no-restore -c Release -p:CIRun=true -warnaserror clean (0 errors). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012wP3c7DLHutUczcBWgs4h8
Contributor
Contributor
Contributor
Contributor
Contributor
Contributor
Contributor
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Root cause (the flaky test was RIGHT — real product leak)
Retention path, identical in both failing runs (main 30068597014, PR #624's 30049945115):
CapturingTextWriter.Capturestored the script stdout/stderr pipe (writer → ActivityLogLogger → activity hub → mesh) directly in itsAsyncLocal. Any long-lived timer (or pending Rx delay) created inside a capture window snapshots the ExecutionContext including that AsyncLocal map; the scope'sRestoreronly rewrites the current flow and can never reach frozen snapshots — so a disposed mesh stays rooted for the timer's lifetime. Intermittency: needs a long-lived timer's creation to land inside a capture window (the #620/#624 poll timers merely changed the timer population; they weren't the rooter).Fix
The AsyncLocal now holds a mutable
CaptureCellindirection; scope disposal nullscell.TargetFIRST, severing the writer graph inside every already-captured ExecutionContext. A stray snapshot keeps only the empty cell; late writes fall back to the real console instead of a disposed pipe.Verification
ConsoleCaptureExecutionContextLeakTest: timer created inside a capture scope, kept alive; asserts the target is collectible after scope disposal — RED pre-fix (pin reproduced in 12 ms), GREEN post-fix.MeshHubDisposalLeakTest,KernelScriptMemoryLeakTest,KernelReplChainingTest(capture routing preserved — script Console output still reaches the ActivityLog) all green, non-empty trx verified.dotnet build --no-restore -c Release -p:CIRun=true -warnaserror→ 0 errors.AsyncLocal<>in src/ — no other instance of this leak class.No What's New entry — internal stability fix.
🤖 Generated with Claude Code