Bound leaked ActiveEnumeration collection behind an off-by-default flag + Heartbeat observation telemetry - #2053
Merged
tyrielv merged 1 commit intoJul 14, 2026
Conversation
tyrielv
force-pushed
the
tyrielv/bound-active-enumeration-leak
branch
from
July 9, 2026 22:37
af22f30 to
0603911
Compare
…t flag ProjFS does not always deliver EndDirectoryEnumeration for a StartDirectoryEnumeration (for example when an enumeration is cancelled), which leaks the corresponding ActiveEnumeration - and the projected item list it pins - in this.activeEnumerations. Over long-lived GVFS.Mount processes this unbounded growth is a suspected contributor to the memory pressure behind the native ProjFS command-completion crashes that surface downstream as STATUS_FILE_SYSTEM_VIRTUALIZATION_UNAVAILABLE (0xC000CE01). "Suspected" is the operative word: Watson shows radar_high_memory in projectedfslib.dll!prjcompletecommand, but we do not yet have direct evidence that the never-ended-enumeration leak is a dominant driver of that memory. This change is therefore split into two parts so we can measure before we change the enumeration hot path anywhere: 1. Observation telemetry (always on). ActiveEnumeration records a monotonic LastActivityTickCount (Environment.TickCount64) on creation and on every GetDirectoryEnumeration read. WindowsFileSystemVirtualizer now contributes ActiveEnumerationCount and ActiveCommandCount to the periodic Heartbeat event via a new FileSystemVirtualizer.AddHeartbeatMetadata hook. This ships the signal needed to confirm (or refute) that activeEnumerations grows without bound on real machines - independent of whether eviction is enabled. 2. Eviction (off by default). When the gvfs.max-active-enumerations git config is set to a positive value, a throttled sweep (at most once per minute, and only once the collection exceeds the configured count) evicts enumerations idle longer than a 5 minute timeout. Config unset or <= 0 disables eviction entirely, so the enumeration hot path is unchanged by default. Monotonic clocks are used for both the throttle and the staleness cutoff so wall-clock adjustments cannot disturb them, and the throttle claim uses an Interlocked.CompareExchange keyed on the previously-read tick so two threads entering the same interval cannot both sweep. Evicting a live-but-idle enumeration is safe: the next GetDirectoryEnumeration for that id misses activeEnumerations and returns HResult.InternalError - it fails that one directory listing loudly rather than returning truncated results as if complete - and every eviction is reported via telemetry. This complements the stabilization-safe "do not crash the mount on a native ProjFS failure" change (microsoft#2042), which is intentionally kept separate and does not touch the enumeration hot path. Eviction stays behind the flag until the Heartbeat telemetry confirms the leak is worth acting on. Tests: ActiveEnumeration records activity time; Heartbeat metadata reports the live enumeration count; with eviction disabled stale enumerations are retained; with eviction enabled stale enumerations are evicted (and their subsequent End fails with InternalError) while freshly-active ones are kept. Full unit suite passes. Assisted-by: Claude Opus 4.8 Signed-off-by: Tyrie Vella <tyrielv@gmail.com>
tyrielv
force-pushed
the
tyrielv/bound-active-enumeration-leak
branch
from
July 9, 2026 23:01
0603911 to
c8eb49c
Compare
tyrielv
marked this pull request as ready for review
July 9, 2026 23:08
tyrielv
enabled auto-merge
July 9, 2026 23:08
Keith Klein (KeithIsSleeping)
approved these changes
Jul 14, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Background
GVFS.Mount crashes surface downstream as
STATUS_FILE_SYSTEM_VIRTUALIZATION_UNAVAILABLE(0xC000CE01): once the mount dies, the ProjFS virtualization root is orphaned and every subsequent placeholder access fails. Watson attributes the crashes toradar_high_memory projectedfslib.dll!prjcompletecommand— GVFS.Mount is crashing under memory pressure in the native ProjFS command-completion path.One suspected managed-side contributor: ProjFS does not always deliver
EndDirectoryEnumerationfor aStartDirectoryEnumeration(e.g. cancelled enumerations), leaking theActiveEnumeration— and the projected item list it pins — inactiveEnumerations. Over a long-lived mount this grows without bound.We do not yet have direct evidence that this leak is a dominant driver of the memory pressure. So this PR is deliberately structured to measure first, act second:
What this PR does
1. Observation telemetry — always on.
ActiveEnumerationtracks a monotonicLastActivityTickCount.FileSystemVirtualizer.AddHeartbeatMetadatahook; the Windows virtualizer reportsActiveEnumerationCount/ActiveCommandCounton the periodic Heartbeat event, regardless of the flag below.2. Eviction — off by default, behind
gvfs.max-active-enumerations.<= 0→ eviction disabled, enumeration hot path unchanged (the default).> 0→ a throttled sweep (≤ once/min, only once the collection exceeds the configured count) evicts enumerations idle longer than 5 minutes.Interlocked.CompareExchangekeyed on the previously-read tick so two threads in the same interval can't both sweep.Evicting a live-but-idle enumeration is safe and loud: the next
GetDirectoryEnumerationfor that id misses the collection and returnsHResult.InternalError(fails that one listing rather than returning truncated results as if complete), and every eviction is traced.Relationship to #2042 (independent — either order)
#2042 addresses a different lever on the same crash: it stops a single failed native ProjFS call from taking down the whole mount (survive-and-report instead of crash). This PR instead targets the memory pressure that triggers those faults, and gates the behavior change off until telemetry justifies it.
The two are independent: this branch does not contain #2042's commit and references none of its code (and vice-versa). They can be reviewed and merged in either order with no rebase — verified with
git merge-tree(clean, no conflicts). #2042's own investigation (a local repro that showed enumeration cancellation self-heals and is not the OOM driver) is part of why the eviction here is unproven and therefore gated off by default.Open questions this is meant to answer
activeEnumerationsactually grow without bound in the field? (HeartbeatActiveEnumerationCount.)Testing
ActiveEnumerationrecords activity time.End→InternalError), freshly-active ones kept.