Skip to content

fix(content): eliminate SaveFileAsync file-lock race in content-collection writes - #349

Merged
rbuergi merged 1 commit into
mainfrom
fix/contentcollections-savefile-filelock-race
Jul 6, 2026
Merged

rbuergi merged 1 commit into
mainfrom
fix/contentcollections-savefile-filelock-race

Conversation

@rbuergi

@rbuergi rbuergi commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

The flake

CI-only, load-dependent, not a timeout — fails fast (~21 ms):

System.IO.IOException : The process cannot access the file
'.../bin/Release/net10.0/Files/NodeContent/TestOrg/sub/hello.md'
because it is being used by another process.
   at MeshWeaver.ContentCollections.FileSystemStreamProvider.SaveFileAsync(...) FileSystemStreamProvider.cs:157
   at MeshWeaver.ContentCollections.ContentCollection.SaveFileAsync(...) ContentCollection.cs:142

Observed on NodeHubContentCollectionTest.CollectionNamedArea_RendersBrowserForFolder_AndContentForFile. This is a new manifestation of the CollectionNamedArea watcher-race flake family — the timeout manifestation was fixed in #238; this is a file-lock / sharing violation on write.

Root cause (exact, proven)

FileSystemStreamProvider writes (WriteStreamAsync, SaveFileAsync) opened the file with FileShare.None, while the read path (GetStreamAsync / GetStreamWithMetadataAsync) opens with the deliberately-tolerant FileShare.ReadWrite | FileShare.Delete.

On Unix, .NET emulates FileShare via flock:

  • FileShare.None → exclusive LOCK_EX
  • a held reader → LOCK_SH

LOCK_EX is refused while a reader holds LOCK_SH, so the write throws IOException: "... used by another process". The concurrent reader is the FileSystemWatcher-driven ContentCollection.UpdateArticle, which reads the file on every change and legitimately overlaps a write. macOS uses flock too, so it reproduces locally under load.

There is no undisposed handle — the readers dispose correctly. The defect is that the write demanded exclusivity against readers the read path is explicitly built to tolerate (content is eventually-consistent and re-ingested on change). FileShare.None on the write is simply inconsistent with that contract.

Confirmed empirically (probe table):

reader share writer share result
ReadWrite|Delete (read path) None (old write) FAILED
ReadWrite|Delete Read OK
ReadWrite|Delete ReadWrite OK

Fix (correct sharing semantics, not a mask)

Both write sites now use single-writer / multi-reader / delete-tolerant sharing — FileShare.Read | FileShare.Delete — matching the read path's intent: a write permits concurrent readers (and deletes, which reads already permit) but does not lock them out. This is not a blanket widening to hide an undisposed handle; it makes the write's sharing contract consistent with the read contract.

Verification

  • Reproduced deterministically: a tight write/read loop over the real provider fails from iteration ~5 (once watcher reads are in flight) without the fix; failures=0 with it.
  • New pin ContentCollectionWriteReadRaceTest drives an explicitly-open reader (FileShare.ReadWrite|Delete) concurrently with 300 writes — throws the exact IOException without the fix, passes with it. No sleeps, no watcher-timing dependence.
  • dotnet build -c Release -warnaserror -p:CIRun=true clean (src + test).
  • Full MeshWeaver.ContentCollections.Test suite green (17 tests), including the previously-flaking CollectionNamedArea test and the fix(content): ingest a collection's own write immediately — kill the CI-only file-render flake #238 ingest pin.

🤖 Generated with Claude Code

…ction writes

FileSystemStreamProvider WRITES opened with FileShare.None while the read
path (GetStreamAsync/GetStreamWithMetadataAsync) opens with the tolerant
FileShare.ReadWrite|Delete. On Unix .NET emulates FileShare via flock:
FileShare.None takes an EXCLUSIVE lock (LOCK_EX) that the OS refuses while a
reader holds the file (LOCK_SH). The FileSystemWatcher-driven UpdateArticle
read legitimately overlaps a write, so under CI load the write threw fast
(not a timeout):

  System.IO.IOException: The process cannot access the file
  '.../Files/NodeContent/TestOrg/sub/hello.md' because it is being used by
  another process.
  at FileSystemStreamProvider.SaveFileAsync (:157)

This is the file-lock manifestation of the CollectionNamedArea watcher-race
flake family (the timeout manifestation was PR #238). There is NO undisposed
handle — the readers dispose correctly; the defect is that the write demanded
exclusivity against readers the read path is designed to tolerate. Content
files are eventually-consistent and re-ingested on change, so a write must
never lock readers out.

Fix: both write sites use single-writer / multi-reader / delete-tolerant
sharing (FileShare.Read | FileShare.Delete) to match the read path.

Pinned by ContentCollectionWriteReadRaceTest: without the fix the write throws
the exact IOException within a few iterations; with it, 300 writes concurrent
with a continuously-open reader never throw. Deterministic, no sleeps.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@github-actions

github-actions Bot commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Test Results (shard 5)

881 tests   699 ✅  4m 10s ⏱️
  9 suites  182 💤
  9 files      0 ❌

Results for commit 9b2b7a4.

@github-actions

github-actions Bot commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Test Results (shard 1)

1 029 tests  ±0   1 028 ✅ ±0   3m 34s ⏱️ -13s
    8 suites ±0       1 💤 ±0 
    8 files   ±0       0 ❌ ±0 

Results for commit 9b2b7a4. ± Comparison against base commit b2263bc.

@github-actions

github-actions Bot commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Test Results (shard 2)

716 tests  ±0   714 ✅ ±0   1m 57s ⏱️ -41s
  9 suites ±0     2 💤 ±0 
  9 files   ±0     0 ❌ ±0 

Results for commit 9b2b7a4. ± Comparison against base commit b2263bc.

@github-actions

github-actions Bot commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Test Results (shard 4)

   10 files  ±0     10 suites  ±0   4m 50s ⏱️ +15s
1 362 tests +1  1 362 ✅ +1  0 💤 ±0  0 ❌ ±0 
1 401 runs  +1  1 401 ✅ +1  0 💤 ±0  0 ❌ ±0 

Results for commit 9b2b7a4. ± Comparison against base commit b2263bc.

@github-actions

github-actions Bot commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Test Results (shard 0)

648 tests  ±0   648 ✅ ±0   4m 57s ⏱️ -5s
 10 suites ±0     0 💤 ±0 
 10 files   ±0     0 ❌ ±0 

Results for commit 9b2b7a4. ± Comparison against base commit b2263bc.

@github-actions

github-actions Bot commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Test Results (shard 3)

819 tests  ±0   819 ✅ ±0   3m 46s ⏱️ -19s
 10 suites ±0     0 💤 ±0 
 10 files   ±0     0 ❌ ±0 

Results for commit 9b2b7a4. ± Comparison against base commit b2263bc.

@github-actions

github-actions Bot commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Test Results

   56 files  ±0     56 suites  ±0   23m 17s ⏱️ - 1m 7s
5 455 tests +1  5 270 ✅ +1  185 💤 ±0  0 ❌ ±0 
5 494 runs  +1  5 309 ✅ +1  185 💤 ±0  0 ❌ ±0 

Results for commit 9b2b7a4. ± Comparison against base commit b2263bc.

@rbuergi
rbuergi merged commit c7ea1da into main Jul 6, 2026
15 checks passed
@rbuergi
rbuergi deleted the fix/contentcollections-savefile-filelock-race branch August 5, 2026 10:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant