Repository navigation
Description
What is failing
MeshWeaver.Hosting.Monolith.Test.RequirePrebuiltParkTest.AValidTypeWithNoAdoptedAssembly_ParksWithTheNamedRefusal_AndNeverCompiles fails intermittently on CI (first seen: run 32882908160, shard 1):
AssertionException : Expected value to be true because a require-prebuilt refusal must park
the type so no later trigger can storm, but found False.
at RequirePrebuiltParkTest.cs(89) // parkRegistry.IsParked(NodeTypePath).Should().BeTrue(...)
The test is right and the production code is wrong. A type that was parked ends up un-parked, so "no later trigger can storm" is not actually guaranteed.
Root cause — an unvalidated Unpark() from a stale snapshot
NodeTypeCompilationHelpers' failed-verdict re-drive subscribes to the NodeType's own stream and, on an emission it judges stale, does:
parkRegistry?.Unpark(hubPath); // ← decided from `node`, a SNAPSHOT
using var systemScope = redriveAccess?.ImpersonateAsSystem();
workspace.GetMeshNodeStream().Update(curr => { … }); // re-checks CURRENT state inside the lambdaThe comment above the DistinctUntilChanged says it outright — "correctness is the re-check inside the Update lambda, not this". That is true for the flip to Pending, and false for the Unpark(): the un-park runs before the Update, is never re-validated, and therefore can remove a park that landed after the snapshot it was judged from.
Nothing re-parks afterwards, because the adopt-only gate only parks on a Pending flip and the type is now settled at Error.
The observed sequence (identical in CI and locally)
.402 CompileWatcher: … PARKING with a named refusal ← gate refuses
.402 NodeType '…' PARKED after compile failure ← park #1 (TryAdd succeeded)
re-drive: FailedBuildInputs is "(never stamped)" ≠ live inputs → Unpark() + flip Pending
.420 CompileWatcher: … PARKING with a named refusal ← gate refuses again
.421 NodeType '…' PARKED after compile failure ← park #2 (TryAdd succeeded again)
.421 NodeType … is STUCK at Error … the automatic re-drive correctly declines
.427 TEST FAILED — IsParked == false
The double log line is the proof, not a cosmetic detail: ParkAndNotify logs only after a successful _parked.TryAdd, and nothing else in the registry removes an entry. Two "PARKED after compile failure" lines therefore mean the entry was removed between them — i.e. Unpark() ran. A third, late Unpark() from an in-flight re-drive subscriber then removes park #2 and leaves IsParked == false.
SettleAsError (the gate's settle) does not stamp FailedBuildInputs, which is why the re-drive fires at all on the very first park: the stamp is written only by the re-drive's own Update, so the first emission always looks "never stamped ≠ live".
Reproduction — deterministic with an amplifier
The window is tiny, so it is rare under a plain runner. Running the test with the detailed console logger widens it (more I/O between the emission and the un-park) and makes it fail every time:
DOTNET_PROCESSOR_COUNT=4 dotnet test test/MeshWeaver.Hosting.Monolith.Test --no-build \
--filter "FullyQualifiedName~RequirePrebuiltParkTest" --logger "console;verbosity=detailed"| arm | result |
|---|---|
| plain runner, isolated | 20 / 20 pass |
| verbose logger, isolated | 0 / 15 pass |
| full assembly, plain runner | 758 / 761 pass (this test passed) |
Every failing run logs exactly the double-park above.
Not caused by #2238
Found while validating PR #2238 (IoPool / MessageService disposal), which CI red-flagged with this test. Measured A/B on the same machine, same amplifier, DOTNET_PROCESSOR_COUNT=4:
| source | result |
|---|---|
| PR #2238 | 0 / 15 pass |
origin/main, both touched files reverted |
0 / 15 pass |
Identical failure and identical double-park signature on unpatched main, so #2238's diff is not involved — as the mechanism already implies: it touches IoPool and the gated-deferral path in MessageService, while this defect is entirely inside NodeTypeCompilationHelpers + NodeTypeCompileParkRegistry, and this test never runs a compile at all (attempt count 0).
The fix is NOT a wait or a retry in the test
The assertion is correct — a parked type really is being un-parked. Options:
- Validate the un-park against current state instead of the snapshot. The registry knows when it parked (
ParkedCompileFailure.ParkedAt) and under which sources; a conditionalUnparkIf(path, parkedBefore: snapshotTime)— or anUnparkthat takes the park token it intends to retire — makes the operation compare-and-swap rather than unconditional remove. This matches the discipline the rest of the file already uses (the Update lambda re-checks). - Or move the
Unpark()inside the sameUpdatelambda that re-checks, so the un-park and the Pending flip commit against one observed state. - Additionally,
SettleAsErrorarguably should stampFailedBuildInputswith the inputs the refusal was formed under, so the very first park does not read as "never stamped" and trigger a re-drive that was never needed.
Where to look
src/MeshWeaver.Graph/Configuration/NodeTypeCompilationHelpers.cs— the failed-verdict re-driveSubscribebody (parkRegistry?.Unpark(hubPath);immediately before theUpdate), andSettleAsErrorin the adopt-only gate.src/MeshWeaver.Graph/Configuration/NodeTypeCompileParkRegistry.cs—ParkAndNotify(TryAdd, logs only on success — the signal that proves the removal),Unpark.test/MeshWeaver.Hosting.Monolith.Test/RequirePrebuiltParkTest.cs:89.
Activity
Merge pull request #2262 from Systemorph/fix/2260-unpark-stale-snapshot