Let the host declare LocalDataService's write-lock budget instead of every caller inheriting the dispatcher's - #3138
Conversation
…d measurement Temporary. Emits a TRX from the normal parallel Lite run, runs the suite a second time with the classes fully serialised, and uploads both for offline analysis. Reverted in the following commit.
|
Reviewed. This PR is scoped to Two findings posted inline, both on the new steps:
No correctness/security/performance issues beyond the above — the rest (trx flag placement, |
…riting the dispatcher's LocalDataService.OpenWriteConnectionAsync bounds its write-lock wait so a WPF dispatcher cannot freeze behind an in-flight archival. DuckDbInitializer's lock is process-wide, so in a host with no dispatcher that budget can expire inside an unrelated class's deliberate hold and fail a store test for a user who is not there. The budget now resolves from a runtime configuration property the host declares in its own project file; the shipped app declares none and keeps five seconds, and Lite.Tests declares one that outlasts every hold its own suite takes.
The two figures it was added for are recorded in the pull request description.
…ite-lock-serialisation
|
Reviewed the net diff (5 files: Correctness —
Lite/Darling parity — no gap. Security — none of the usual concerns apply (no SQL, no network/file/process I/O beyond the existing DuckDB path, no secrets). The Style — comments follow the repo's "explain WHY at length" convention, XML doc comments present, No findings. This is a well-scoped, well-tested fix for a real flaky-test cause (#2374/#3134's shared process-wide lock across parallel test classes). |
CrossAppGuardCiGateTests reads Lite.Tests.csproj with a regex whose single-quote alternation treats an apostrophe as an attribute delimiter, so an unbalanced pair swallows the quoted attribute values after it - including the linked cross-app Compile of Darling.Tests/CSharpSourceWalker.cs, whose absence the guard reports as a moved blind spot. The comment says so, next to the constraint.
…ite-lock-serialisation
|
Reviewed the diff (Lite.Tests/Lite.Tests.csproj, Lite.Tests/SharedDuckDbFixture.cs, Lite.Tests/WriteLockBudgetTests.cs, Lite/Database/DuckDbInitializer.cs, Lite/Services/LocalDataService.cs). Scope check: this is Lite-only infrastructure — Correctness: left one inline comment on Everything else checks out:
|
…er total NumberStyles.Float admits exponents and .NET parses the invariant Infinity symbol whatever the style, so "1e300" and "Infinity" reached TimeSpan.FromSeconds as positive doubles that overflow it - out of a static initializer, a TypeInitializationException on the first store call in the process rather than one timeout. double.Epsilon is the same hole at the other end: positive, under any ceiling, and rounds to TimeSpan.Zero, which is a timeout that never waits. The resolver now bounds the declared seconds by MaxWriteLockBudget and checks the TimeSpan it constructed rather than the double it parsed. The ceiling is one hour rather than TimeSpan's representable range, which reaches about 29,000 years and would accept a budget that makes a wedged lock hang for the life of the process.
…ite-lock-serialisation
|
Reviewed the diff ( Parity: no concern. Darling has no DuckDB store and no Production path: One finding posted inline on Everything else — the apostrophe fix to keep |
A cref names a symbol this repository spells, and TypeInitializationException is a framework type no C# here declares - DocCommentHygieneTests reports it as a target that resolves to nothing. The two mentions that predate this both spell it <c>, which is what naming runtime behaviour rather than a navigable symbol looks like.
|
Reviewed the write-lock-budget change (Lite.Tests.csproj, SharedDuckDbFixture.cs, WriteLockBudgetTests.cs, DuckDbInitializer.cs, LocalDataService.cs). Correctness —
Lite/Darling parity — no drift here. Security — no user-facing input; the parsed value only ever comes from the host's own compiled-in Performance — this only widens the timeout for the Left one inline nit on a minor test-robustness gap ( |
…ds up The hold scan claimed both its approximations round the bar up. Both rounded it down. It searched only the top level of Lite.Tests while the build-file scan recursed, so a helper under Lite.Tests/Helpers taking the write lock would never be seen. And it read TimeSpan.From* and .Join(<int>) but not a bare-millisecond .Wait(30000) - a file spelling its hold that way still matched the write-lock scan, joined the population and contributed TimeSpan.Zero through DefaultIfEmpty. gate.Wait(TimeSpan.FromSeconds(30)) and gate.Wait(30000) are the same thirty second hold; one measured 30 s and the other measured nothing. The scan now recurses, reads the bare-millisecond spellings, and reports a duration it cannot read as a FAILURE rather than a zero, so a file with no duration site at all stays distinguishable from a file whose duration was unreadable. Read off CSharpSourceWalker.StripCommentsAndStrings, so a duration in a comment cannot raise the bar and string.Join is not mistaken for Thread.Join.
|
Reviewed the diff (Lite.Tests + Lite/Database/DuckDbInitializer.cs + Lite/Services/LocalDataService.cs). This is Lite-only test infrastructure plus one config-driven timeout — no Darling-side counterpart is needed since the dispatcher/process-wide-lock scenario this addresses is WPF/DuckDB-specific and Darling has no equivalent. Correctness Traced
Math check on the budget-vs-holds bound Verified the "47s sum, 120s budget" claim in the PR description by grepping actual hold durations: One file the PR description doesn't mention — AppContext round-trip risk The one part I couldn't verify statically: the design relies on Style / parity No T-SQL in this diff. C# style (naming, XML doc comments, partial class organization) is consistent with the surrounding files. The apostrophe-avoidance in the new Nothing blocking found. |
LocalDataService.OpenWriteConnectionAsyncis the one write-lock caller in Lite that passes a timeout, and its own remarks say why: it sits on the path a WPF dispatcher awaits, so it cannot afford the unbounded wait the app's other nineteen write-lock acquisitions take.DuckDbInitializer.s_dbLockis process-wide. Put those two facts in a host with no dispatcher and the timeout is protecting nobody while still deciding outcomes — a store call queues behind every other class in the process and then fails on a budget sized for a user who is not there.The budget now resolves from a runtime configuration property the host declares in its own project file. The shipped app declares none and resolves
LocalDataService.DefaultWriteLockBudget, five seconds, unchanged.Lite.Testsdeclares 120 seconds, so an acquisition queued behind a neighbour's deliberate hold waits it out instead of expiring inside it.Why the wait is now bounded rather than merely luckier
Four test methods in three classes take the process-wide lock EXCLUSIVELY and hold it while they assert on timing —
AnalysisPassTokenThreadingTests.TheReadLockWaitIsAbandonableWhileAWriterHoldsIt(holder net 30 s),StatusBarSizeReadLockTests.GetUsedDataSizeMb_WhenTheWriteLockIsHeld_GivesUpInsteadOfBlocking(15 s), andDismissReliabilityTests' twoWriteLock_*methods (2 s each). Every one of those nets is a safety valve on an event wait, so the hold length is decided by scheduling rather than by the ceiling, and three of the four ceilings sit at or above five seconds. Their file-wise maxima sum to 47 s, which is the most that can queue in front of one acquisition; 120 s clears it by 2.5x. Two pins carry that.EveryDeliberateHoldInThisSuiteIsReadablerequires every duration-shaped call site in a holder file to be one the scan can READ, andTheBudgetOutlastsEveryDeliberateHoldInThisSuitederives the sum from the suite's own source on every run rather than restating it. The readability pin is what makes the sum a measurement rather than a shape: a spelling outside the recognised set does not go uncounted, it joins the population and contributes zero, so an unreadable duration fails loudly instead.Short of infinite on purpose: a genuinely wedged lock — the leaked-reader failure
DuckDbInitializer.LockReleaserdocuments — still fails, with this method's own message, instead of hanging the step.It does not hide a product regression that the old budget would have caught. A five-second fuse that fires in 1 of 194 runs is a coin flip rather than a detector, and the tests that exist to pin lock-hold behaviour pass their own explicit timeouts —
DismissReliabilityTests' 50 ms and 100 ms,DuckDbInitializer.StatusBarReadLockTimeout's 100 ms — none of which this touches. A product path that genuinely started holding the lock for tens of seconds would show up as suite duration, which is measured here to the second.The two measurements the issue asked for, and how they were taken
A shared collection costs the Lite suite +186 s, roughly doubling the step. Measured on one runner, one commit, back to back, by running the suite twice in the
buildjob — once normally, once with-parallel none— and uploading both TRX files (run 34122782002; the instrumentation is commit65b4065af, reverted in9446d754e, so it is not in this diff). Parallel span 231.7 s, fully serialised 449.3 s: +217.6 s, 1.94x, at an effective 2.90-way parallelism. Those two ratios are different quantities and neither is derived from the other: 2.90 is observed concurrency (summed per-test durations over the parallel span), 1.94 is end-to-end speedup. The +186 s below rests on the summed per-class intervals rather than on either ratio. Charging each class the serialised interval since the previous test ended — which bills it for its own fixture schema build — the 75 classes that touchs_dbLockcarry 417.9 s of that 449.3 s, 93.0%. Leaving the other 232 classes parallel recovers only ~21 s, so a collection over the set that would actually have to be in it lands at ≈418 s: +186 s, 1.80x, which would put the step's new median above today's p95 (389 s). That +186 s is a LOWER bound on option 1 rather than an estimate of it: the per-class costs come from a run with no contention at all, while a collection would still have 232 classes running alongside it.The timeout fires in 1 of 194 runs, 0.52%. Across the last 300
build.ymlruns (2026-09-05T13:17Z to 2026-09-07T10:14Z, ~45 h) theRun Lite testsstep executed 194 times — 63 skipped by the path gate, 41 cancelled by a re-push. Five of the 194 failed, and reading each failure's log: two were genuine pin failures (WatermarkPolicyTests,CrossAppMcpToolInventoryPinTests), two wereCrossAppGuardCiGateTestsfile-in-useIOExceptions, and exactly one was this timeout — run 34091806113, whose 487 s step sits at p97.9 of the 194 step durations against a p50 of 262 s. It is a slow-runner failure, and four runs that were slower still passed.The choice, against those figures
The collection was not chosen, and cost is the second reason rather than the first. As scoped in the issue it cannot bound the race at all:
[Collection]only stops classes in the same collection running concurrently with each other, and three of the four longest holders build their ownDuckDbInitializerand are not classes ofSharedDuckDbFixture, so a collection over the store-touching classes leaves the 30 s and 15 s holders running alongside it.DatabaseStateWriteLockTests' own remarks already say this — "DisableParallelizationonly orders collections inside the non-parallel bucket — it cannot stop other classes from contending on a static lock". Widening it to all 75 lock-touching classes is what the +186 s buys, and it stays unenforceable: nothing makes the next store test remember to join.Against that, the host budget costs no wall clock, applies at the one choke point every caller already goes through so no future test class can forget it, and turns the failure into a bounded wait. It also does what
DatabaseStateWriteLockTestsrecords as needed — "that arm needs the timeout injected, i.e. production shape changed for testability. Left visible rather than implied-covered."The per-database lock was not chosen either, and what
s_dbLockprotects across instances was established first rather than assumed. The rule on the field says: "ONE lock for the whole process, deliberately: Lite constructs severalDuckDbInitializerinstances over the same file (MainWindow, DatabaseStateOverridesWindow, DuckDbAlertHistoryStore), and making it per-instance would trade a slow test suite for a real data race." So it protects instances over the SAME FILE, and keying it per path would preserve that in an app that has exactly oneApp.DatabasePath. It was still refused: its failure mode is two spellings of one path resolving to two locks, which is silent data corruption rather than a loud timeout — the wrong direction for a mistake to fail in — and it would additionally rewrite the #2463 rule and the four-file source pin (DuckDbLockModelTests.TheLockRuleIsWrittenWhereEveryCallerAlreadyLooks) that keeps it findable.This is a product change, deliberately
Option 1 would have been test-only; this is not. The measurement is what justifies crossing that line: the test-only option is a permanent +186 s on every Lite run, does not cover the holders it needs to, and cannot be enforced — so "cheaper class of change" was not available at any price worth paying. What crossing it costs is a demonstration that the app is unchanged, and that is what the first two pins are for:
OnlyTheTestHostDeclaresABudgetscans every.csproj,.propsand.targetsin the repo outside build output and requires thatLite.Tests/Lite.Tests.csprojis the only one naming the key — whole-tree, because a sharedDirectory.Build.propswould reach the shipped app just as effectively as its own project file — andTheAppKeepsTheDispatchersFiveSecondspins the value a host that declares nothing resolves.ThisHostResolvedTheBudgetItsProjectFileDeclaresis the one that catches a seam that is wired but inert. A property that never reachesAppContextresolves the default silently, which is indistinguishable from the pre-change behaviour and would leave the flake in place with every other pin green; so the expected value is read out of the project file and compared against what the running host resolved, and the budget is additionally required to differ from the default, because matching the default is exactly what a dead seam looks like.The seam had a third outcome, at both ends
Review caught that the resolver bounded the wrong end, and it bears directly on the thesis above: the argument for this change is that a budget either lets the wait succeed or fails loudly, and a declared value that throws during resolution is neither.
NumberStyles.Floatadmits exponents and .NET parses the invariantInfinitysymbol whatever the style, so"1e300"and"Infinity"arrive as positive doubles and overflowTimeSpan.FromSeconds. Out of a static initializer that is aTypeInitializationExceptionon the first store call anywhere in the process — strictly worse than the timeout it replaces.seconds > 0is mutation-tested and bounds the other end.Fixed rather than justified, and the fix is not the obvious one. Bounding by
TimeSpan's representable range would have been wrong: measured,TimeSpan.MaxValue.TotalSecondsis 9.22e11 — about 29,000 years, not the 9.22e14 an eye-estimate gives — andFromSecondsof it round-trips without complaint. That bound accepts a budget which makes a wedged lock hang for the life of the process, converting the loud failure into a silent one. So the ceiling is one hour, on the ground that a wait outliving the process or the job containing it cannot produce this method's timeout at all, and being short of forever is the whole reason there is a number.Writing the hostile-input pin then found the same hole at the low end, which neither the review nor the original guard covered:
double.Epsilon("5E-324") parses, is positive, is far under any ceiling, andTimeSpan.FromSecondsrounds it toTimeSpan.Zero— a timeout that gives up without waiting, a budget in name only. So the resolver now checks theTimeSpanit constructed, not the double it parsed, which is the shape that catches both ends by construction rather than by enumerating them.The framework exception is named as prose rather than as a cref throughout, which
DocCommentHygieneTestsrequires: a cref names a symbol this repository spells, andTypeInitializationExceptionis declared nowhere here. The two mentions that predate this change both spell it<c>.Two pins, and they are not redundant.
TheResolverIsTotalAndAlwaysReturnsAUsableBudgetasserts over hostile inputs that resolution never throws and never leaves(TimeSpan.Zero, MaxWriteLockBudget].TheCeilingIsAcceptedAndAnythingPastItResolvesTheDefaultderives both figures from the constant. Setting the ceiling toTimeSpan.MaxValueleaves the totality pin GREEN — nothing throws and everything is in-band once the band is 29,000 years wide — and only the ceiling pin reds. The totality pin alone would have accepted the bound that was wrong.The derivation claimed to round up. It rounded down, twice.
Review turned the "what legal spelling would this miss" question on the scan I wrote and found two gaps, both in the unsafe direction — which falsifies the justification I gave for the bound, not merely its coverage.
It searched only the top level of
Lite.Testswhile the build-file scan recursed.Lite.Tests/Helpers/already exists with.csfiles, so a helper there taking the write lock would never be scanned. Verified with a positive control: the same 200 s holder placed underHelpers/reds the bound with recursion on, and with recursion off is entirely invisible — 31 green, no finding at all.The worse one is a different failure: a zero that looks like a measurement. The scan read
TimeSpan.From*and.Join(<int>)but not a bare-millisecond.Wait(30000). Such a file still matches the write-lock scan, so it joins the population and contributesTimeSpan.ZerothroughDefaultIfEmpty. Measured against the old parser:gate.Wait(TimeSpan.FromSeconds(30))scores 30.0 s andgate.Wait(30000)scores 0.0 s — the same thirty-second hold, one visible and one reporting itself as no hold at all. Gap one omits a file; gap two names a population it did not measure, which is worse.Fixed structurally rather than by documenting the blind spots, because a labelled wrong sum is still a wrong sum. The scan now recurses, reads the bare-millisecond spellings, and reports a duration it cannot read as a FAILURE rather than a zero — so a file with no duration site at all remains a real zero (it holds the lock across straight-line code) and stays distinguishable from a file whose duration was unreadable. That distinction is what was missing, and it is the same move as checking the
TimeSpanthe resolver constructed instead of the double it parsed: make the derivation total rather than enumerate the ways it can be wrong.It reads
CSharpSourceWalker.StripCommentsAndStrings— the shared walk this project already compiles in, rather than a fifth private copy — so a duration written in a comment cannot raise the bar andstring.Joinis not mistaken forThread.Join. That is load-bearing: reading raw text instead reds the readability pin on exactly thatstring.Join.The bar itself is unchanged at 47 s, so the 120 s budget and its 2.5x margin stand. What changed is that the figure is now a measurement of a population the scan can actually see.
Two figures worth accounting for before they are read as evidence
Lite.Tests Total3437 to 3457 is fully accounted and none of it is drift. The issue quotes 3437 from an olderdev; at this branch's measurement base (641c4db7f) the suite was already 3441, measured on both passes of run 34122782002.WriteLockBudgetTestsadds seven[Fact]plus nine[Theory]cases — sixteen — and 3441 + 16 = 3457 exactly. The two dev merges in this branch brought onlyDarling/Darling.Testschanges (assertion shapes in eight files, thenRepoFileAdoptionTests), which move no Lite count.The suite did not get twice as fast; the 485 s in the issue is the outlier. Across the 194 executions the step's p50 is 262 s and its p90 is 346 s; the failing run the issue was filed from sits at p97.9. On one runner, back to back, this branch measured 231.9 s parallel and 449.4 s serialised, and the 259.8 s run is the median of the population, not a halving of it. Nothing in this diff changes how the suite is invoked:
.github/workflows/build.ymlis byte-identical todev(git diff origin/dev -- .github/workflows/build.ymlis empty). The instrumentation commit did alter the invocation — a-trxargument and a second-parallel nonepass — and that is precisely why it is not in this diff.Instrumenting the workflow did not break a guard that reads the workflow. Both passes of the instrumented run reported
Failed: 0withbuild.ymlmodified, so the measurement was taken on a green suite and is not a reading off a failing one.CrossAppGuardCiGateTestsdoes readbuild.yml— inEveryCrossAppSourceRead_IsReachableByTheFilterThatGatesItsSuite, which stayed green throughout — and the arm that failed later reads project files, not workflows.One self-inflicted red, isolated
The first push of this fix failed
CrossAppGuardCiGateTests.TheEvaluatedSet_ContainsEverythingAParseOfTheSameXmlFindswithSet: [](run 34124608714), and it was this change rather than the class's known parallelism flake — which is a file-in-useIOExceptionin a different method. The empty set at line 878 isparsedCross, the CRUDE XML PARSE side, not the evaluated side.ParsedProjectXmlPathsscans with"(?<dq>[^"]*)"|'(?<sq>[^']*)'|>(?<text>[^<>]*)<, so an apostrophe is an attribute delimiter: the five in the new project-file comment left the pairing unbalanced and swallowed every quoted attribute value after it, includingInclude="..\Darling\Darling.Tests\CSharpSourceWalker.cs". Established by a single-variable revert —Lite.Tests/Lite.Tests.csprojback todevwith everything else unchanged turns the real test green in 0.7 s in anet10.0host, and restoring the ItemGroup reds it in 0.3 s — not by re-running. The comment now carries no apostrophe and says why, next to the constraint.That guard under-reads, and its own doc says it cannot. It is written to be crude in the safe direction — "it over-reads — a path inside a comment ... counts — and it under-reads anything needing evaluation" — but an unbalanced apostrophe makes it under-read arbitrary quoted values, which is the unsafe direction, and it reports the result as a moved blind spot rather than as a parse it could not complete. Not touched here: it is a pre-existing defect in a guard with its own issue lineage (#3063, #3074, #3076, #3082), and folding it in would put a change to the cross-app guard inside a write-lock PR. Left as a finding.
Verification
Lite.Testsisnet10.0-windowsand cannot run on macOS, soLite.Tests/WriteLockBudgetTests.csandLite.Tests/ParitySource.cswere compiled unmodified into anet10.0xunit.v3 console host, with the budget members sliced byte-for-byte out ofLite/Services/LocalDataService.csby an extractor that asserts on the real file's text. 31 tests green. Eighteen mutations, each applied and reverted on its own, each verified to have actually landed before the run — one first attempt was a no-op substitution and was redone rather than counted as a pass, and one--no-buildrun reported 28 green off a stale binary while the build was failing, which is why the runner rebuilds every time:DefaultWriteLockBudget5 s to 10 sTheAppKeepsTheDispatchersFiveSeconds+ the 7 default-resolvingOnlyAUsableDeclarationDisplacesTheDefaultcasesOnlyTheTestHostDeclaresABudget,ThisHostResolvedTheBudgetItsProjectFileDeclaresThisHostResolvedTheBudgetItsProjectFileDeclaresonlyOnlyTheTestHostDeclaresABudgetonlyAcquireWriteLock(timeout: TimeSpan.FromSeconds(5))TheCallSiteTakesTheBudgetRatherThanALiteralonlyseconds > 0"0"and"-30"cases onlyCurrentCultureTheDeclarationIsReadInTheInvariantCultureonlyToString()ANonStringDeclarationResolvesTheDefaultonlyTheBudgetOutlastsEveryDeliberateHoldInThisSuiteonlyTheBudgetOutlastsEveryDeliberateHoldInThisSuiteonlyLite.Tests/Helpers/TheBudgetOutlasts…— and nothing at all with recursion off.Wait(200000)TheBudgetOutlasts…— scored zero under the old scanEveryDeliberateHoldInThisSuiteIsReadable, naming the siteStripCommentsAndStringsEveryDeliberateHoldInThisSuiteIsReadableon astring.JoinTheResolverIsTotal...,TheCeilingIsAccepted...TheResolverIsTotal...onlyTimeSpan.MaxValueTheCeilingIsAccepted...only — totality stays greenTheResolverIsTotal...onlyThe
RuntimeHostConfigurationOptiontoAppContext.GetDatapath was verified separately: MSBuild writes it intoconfigPropertiesas a JSON number and the host hands it back asSystem.String, which is why the resolver parses rather than casts, and an absent key returns null.The real
CrossAppGuardCiGateTestswas compiled into the same host for the apostrophe diagnosis above; it andWriteLockBudgetTestsare 30 green together on the merged tree. The parse and overflow behaviour above was measured in the same way rather than reasoned about.The sibling flake is a different mechanism, and this does not fix it
CrossAppGuardCiGateTests.TheEvaluatedRead_FailsLoudlyRatherThanReturningNothingfails with a file-in-useIOExceptionatCrossAppGuardCiGateTests.cs:811— thefinally'sDirectory.Delete(scratch, recursive: true). Its last arm deliberately runsdotnet msbuildwith a 1 ms deadline to prove a hung child fails rather than hangs;RunDotnetthen does a best-effortprocess.Kill(entireProcessTree: true), and the delete races the OS releasing the killed tree's handles on that scratch directory. Same family in the sense that a test manufactures an adverse timing condition and then loses to it under runner load, but the contended resource is a filesystem handle rather thans_dbLock, so nothing here reaches it — and neither would either of the issue's other two options. It fired twice in the 194 runs (34004387846, 34048086326), so it is the more frequent of the two.Left out
DatabaseStateWriteLockTestsrecords that it needs the timeout injected; a host-wide property is the wrong seam for it, because forcing a short budget to observe the skip would put every parallel neighbour on that fuse — a smaller copy of this defect. Covering it wants a per-instance override, which is a separate change from this one.TheCallSiteTakesTheBudgetRatherThanALiteralguards the existing call site by text; a second caller with its ownTimeSpan.FromSecondswould slip past it.CHANGELOG entry text
Not applied here — every lane appends to the same
[Unreleased]block. Under### Fixed:Needs
[#3134]: https://github.com/erikdarlingdata/PerformanceMonitor/issues/3134in the link block.