Skip to content

Lite has two conventions for locking a DuckDB write, and one of them is written down as the rule #2463

Description

@erikdarlingdata

Found while ruling on #2455, and deliberately not fixed there — #2455 is one file's lock choice, this is the model it sits in, and changing that inside a three-method PR is how a repo-wide convention gets altered without anyone deciding to.

#2455 asked whether FindingStore's read-lock-around-a-write is correct. It is, on the axis the class doc names: DuckDbInitializer.s_dbLock coordinates everyone against MAINTENANCE (CHECKPOINT, archive DELETEs, compaction), which takes the exclusive write lock, and a held read lock blocks EnterWriteLock — so holding one is how a write says "not while I am in flight". Verified rather than assumed, on DuckDB.NET 1.5.5: two connections appending to the same table in concurrent transactions both commit, and the retention DELETE overlaps an insert batch cleanly.

What that ruling surfaced is that the codebase does not have one convention here. It has two.

The split

Writes that take the WRITE lock — and one of them states it as the rule:

  • LocalDataService.OpenWriteConnectionAsync (Lite/Services/LocalDataService.cs:62), whose own doc comment reads: "Creates and opens a DuckDB connection wrapped in an exclusive write lock. Use for UPDATE/DELETE/INSERT operations that must not race with archival or compaction." That is a house rule, written down, with the same justification FindingStore uses to reach the opposite answer.
  • DuckDbAlertHistoryStore — 6 sites, all ordinary alert-state writes (alert log INSERT, watermark upsert, incident-occurrence delete+insert).
  • DuckDbMuteRuleStore — 5 sites, ordinary mute-rule CRUD.

Writes that take the READ lock:

  • FindingStore.InsertFindingsAsync, MuteStoryAsync, CleanupOldFindingsAsync.

The genuinely maintenance-shaped callers (ArchiveService, QueryStoreSliceRepairService, DuckDbInitializer, the post-collection CHECKPOINT in RemoteCollectorService) take the write lock and are not in question.

So DuckDbMuteRuleStore.InsertAsync writes config_mute_rules under a write lock while FindingStore.MuteStoryAsync writes analysis_muted under a read lock. Same species of write, same store, opposite lock, no note anywhere saying why either is right.

Why it matters more than a tidiness complaint

Both cannot be the rule, and the difference is not free in either direction.

  • If the write lock is the rule, FindingStore is wrong and every finding batch can currently overlap a CHECKPOINT-adjacent operation the other stores exclude.
  • If the read lock is the rule, then 11 write sites serialize themselves against every UI read for no reason — and that is a real cost, because AcquireWriteLock() with no timeout blocks behind archival indefinitely. LocalDataService is the only one that passes a timeout (5 s, explicitly "prevents UI freeze if archival currently holds the lock"); the other ten would simply wait.

My reading is that the read lock is right and the write-lock sites are over-locking, because DuckDB does its own concurrency control and the only thing the lock has to exclude is the file being reorganized underneath a running statement. But that is a judgement about eleven call sites in three files I did not measure, and it should be made deliberately, with the answer written into DuckDbInitializer's class doc so the next writer inherits a rule instead of picking one.

A second, latent hazard in the same model

Worth recording while the lock is in view, and explicitly not a live defect today.

ReaderWriterLockSlim is thread-affine: ExitReadLock throws SynchronizationLockException when called from a thread that did not enter. LockReleaser.Dispose calls it unguarded, and every one of these locks is held across await:

using var readLock = _duckDb.AcquireReadLock(context.CancellationToken);
using var connection = _duckDb.CreateConnection();
await connection.OpenAsync(context.CancellationToken);   // a resumption here on another thread
...                                                       // would make the Dispose below throw

The analysis pass runs under Task.Run with no SynchronizationContext, so a continuation is free to resume on a different pool thread. It does not, and the reason is measured rather than lucky: DuckDB.NET 1.5.5's async methods complete synchronously. Thread id across OpenAsync, a DDL statement, 200 ExecuteNonQueryAsync calls and an ExecuteReaderAsync/ReadAsync drain: 4, 4, 4, 4, 4. Nothing ever yields, so the entering thread is always the exiting thread.

That makes the whole AcquireReadLock/await pattern — 66 call sites in Lite/Analysis alone per #2443 — dependent on an undocumented property of the driver's async implementation. A DuckDB.NET release that made any of these genuinely asynchronous would turn it into intermittent SynchronizationLockExceptions across the store layer, in the paths that hold locks longest. Cheap mitigations if it is worth pre-empting: guard LockReleaser.Dispose with IsReadLockHeld/IsWriteLockHeld, or replace the lock with a SemaphoreSlim (not thread-affine, and it has a genuine WaitAsync(CancellationToken) — which would also retire #2443's 50 ms poll).

Suggested shape

  1. Decide the rule and write it into DuckDbInitializer's class doc, which is where every caller already looks.
  2. Reconcile whichever set of call sites is on the wrong side of it.
  3. Decide the thread-affinity question separately — it is a property of the lock primitive, not of any caller.

Not urgent on any of the three. Filed so the split is recorded rather than rediscovered by the next person who reads FindingStore and reaches for AcquireWriteLock.

Related

Activity

  1. added 2 commits that reference this issue on Aug 21, 2026
  2. erikdarlingdata commented on Aug 21, 2026

    @erikdarlingdata
    OwnerAuthor

    Resolved on dev by #2469 — with zero executable lines changed, which is the correct outcome and not the one the issue expected.

    Two of this issue's premises were wrong, and the second one matters.

    The census was 11 write sites; it is 25. OpenWriteConnectionAsync has fourteen callers the issue missed (ServerTags ×8, DatabaseStates ×3, AlertHistory ×3).

    #2208 had already ruled this axis, in the opposite direction. It moved GetDatabaseStateDeviationsAsync from the read lock to the write lock, off a measured failure. So "unify on the read lock" — the tidy answer this issue invites — would have reverted a deliberate, evidence-based ruling using evidence from a different file. That is exactly the trap in filing a consistency complaint without checking whether the inconsistency was decided.

    They are not two answers to one question. They are answers to two questions, and there is one rule. The read lock excludes maintenance, which is all an append needs. The write lock additionally excludes other writers of the same rows — and that matters here specifically because DuckDB's concurrency is optimistic: it does not queue the loser, it fails it.

    Measured on 1.5.5 with the transactions strictly interleaved: two appends commit; a disjoint retention DELETE commits; two UPDATEs, two upserts, and two delete-then-reinsert compounds each fail the second. Worth noting the first attempt at that measurement, without a barrier, returned "both committed" for everything — a useless artefact, and precisely how this question reads as a tidiness complaint rather than a correctness one.

    That rule describes the existing code with two named exceptions (RecordAlertAsync and MuteRuleStore.InsertAsync, appends holding the write lock), recorded as over-locked and deliberately unchanged. And the headline pair in this issue turns out not to be the same species at all: nothing but the analysis pass writes analysis_muted, while config_mute_rules is UPDATEd by an operator and DELETEd by a timer.

    On the latent Dispose hazard — the mitigation I suggested is a bug amplifier, and that was measured rather than argued. A foreign thread sees IsReadLockHeld == false, so a guard skips the exit; the original thread's entry is then held forever, and with it held TryEnterWriteLock(400ms) fails. The guard would trade a loud SynchronizationLockException for a silently leaked reader that permanently wedges every CHECKPOINT, archival and compaction. Throwing is the better failure, and the doc now says so — so the next reader does not reach for the guard I would have reached for.

    What ships instead is a tripwire on the premise: DuckDbAsyncStillCompletesOnTheCallingThread (200 awaits, Task.Run, no SynchronizationContext). If DuckDB.NET ever goes genuinely async, that fails and the hazard stops being latent.

    Two things deferred deliberately: the two recorded over-locks (a behaviour change for an unmeasured benefit, in the files a reader is most likely to copy from), and replacing ReaderWriterLockSlim with a non-thread-affine primitive — unfiled because there is no trigger yet, and the tripwire is the trigger.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions