feat(mem-wal): let a backpressure controller hold back memtable freezes

[hamersaw]

Problem

The memtable seal trigger is blind to whatever the flush lands in. All four arms of memtable_reached_flush_threshold are memtable-local, so an embedder that bounds a downstream tier — a shared page cache over L0, in our case — has no way to stop that tier growing.

Refusing writes does not help: a refused write never runs its seal, and the memtables already frozen flush regardless. The tier grows at (flush rate − compaction rate) × time with nothing to bound it.

Change

BackpressureController gains may_seal(), defaulted to true so lance's own behaviour is unchanged — only an embedder bounding a tier lance cannot see has a reason to say no. When a controller says no, maybe_trigger_memtable_flush returns SealOutcome::Blocked instead of freezing, and the active memtable pins at its cap.

A blocked freeze has to refuse the put that needed it. Growing the memtable instead is not an option: the in-memory indexes are pre-allocated to exactly max_memtable_rows, so an overshoot fails the index apply. So:

• the two pre-insert call sites turn Blocked into Error::backpressure — retryable by construction, since the controller admits freezes again once the tier drains;
• the post-insert rotation ignores it, because those rows already landed; the memtable is left full for the next put's pre-insert check to refuse on.

ShardMemory now carries seal_required, which lets a controller tell a write that still fits in the active memtable from one that can only land after a freeze. The first costs the tier below nothing and there is no reason to slow it; the second is what puts the next generation in it. Gating on that distinction bounds the tier without throttling writes that would not have grown it.

It is answered off the published ArcSwap snapshot rather than under the write lock, so admission can decide before the lock the seal is taken under. To keep one predicate, memtable_reached_flush_threshold now delegates to fill_reached_flush_threshold, which both the writer (under the lock) and the snapshot path call — the check the controller decides on and the one the writer acts on cannot drift.

What is deliberately not gated

force_seal_active. It is how drain and drop-table get bytes out of memory and into storage, and they have to be able to seal whatever the tier below looks like.

Testing

test_a_blocked_seal_pins_the_memtable_and_refuses_the_put drives a real ShardWriter with a controller that blocks freezes and asserts the put is refused with a backpressure error, the generation does not rotate, no rows land, and the seal happens once the block lifts.

cargo test -p lance --lib mem_wal::write::tests — 103 passed.

🤖 Generated with Claude Code

https://claude.ai/code/session_01RY5TX2epNsh5VY6KtbrqyU

[lance-gatekeeper]

⚠️ Gate recommendation: approve with a non-blocking risk.

Automatic seal attempts now honor the controller at the writer-lock boundary, and the retry/no-mutation path is covered. The remaining operational risk is that this boolean hook is a soft observation rather than a cross-shard reservation: queued or concurrent seals, plus the deliberate force/close bypass, can still add downstream work after capacity appears full. Embedders using this as an OOM boundary should coordinate reservations and leave shutdown headroom; a permit-bearing seal contract would be the stronger option if a hard cap is required.

[xuanyu-z]

I like this idea, when write can still land without trigger a flush, we should just admit it

[xuanyu-z]

Though just calling out here that for wide rows which likely the ones triggering more frequent freeze, this may always return true unless whoever build on this also scale up the memtable size

[xuanyu-z]

nit: maybe add some seals_blocked_total or puts_refused counters