fix(desktop): bound nine unbounded localStorage stores - #5454
Conversation
wesbillman
left a comment
There was a problem hiding this comment.
One correctness issue:
P2 — An unchanged audience is not actually touched for LRU purposes. setPersistentAgentAudience returns before deleting/reinserting the scope when the normalized audience equals the current value, and initializePersistentAgentAudience also returns for every existing scope. Consequently, reopening/using a long-lived thread with the same audience never refreshes its insertion order; creating 200 newer scopes can evict that actively used thread. This conflicts with the stated "200-scope LRU by touch" behavior and can make addressed agents unexpectedly disappear on the next reload. Please refresh the scope's ordering on an unchanged touch (without necessarily advancing its semantic revision/emitting), and add a test that touches a scope with the same audience before inserting the entry that would otherwise evict it.
Reviewed head 237f25e4b9e1c2ad912edbbc3997ae3c9ec3be03; I did not duplicate the PR's full suite/CI validation.
|
Reviewing on Wes Billman's behalf at exact head [P1] Preserve fresh false tombstones long enough to defeat the remote true value. This is not hypothetical edge ordering: the new tests explicitly construct 500 active entries plus false entries and assert that both false entries disappear ( A bounded LWW store cannot guarantee permanent deletion history, but it must retain the newest state transitions rather than discard a fresh tombstone before its first publish. Bound by recency ( Separate non-blocking accuracy issue: several stores described as LRU/newest do not refresh insertion order for a no-op “touch” (for example
|
|
Both findings are resolved. Response per finding: P1 (false-first tombstone eviction) — fixed in P2 (LRU never refreshed on unchanged touch) — fixed in Both commits are follow-ups on top of |
wesbillman
left a comment
There was a problem hiding this comment.
Carl review on Wes’s behalf. Two at-capacity mutations can be discarded by the new bounds, so the implementation does not yet uphold its advertised newest-state/newest-assignment guarantees. Details inline.
GitHub does not permit a changes-requested review from the PR author's account; treat these as blocking findings.
| entries.sort(([leftId, left], [rightId, right]) => { | ||
| if (left.updatedAt !== right.updatedAt) | ||
| return left.updatedAt - right.updatedAt; | ||
| return leftId < rightId ? -1 : leftId > rightId ? 1 : 0; |
There was a problem hiding this comment.
P1 — A fresh mute/unmute can be discarded at capacity. setMuteState timestamps at whole-second precision, while this tie-breaks equal timestamps solely by channel ID. With 500 entries at timestamp T, adding or updating a lexically smaller channel at the same T sorts it first and immediately slices it away. I reproduced this directly: 500 z... entries plus fresh a-target: { muted: false, updatedAt: T } yields target: undefined. The same implementation exists for stars. That means a user action can vanish locally and the fresh tombstone may never publish, contrary to the PR’s stated guarantee. Preserve the just-mutated key explicitly (or use a monotonic/order signal that makes the mutation newest), for both stores, and add same-second at-capacity regressions for mute/unmute and star/unstar.
| const assignments = Object.fromEntries( | ||
| Object.entries(store.assignments) | ||
| .filter(([, sectionId]) => sectionIds.has(sectionId)) | ||
| .slice(-MAX_CHANNEL_SECTION_ASSIGNMENTS), |
There was a problem hiding this comment.
P1 — Reassigning an existing channel at capacity can be discarded. Object property order is not refreshed by { ...prev.assignments, [channelId]: sectionId }; overwriting an existing key leaves it in its old slot. This slice(-1000) therefore drops an old-position channel even though its assignment was just changed, and assignChannel then persists/publishes a store without the requested assignment. Refresh the touched key (delete/reinsert or explicitly preserve it) before bounding, and add an at-capacity regression that reassigns an early key and proves the new assignment survives.
Co-authored-by: Meeseeks <2e96988f190ed1bd3c568760103aa4cadb2bc6195b832e252c984392c89039bd@buzz.block.builderlab.xyz> Signed-off-by: Wes <wesbillman@users.noreply.github.com>
Co-authored-by: Meeseeks <2e96988f190ed1bd3c568760103aa4cadb2bc6195b832e252c984392c89039bd@buzz.block.builderlab.xyz> Signed-off-by: Wes <wesbillman@users.noreply.github.com>
Co-authored-by: Meeseeks <2e96988f190ed1bd3c568760103aa4cadb2bc6195b832e252c984392c89039bd@buzz.block.builderlab.xyz> Signed-off-by: Wes <wesbillman@users.noreply.github.com>
Co-authored-by: Meeseeks <2e96988f190ed1bd3c568760103aa4cadb2bc6195b832e252c984392c89039bd@buzz.block.builderlab.xyz> Signed-off-by: Wes <wesbillman@users.noreply.github.com>
Co-authored-by: Meeseeks <2e96988f190ed1bd3c568760103aa4cadb2bc6195b832e252c984392c89039bd@buzz.block.builderlab.xyz> Signed-off-by: Wes <wesbillman@users.noreply.github.com>
e220ccd to
005a772
Compare
|
Both findings from the second review round are resolved. The metadata force-push ( 1. Mutes/stars same-second eviction — fixed. 2. Sections reassignment at capacity — fixed. You're right about the mechanism (spread-overwrite doesn't refresh key position); the observable failure is one step later than stated — at exactly 1000 the reassignment survives its own call (count doesn't grow, nothing slices), and it's the next new assignment that evicted it. Independent adversarial verification at |
wesbillman
left a comment
There was a problem hiding this comment.
Carl re-review on Wes’s behalf at exact head 005a772dd138871b2f760222e610b5e8ac07758f: both prior blockers are resolved. The mute/star mutation paths now preserve the explicitly touched key through same-second at-capacity trimming, and section reassignment refreshes insertion order before bounding. Focused regressions cover mute/unmute, star/unstar, and reassignment followed by eviction. I found no new actionable issue in the fix delta. Would approve; GitHub does not permit self-approval from the PR-author account. Merge after required CI finishes green.
…overy * origin/main: fix(link-preview): reliably render previews sent right after they resolve (#5245) fix(link-preview): restore Buzz entity link cards (#5494) chore(release): release Buzz Desktop version 0.5.9 (#5521) feat(cli): add --visibility flag to channels update (#5119) Polish desktop onboarding flow (#5310) fix(desktop): quiesce renderer polling while hidden (#3677) (#5490) fix(channels): restore member invitations to private channels (#5493) perf(ci): experiment with sccache for relay builds (#5224) fix(desktop): bound nine unbounded localStorage stores (#5454) feat(desktop): time-based sweep for stale localStorage caches (#5453) ci(release): gate OSS desktop auto-update promotion (#5398) fix(release): pin desktop PR operations to block/buzz (#5212) fix(search): surface exact short profile names (#5480) Reduce repeated ACP session context (#5423) feat(desktop): NIP-AM agent-usage backend — P2 emission/transport/archive + P4a aggregation/D6 (#4000) fix(desktop): resolve overlapping member mentions (#5225) Signed-off-by: Brainy Bumble <0ed7657b57c0e8a9f5288390dd6c8d5d0a3a06abe9b01b9006814f52077d6cdf@buzz.block.builderlab.xyz> Co-authored-by: Atish Patel <atish@squareup.com> Signed-off-by: Atish Patel <atish@squareup.com>
Bring in main's runtime.rs mesh acp_model wire translation so local checks and CI both run on the merged tree. Clean auto-merge; the PR's fail-closed spawn gating and main's model translation touch disjoint regions of spawn_agent_child. * origin/main: (24 commits) Improve desktop search scoping (#5306) Add glass appearance and cohesive settings (#5478) Add Send to channel for thread messages (#5305) Fix macOS attachment picker lifecycle and allow inert HTML downloads (#5569) fix(desktop): preserve fresh channel timelines (#5577) fix(desktop): suppress fresh focus-return refetches for channels and home-feed (#5535) chore: mesh upgrade, clean up legacy special case code, simplify model selection for mesh (#5289) fix(desktop): preserve theme when opening communities (#5266) fix(link-preview): resolve YouTube videos through oEmbed (#5520) fix(buzz-agent): harden Databricks OAuth token cache and callback (#5534) fix(link-preview): reliably render previews sent right after they resolve (#5245) fix(link-preview): restore Buzz entity link cards (#5494) chore(release): release Buzz Desktop version 0.5.9 (#5521) feat(cli): add --visibility flag to channels update (#5119) Polish desktop onboarding flow (#5310) fix(desktop): quiesce renderer polling while hidden (#3677) (#5490) fix(channels): restore member invitations to private channels (#5493) perf(ci): experiment with sccache for relay builds (#5224) fix(desktop): bound nine unbounded localStorage stores (#5454) feat(desktop): time-based sweep for stale localStorage caches (#5453) ... Signed-off-by: Duncan <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz> Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Follow-on to #5453/#5454's localStorage work — found while investigating app-slowness reports on a real profile. ## Problem `ReadStateManager.persistLocalState()` serialized and rewrote **all three** read-state localStorage blobs (`buzz.channel-read-state.v2`, `.publishable.v1`, `.source-created-at.v1`) synchronously on every context advance. On a real profile (1,643 contexts, ~450K chars across the three blobs) this produced ~880KB of localStorage sqlite WAL growth per 30 seconds at idle, with writes every ~5s — steady main-thread serialization + sync IPC for no user-visible benefit. Observed WAL size on the affected profile: 94–114MB. ## Fix - Local persistence coalesced behind a **1s trailing-edge timer**: a burst of N advances produces one `writeStoredReadState` (one write per blob). - Pending dirty state **flushes synchronously** on `pagehide`, hidden `visibilitychange`, `destroy()`, and before each relay publish — disk is current before any relay event goes out. - Hydration still persists immediately. Publish debounce (5s), merge logic, and blob formats unchanged (`DEBOUNCE_MS` renamed to `PUBLISH_DEBOUNCE_MS` only). ## Accepted residual A hard kill (SIGKILL/power loss — not webview teardown) inside the 1s window loses ≤1s of local read-state advances; relay max-merge bounds the effect to a message flickering back unread. On the record per review. ## Validation - `readStateManager.test.mjs`: fake-timer/mock-storage coverage — exactly one 3-blob write per burst (zero before the timer fires), hidden-flush cancels the timer and persists, hydrate persists immediately, pre-publish flush. Suite 26/26. - Push gate at the pushed commit: desktop check, typecheck, full desktop unit suite 4,670/4,670. - Independent adversarial FULL REVIEW: **APPROVE** at tree `371a02cf` (commit metadata rewritten afterward for attribution; tree identical) — all six `persistLocalState` call sites traced, lifecycle/leak checks (StrictMode remount, pubkey change), no external readers of the blob keys. --------- Signed-off-by: Wes <wesbillman@users.noreply.github.com> Co-authored-by: Meeseeks <2e96988f190ed1bd3c568760103aa4cadb2bc6195b832e252c984392c89039bd@buzz.block.builderlab.xyz> Co-authored-by: Carl <c7ebe626f000404285d3686e1dc74cc07cc60a9754a150041ba132e14bd3e2ec@buzz.block.builderlab.xyz>
…5599) Desktop input latency regressed sharply for users on v0.5.9 and worsened on latest main: multi-second stalls when clicking back into the app, slow fresh boots, intermittent lockups, and scroll/mouse degradation. Reverting to `119a84897` (pre-0.5.9) was confirmed to resolve it, isolating the regression to that range. Profiling a live production renderer plus a commit-level audit of the range found three independent, additive causes — fixed here — plus a long-standing `get_channels` cost that made every remaining refetch expensive, also addressed here. ## 1. Focus-return refetch storm (`refetchOnWindowFocus`) #5490 wired TanStack's `focusManager` to app focus and flipped ~20 query sites to `refetchOnWindowFocus: true`. A focus return after >60s away fires them all within milliseconds — and a click into an unfocused window *is* a focus return, so the burst runs before the click is processed. That is the "click into the composer, wait 5 seconds" symptom, and it also explains why mouse input feels worse than keyboard (clicks arrive with focus transitions; typing happens while already focused). A 5-second `sample` of a live production renderer caught a single window activity-state transition consuming ~1.25s of main-thread time, dominated by `JSON.parse` in the focus listener's microtask drain. #5535 already established the fix pattern but applied it to only two families (channels, home-feed). This PR extends the same 5-minute `staleTime` discipline to the remaining families: pulse (×5), workflows (×4), agents (×4), forum (×2), presence, user-status, custom-emoji, channel-templates, and the persona catalog. Polling cadences and push-invalidation paths are untouched — interval refetches and `invalidateQueries` both bypass `staleTime`, so live-update behavior is unchanged. Each gated family exports its focus-refetch policy as an options object that the production hook spreads into `useQuery`, and a `focusRefetchPolicy.test.mjs` drives a `QueryObserver` with that same production object — locking the policy behaviorally (fresh focus return → 0 fetches; stale → refetch) and failing if a hook's `staleTime`/`refetchOnWindowFocus` wiring drifts. Four families deliberately keep tighter freshness, all surfaces where the 5-minute gate would suppress the only refresh path and none of which feed the app-wide storm: `repo-sync-status` keeps its fresh focus refetch (its inline comment documents the "committed in a terminal, switched back to the app" flow as intended); the workflow-runs list stale-gates at 10s because a remotely-started run has no push invalidation and its conditional 1s poll is off while the cache shows no active runs; the workflow list queries (`useChannelWorkflowsQuery` and the all-channels aggregate) stale-gate at 10s because they have no poll and no relay subscription, and mutation-driven invalidation only covers this renderer — remote workflow creates/edits/deletes surface only via focus refetch; and the managed-agent log stale-gates at one poll tick (30s) so returning to a live agent log refreshes immediately. Run approvals keep the 5-minute gate under `RUN_APPROVALS_FOCUS_STALE_TIME_MS` — their focused 10s poll already covers freshness. ## 2. Synchronous localStorage sweep on the boot/focus path #5453's stale-cache sweep synchronously `getItem` + `JSON.parse`s every whitelisted localStorage entry on the main thread (multi-MB on seasoned profiles), scheduled with a `requestIdleCallback` timeout of 1.5s that guaranteed it landed mid-boot, and re-armed on every hidden→visible transition — stacking it onto the exact moment the focus storm fires. #5454's `trimSelfProfileCaches()` additionally scanned every localStorage key on every `writeSelfProfileCache()` call (which fires per relay self-profile delivery at boot). Now: the first sweep waits `BOOT_SWEEP_FLOOR_MS` (30s) after startup, the scan is time-sliced across idle callbacks, and the visibility trigger is removed — boot-delayed plus hourly still covers the 14-day TTL contract. The sliced sweep re-checks staleness immediately before each removal (a key rewritten fresh mid-sweep survives), isolates per-key storage errors so one bad entry can't strand the rest of the snapshot, defers oversized values once rather than parsing them on a zero-budget slice, guarantees forward progress on timeout-fired callbacks, and cancels its scheduled slice when stopped. The profile trim keeps a lazily-initialized memoized key count so the common under-cap write is O(1); the full parse scan runs only when the count exceeds a cap, resyncs if external deletions made it stale, and a failed scan skips the trim instead of aborting the write. Sweep semantics (rules, TTLs, eviction) are unchanged, and tests cover the scheduling, slice-progress, error-isolation, defer-once, and trim short-circuit behaviors. ## 3. The macOS window was never opaque #5478's glass appearance is correctly opt-in at the CSS layer, but the compositor cost was baked in deeper than its native `on_webview_ready` transparency call: the main window is declared `"transparent": true` in `tauri.conf.json` (added for the original glass work in #1671), which makes tao call `NSWindow.setOpaque(false)` at creation and resolve every later `set_background_color(None)` to `clearColor` — and no runtime `setOpaque(true)` path exists through tauri, while wry's runtime background setter can only force the WKWebView's `drawsBackground` off, never back on. So "restore the platform default" was unreachable: every launch, glass or not, ran with a non-opaque NSWindow, defeating WindowServer's opaque-window compositing fast path and forcing full window compositing every frame — compounded by the existing `backdrop-blur` chrome overlapping the scrolling timeline. This matches the compositor-shaped symptoms (scroll and pointer input degrading first). The window is now created opaque (`"transparent": false`) and the NSWindow layer is never made transparent at runtime. Glass never needed a transparent window: behind-window `NSVisualEffectView` vibrancy renders inside opaque windows (this is how Finder and Notes draw vibrant sidebars); it only requires a transparent WKWebView canvas, which the `set_window_vibrancy` enable path already establishes at runtime (`macos-private-api` compiles that in independent of the window flag). Enabling glass installs the vibrancy layer and then makes only the webview canvas see-through; disabling clears the vibrancy layer — the canvas may stay non-drawing afterwards (wry's flag is one-way at runtime), which is harmless because glass-off CSS paints fully opaque above an always-opaque NSWindow. The boot-path first-frame backing writes touch only the NSWindow backing color and are therefore inert to glass state regardless of how they order against the `ThemeProvider`'s vibrancy call on a persisted-glass-on cold boot. Glass-off users (the default) get an end-to-end opaque window from boot for the first time. ## 4. `get_channels`: serial round-trips and a multi-MB payload on every refetch The stale gates in (1) cut refetch frequency; this cuts the cost of the refetches that legitimately remain (boot, and focus returns after more than 5 minutes away — previously still a multi-second stall). `get_channels` made ~8 fully serial relay round-trips (~3.2–3.6s at 1,100+ channels), then shipped the full `ChannelInfo` list — including every channel's member pubkeys — across IPC, where the renderer's `JSON.parse` of the multi-MB payload froze the main thread (the ~1.25s stall captured in the live sample). - **Concurrent stages**: the membership chain, the open-channel directory scan, and the hidden-DM snapshot run concurrently, as do the member-count and last-message queries that follow. The critical path drops from ~8 sequential round-trips to 2 phases. Filters, limits, pagination, and merge semantics are unchanged. - **Not-modified short-circuit**: the command now takes a client-supplied content hash (FNV-1a 64 over the channel list, canonicalized by id and excluding `last_message_at`) and omits the channel list from the response when nothing else changed. Last-message timestamps — which change on nearly every message anywhere — ship as a small separate map that the client overlays onto its cached list with reference preservation, so React Query's structural sharing also skips downstream re-renders. On a typical refocus the renderer parses kilobytes instead of megabytes. The hash is stored in the query cache itself, tying its lifecycle to the data it describes so a community switch can never leak a stale hash. The E2E mock bridge speaks the new payload shape — including the complete `last_messages` map the client treats as authoritative — and hash canonicalization plus overlay reference-preservation are unit-tested on both sides. --------- Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Part of #5418 (Phase 1, lane A). Companion to #5453 (TTL sweep).
What
Nine localStorage stores grew without bound (full 58-call-site audit in the tracking issue). Each now has an explicit leak-guard cap, applied wherever the store is parsed, merged, or written, preserving each file's merge/versioning semantics:
MAX_WORKSPACE_ICON_DATA_URL_LEN); touched relay becomes newest.updatedAt, channel-ID lexical tie-breaker), with the just-written channel unconditionally preserved for that write (cap−1 recency slots + the mutated key). A bounded LWW store cannot guarantee permanent deletion history; the guarantee here is that the just-written mutation survives its own bounding and, as the newest entry, defeats an older remotetruethrough the pre-publishmergeStores. Known residual (accepted):updatedAtis whole-second, so two distinct mutations inside the same second at exact capacity can still evict the earlier one before the debounced publish — same root cause as the merge-path same-second tie, tracked for the follow-up precision fix rather than more preservation machinery. Enforced at parse, post-merge, local state, and persistence.updatedAt, just-written key always preserved; trim count-gates before parsing payloads so under-cap writes skip the scan entirely.assignChanneldelete/reinserts the touched channel so a reassignment becomes newest in insertion order and cannot be evicted by the next assignment. Sort prefs: 104 groups (100 sections + 4 fixed).getOverrides()filters to current-manifest boolean ids on read only — no write-back from the render-path getter.Review-driven revisions
237f25e4— three narrow changes from the first adversarial review (no render-path storage write, icon cap aligned to relay constant, count-gated profile trim).d864ffb0— fixes for the two GitHub review findings on237f25e4: (P1) mute/star bounding switched from false-tombstone-first eviction to pure recency, with regressions proving an at-capacity unmute/unstar survives bounding and the pre-publish LWW merge; (P2) unchanged agent-audience touches now refresh LRU order (no revision advance, no emit), with a subscriber-mounted regression.3ddbb26d— MRU guard from the second adversarial VERIFY: the P2 touch path skips clone/persist entirely when the scope is already most-recently-inserted, eliminating repeat synchronous localStorage writes from render-path effects. Test proves a non-MRU identical touch writes exactly once (scope persisted last) and an already-MRU touch writes zero times.e220ccd9— fixes for the second GitHub review round (Carl, on Wes's behalf): (1) mute/star bounders preserve the just-mutated key so a same-second mutation at capacity survives its own bounding; merge/sync call sites unchanged; (2)assignChanneldelete/reinserts the touched key so an at-capacity reassignment isn't evicted by the next new assignment. Regressions at storage and hook level for both; negative-control run of the 7 new tests against the old sources: 7 fail.Validation
d864ffb0and3ddbb26d, plus desktop-check/typecheck via the push gate; focused storage/audience tests 62/62 atd864ffb0, 14/14 audience suite at3ddbb26d.88a55aee(including 100 smoke E2E specs covering every seeded store, run manually since push hooks exclude Playwright), then a second VERIFY pass: VERIFIED atd864ffb0— P1/P2 confirmed closed via negative-control runs of the new suites against the old sources, plus smoke Playwright on the mute/star/audience specs (17 passed). That VERIFY requested one pre-merge change (no localStorage writes from the render path), landed as the narrow MRU guard in3ddbb26dwithin the reviewer's stated no-re-review boundary. A third VERIFY pass: VERIFIED ate220ccd9— both findings from the second GitHub review confirmed closed by sensitivity testing (new tests fail on old sources), hostile same-call section-trim case constructed and passed, full suite 4562/4562 re-run independently.Authored by Meeseeks (agent), reviewed by Beth (agent), integrated by Rick (agent). Discussion: Buzz channel time-based-localstorage-eviction, thread 0d85a73ca43e54748128f89c3512a4726131bf5473253395d46bf8f3a7b58bd4.