Skip to content

fix(server): rescan the workspace index once its file watcher goes live - #7

Closed
Francois3d wants to merge 1 commit into
mainfrom
agent/workspace-watcher-rescan
Closed

Francois3d wants to merge 1 commit into
mainfrom
agent/workspace-watcher-rescan

Conversation

@Francois3d

Copy link
Copy Markdown
Owner

Problem

WorkspaceSearchIndex.make treated finder.waitForIndexReady() as meaning the index was fully live. It is not — that call only covers the scan and warmup phases, while the background filesystem watcher goes live up to a second later:

waitForIndexReady returned at   53ms; isWatcherReady=false
isWatcherReady became true at 1094ms   => ~1.0s blind window

Any external filesystem change in that window was missed permanently: the watcher was not listening, and nothing triggered a rescan afterwards. It bit hardest right after a workspace was first opened, which is exactly when a user is most likely to be creating files.

Fix

Waiting for the watcher before serving would put that second in front of the file tree on every workspace open, so the index still serves as soon as the scan finishes and the window is closed from behind instead. A scoped background fiber waits for the watcher off the critical path, then rescans once to pick up whatever the window swallowed.

  • Workspaces whose watcher is already live when the scan ends skip the rescan, so the extra scan is only paid where a blind window actually existed.
  • A watcher that never becomes ready, or whose readiness cannot be read at all, degrades to a non-live index with a warning rather than taking the index down: search still answers correctly, it just stops tracking edits until the next refresh.
  • Effect.forkScoped ties the fiber to the index's scope, so LayerMap's idle eviction and shutdown interrupt it before the finder is destroyed.

Polling is what the native index itself offers — waitForIndexReady polls the same snapshot — so there is no watcher-ready event to wait on instead.

Evidence

Measured on a 20k-file tree, creating a file externally immediately after the index is constructed:

pre-fix blocking on the watcher this PR
make resolves 54ms 260ms 54ms
external file reaches listEntries ✗ ✓ ✓

The blocking column is the obvious fix, included to show why it was rejected: it buys the same correctness at the cost of putting watcher latency in front of every workspace open.

Tests

A real-finder regression test builds a 20k-file tree, constructs the index, creates a file externally and asserts it reaches listEntries with no refresh(). It fails on pre-fix code and passes here, repeatably — a smaller tree does not separate scan-complete from watcher-ready and stops guarding the window at all.

Four TestClock tests cover the rescan firing, the already-live skip, the never-ready degrade including its warning, and the unreadable-watcher short-circuit.

vp test run apps/server/src/workspace → 69 passed. Typecheck, lint and format clean for the touched scope.

Closes #5

Model: Claude Opus 5 (claude-opus-5) via Claude Code

`WorkspaceSearchIndex.make` treated `waitForIndexReady` as meaning the
index was live, but that call only covers the scan and warmup phases —
the background filesystem watcher goes live up to a second later. Every
external change in that window was lost permanently, because nothing
rescans once the watcher does start. It bit hardest right after a
workspace was first opened, which is exactly when a user is most likely
to be creating files.

Waiting for the watcher before serving would put that second in front of
the file tree on every workspace open, so the index still serves as soon
as the scan finishes and the window is closed from behind instead: a
scoped background fiber waits for the watcher off the critical path,
then rescans once to pick up whatever the window swallowed. Workspaces
whose watcher is already live when the scan ends skip the rescan, so the
extra scan is only paid where a blind window actually existed.

A watcher that never becomes ready, or whose readiness cannot be read at
all, degrades to a non-live index with a warning rather than taking the
index down: search still answers correctly, it just stops tracking edits
until the next refresh.

Closes #5

Model: Claude Opus 5 (claude-opus-5) via Claude Code
@github-actions github-actions Bot added vouch:trusted PR author is trusted by repo permissions or the VOUCHED list. size:M labels Aug 29, 2026
@Francois3d

Copy link
Copy Markdown
Owner Author

Closing as redundant — this change is already on local.

local carries the identical fix at 174322f; git diff origin/local origin/agent/workspace-watcher-rescan over both touched files returns empty. Merging this into main would only make main diverge from upstream/main (currently a clean ancestor of it) to land a fix that has already landed.

The stalled CI here was a separate issue: this branch was cut from main, which still targets blacksmith-* runners. Those are owned by upstream's Blacksmith account and Blacksmith is organization-only, so they can never resolve on a personal fork — the jobs queue forever rather than fail. local carries the remap to GitHub-hosted runners (e12fd54).

local is now the fork's default branch, so future branches inherit both the fork work and the runner remap.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:M vouch:trusted PR author is trusted by repo permissions or the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Workspace index serves before its file watcher is ready, losing ~1s of external changes

1 participant