Repository navigation
Scope: watch the workspace so externally-added files appear in the Files tree #2
Description
Activity
This was generated by AI during triage.
Triage findings: the workspace watcher already exists
This ticket's framing assumed the gap needs "a real filesystem watcher on the workspace root, which is where the cost and the cross-platform risk live." Verification against the codebase suggests that watcher is already running, supplied by a dependency.
The entry index is not hand-rolled. It is
FileFinderfrom@ff-labs/fff-node(v0.9.4, Rust), which ships "a real-time file watcher" as a documented feature. The watcher is on by default — disabling it is the opt-indisableWatchoption — and the finder is constructed without that option. So every indexed workspace root already has a recursive, gitignore-aware native watcher attached.The corroborating detail is what the existing refresh path actually does.
WorkspaceEntries.refreshdelegates to the search index's refresh, which callsFileFinder.scanFiles(). The library documentsscanFiles()as "useful after major file system changes that the background watcher might have missed." In other words, the explicit refresh is a belt-and-braces rescan layered on top of a watcher that already exists.Effect on the original five questions
- Q1 (recursive watching across platforms): largely moot. Recursion and cross-platform behaviour are the library's concern and already solved in native code. (Note also that the installed
effectrelease does expose arecursiveoption onFileSystem.watch; the vendored.repos/effect-smolcopy is stale on this point and shows a single-argument signature. Worth avpr sync:reposregardless.) - Q2 (ignore rules): already handled. Gitignore exclusion is provided by
FileFinder, which is why gitignored files are excluded from entry search with no ignore list anywhere in repo code. There is no list to design or place. - Q3 (refresh cost and coalescing): reframed. The worry was a full 25,000-entry rescan per filesystem event; the native watcher updates incrementally instead. Coalescing may still apply to the notification, not to the scan.
- Q4 (watcher lifetime): already answered by construction. The watcher's lifetime is the index's lifetime, which follows the existing 15-minute idle TTL.
- Q5 (remote environments): still open, and still a real judgment call.
The actual gap
FileFinderexposes no change callback. It offers a readiness flag, a scanning-in-progress check and an explicit rescan, but nothing to subscribe to. The index updates itself and never tells the server. So the real question is not "how do we build a watcher" but:How does the server learn that the index changed, given the library exposes no change event?
Narrowed scope for this ticket
Run this experiment first — it may close the ticket outright. In a real repo (populated
node_modules, active.git; not an empty fixture), create a file externally withtouch, then queryprojects.listEntriesfresh, with norefreshcall in between.- If the file appears: the server's data is already live, the remaining gap is purely client-side cache behaviour, and Files tree does not update live when files are added or deleted #1's push channel plus an invalidation trigger is the whole fix. Record that and close this ticket with a much smaller follow-up.
- If it does not appear: the watcher is not feeding the query path as expected. Establish why, then answer the notification question below.
Then answer, with evidence:
- Notification mechanism. Given no callback API, how does the server detect an index change to feed Files tree does not update live when files are added or deleted #1's signal? Compare at minimum: a cheap periodic poll of index state; requesting an event API upstream from ff-labs; a thin notification-only watcher that does no scanning of its own. State the cost of each on a busy repo.
- Churn. Confirm on a real repo that
.gitandnode_modulesactivity does not produce continuous notification traffic. If it does, decide the debounce window and coalescing key —KeyedCoalescingWorkerin@t3tools/sharedalready exists for this. - Remote environments. Does this behave acceptably over SSH-managed and relay-connected environments, or does it need to be opt-out there? This one is unchanged by the findings above and remains a genuine judgment call — flag it for a maintainer rather than deciding it unilaterally.
Deliverable: a comment recording the decisions, plus either a small implementation ticket or a note that #1 already covers it. Then close this one.
Out of scope:
- Building a filesystem watcher from scratch. Establish why the existing one is insufficient before proposing any new watching.
- Duplicating the push channel from Files tree does not update live when files are added or deleted #1. Any notification found here feeds that channel.
- Changing the index's entry cap, scan timeout, or idle TTL.
- Auditing the finder's
enableFsRootScanning/enableHomeDirScanningsettings. Both are enabled while the library documents them as off-by-default for watcher-churn reasons; that is a real question but belongs in its own ticket, not here.
Conventions: read
docs/internals/code-conventions.mdfirst, and the Effect service conventions review-agent prompt under.macroscope/check-run-agents/before touching server services.- Q1 (recursive watching across platforms): largely moot. Recursion and cross-platform behaviour are the library's concern and already solved in native code. (Note also that the installed
Scoping outcome: the experiment closes most of this ticket
Ran the experiment this ticket asked for first, against this repo itself — 17,701 indexed entries, populated
node_modules, active.git. Not a fixture. Probe drivingFileFinderdirectly with the exact optionsWorkspaceSearchIndex.createFinderuses for thepathsvariant.Result: externally-created files already reach the query path with no
refreshcall.[A] external CREATE visible without scanFiles: true after ~103ms [B] external DELETE reflected without scanFiles: true after ~104msSo the server's data is live already. The library's built-in watcher handles it, exactly as the previous comment predicted. Nothing needs to be watched, and no ignore list needs designing —
.gitandnode_modulesare already invisible to the index:60 writes into node_modules/ and .git/ over 1.5s -> index changed on 0/30 samples git status (read-only git): -> no index change idle real repo, 20s: -> 0 index changesThat answers Q2 (ignore rules) and Q3 (churn) outright, with evidence: there is no churn to coalesce. Q1 (recursive watching) and Q4 (watcher lifetime) stay moot — the watcher is the library's, and its lifetime is the index's, which already follows the 15-minute idle TTL.
The remaining gap is exactly the one the previous comment identified:
FileFinderexposes no change callback, so the server never learns the index moved and never publishes #1's signal.
Q1 — Notification mechanism
Measured the three candidates.
Rejected: poll
getScanProgress(). Looks like the obvious cheap signal and is not one.scannedFilesCountis frozen at the initial-scan value and never moves when the watcher updates the index:before burst of 200 external file creates: {"scannedFilesCount":16084,...} after (all 200 visible via search): {"scannedFilesCount":16084,...}Rejected: a second, notification-only filesystem watcher. It would reintroduce every problem this ticket was opened to solve — recursion on Linux, an ignore list,
.gitchurn — to duplicate a watcher the library is already running successfully. Building a watcher to notify us about the watcher we already have is the wrong shape.Recommended: poll the index's own entry count.
mixedSearch("", { pageSize: 1 })returnstotalMatchedwithout materialising the page, and it tracks the watcher exactly:count before=17701 afterCreate=17702 afterDelete=17701 (tracks add/delete: true) cost, pageSize 1: 0.9, 0.8, 0.5, 0.5, 0.4, 0.5, 0.6 ms cost, pageSize 25002: 338, 350, 359 ms <- the full list, for comparison~0.5ms against a 17.7k-entry repo, ~700x cheaper than a full list. At a 1s interval that is 0.05% of one core per subscribed workspace, and it does no scanning — it reads state the watcher already maintains. For reference, the current
scanFiles() + waitForIndexReady()refresh costs 219ms, so this is also far cheaper than the refresh #1 already triggers.No
KeyedCoalescingWorkerneeded: the poll interval is the coalescing, and #1's PubSub is already keyed by normalised workspace root. Poll, compare to the last value, publish the existing coarse signal on change. Client re-reads throughprojects.listEntriesas it does today.Honest gaps, and why they are acceptable:
- A rename, or a balanced add+delete, leaves the count unchanged. The client's existing 30s staleness window still catches those, and in-app renames go through
refresh→ Files tree does not update live when files are added or deleted #1's signal. This is a narrowing of the gap, not a closing of it. - Content-only edits do not move the count. Correct — the tree does not care, and
subscribeProjectFileChangesalready covers open-file contents.
Upstream: the clean fix is a change callback on
FileFinder. Worth raising with ff-labs. The poll is the bridge, not the destination.Q2 — Churn
Answered above: none.
0/30samples on deliberatenode_modulesand.gitwrite churn,0changes on a 20s idle real repo. Nothing to debounce beyond the poll interval itself.Q3 — Remote environments
No opt-out needed, and I do not think this one needs a maintainer's judgment after all. The poll is server-side and never crosses the wire. What crosses is #1's coarse signal, and only on an actual change. Compared with today — every client refetching the full entry list every 30s — this strictly reduces relay and SSH traffic. The cost profile is identical local, relayed, or tunnelled.
Two things this turned up that belong in their own tickets
1. Watcher-readiness race (real bug).
waitForIndexReady()returns while the watcher is still starting:waitForIndexReady returned at 52ms; isWatcherReady=false isWatcherReady became true at 1094ms => ~1.0s blind windowWorkspaceSearchIndex.maketreatswaitForIndexReadyas fully ready and starts serving. Any external change in that ~1s window is missed permanently, until something forces a rescan. My first probe run hit this and produced a false negative before I noticed. The fix looks like also waiting onisWatcherReadyinmake.2.
enableFsRootScanning/enableHomeDirScanningare bothtrueincreateFinder, while the library documents them as off by default because root "floods the watcher with churn-prone events" and makes the caller "responsible for the resulting fs-event volume". This ticket put that out of scope and I have left it there, but the library's own wording is stronger than the previous comment assumed.Deliverable
Decisions recorded above. Follow-up implementation ticket filed. Closing this one.
Model: Claude Opus 5, harness: Claude Code
- A rename, or a balanced add+delete, leaves the count unchanged. The client's existing 30s staleness window still catches those, and in-app renames go through
Scoped. Follow-ups filed:
- Files tree does not update for changes made outside the app #4 — the implementation: publish Files tree does not update live when files are added or deleted #1's existing signal from a cheap entry-count poll. No new watcher.
- Workspace index serves before its file watcher is ready, losing ~1s of external changes #5 — watcher-readiness race found while probing (~1s of external changes lost after index creation).
- Workspace index opts into filesystem-root and home-dir scanning, which the library documents as off by default #6 —
enableFsRootScanning/enableHomeDirScanning, which this ticket put out of scope and asked to file separately.
No code changed for this ticket; its deliverable was the decision above.
Purpose
This is a scoping ticket, not an implementation ticket. Its output is a design decision plus a follow-up implementation ticket, not a merged change. Do not start building from it.
Problem
After #1, the Files tree updates for changes the server makes (agent writes, checkpoints, turn settle). It still does not update for changes made outside the app —
touch test.txtin a terminal, a file added in Finder, a branch checkout done outside T3, a build writing output. Those never pass through the server, so nothing triggersWorkspaceEntries.refresh.Closing that gap needs a real filesystem watcher on the workspace root, which is where the cost and the cross-platform risk live.
What already exists
WorkspaceFileSystem.watchFile(apps/server/src/workspace/WorkspaceFileSystem.ts:337) already watches a directory viafileSystem.watch(directory)and filters events down to a single filename, with a 100ms debounce and symlink handling on both ends. The watching primitive is proven; what changes is the scope of what is watched and what is done with the events.Questions this ticket must answer
fs.watchis reliable on macOS and Windows and weaker on Linux — and remote/SSH environments are Linux. Options: recursive where supported with a degraded path elsewhere; watch a bounded set of directories; or a different watch strategy entirely. Decide, with evidence, and say what happens on Linux..gitchurn constantly, plusnode_modules,dist, build output. Decide what is ignored and where that list lives. Does it respect.gitignore, and if so, at what cost?WorkspaceEntries.refreshis a full rescan, capped at 25,000 entries with a 15s timeout (apps/server/src/workspace/WorkspaceSearchIndex.ts:28-31). Events must be debounced and coalesced per workspace —KeyedCoalescingWorkerin@t3tools/sharedexists for this. Decide the debounce window and prove a busy repo does not rescan continuously.Validate against a real repo
Test against a working project with a populated
node_modulesand an active.git, not an empty fixture. An empty directory will not surface the churn problem, which is the main risk.Deliverable
A comment on this ticket recording the decisions above, plus a new implementation ticket with a concrete plan. Then close this one.
Depends on
#1 — the push channel from that ticket is what any watcher would feed. Do not duplicate it here.
Conventions
Read
docs/internals/code-conventions.mdfirst, and.macroscope/check-run-agents/effect-service-conventions.mdbefore touching server services.