Repository navigation
[Bug]: New worktrees stay racily clean, so every checkpoint capture and status refresh re-hashes the whole checkout (2 s instead of 0.2 s on t3code) #21328
Description
Note
Claude Opus 5.5 in Claude Code, writing on behalf of @josephv123. Measured on his machines on 2026-10-18.
Before submitting
- I searched existing issues and did not find a duplicate. Related but different:
captureCheckpointre-hashes the whole worktree on every checkpoint, so large repos always hit the 30s VCS timeout #12924 and [Bug]: Checkpoint capture on large monorepos retries a guaranteed 30s git-add timeout every turn — permanent CPU burn + tmp_pack disk litter #3646: captures seeded a fresh index with no stat data. fix(server): reuse Git index metadata during checkpoint capture #10792 fixed that by copying the user's index. Here the copied stat data exists, but Git doesn't trust it.- [Bug]: Non-sparse skip-worktree entries still force full checkpoint index rebuild after #10792 / #12154 #13741: special index flags make capture throw away the copied index.
- Track performance audit fixes #9661 S3: checkpoint index reuse.
- fix(server): status polling no longer locks the git index #14718: why status polling no longer writes the index.
- I included enough detail to reproduce or investigate the problem.
Area
apps/server
Steps to reproduce
- Create a worktree the way T3 does for a new thread (
GitVcsDriverCore.createWorktree), in a large repository. I used t3code itself (28,271 files, 396 MB):
git -c checkout.workers=0 worktree add -b t3code/<id> -- <path> origin/main - Count the index entries whose mtime falls in the same second as the index file's mtime, or later:
git ls-files --debugagainststat <git-dir>/worktrees/<name>/index. These are the entries Git calls racily clean. - In that worktree, call
CheckpointStore.captureCheckpointa few times with a small edit in between. Also call T3's local status (GitVcsDriver.statusDetailsLocal, which the status broadcaster runs after every turn). - Run
git update-index -q --refreshonce in the worktree, then repeat step 3.
Expected behavior
After at most one full pass, a capture or status refresh in a new worktree costs about the same as in any settled checkout. Git then only re-reads files that changed.
Actual behavior
Every capture and every status refresh re-hashes the whole checkout, and nothing T3 runs ever ends that state.
Measured on lenovolinux (Ryzen 7 8840HS with 16 threads, ext4 on NVMe, Git 2.53.0). I drove the real CheckpointStore, VcsDriverRegistry, VcsProcess and GitVcsDriver layers from origin/main d8e9fd2. The worktree was created as in step 1, then the t3code setup script (node scripts/setup-worktree.ts, which runs vp i) ran as T3 runs it:
| t3code, 28,271 files | fresh worktree | after one git update-index -q --refresh |
|---|---|---|
| Racily clean entries | 28,268 (the checkout took 0.62 s) | 0 |
captureCheckpoint, 5 in a row |
1,987–2,030 ms each | 209–266 ms each |
…of which git add -A |
1,827–1,857 ms | 76–93 ms |
T3 local status (statusDetailsLocal), 3 in a row |
3,734–3,774 ms each | 83–127 ms each |
GIT_OPTIONAL_LOCKS=0 git status --porcelain=2 --branch |
1,722 ms | 67 ms |
git diff-index -M --numstat HEAD -- |
1,964 ms | 18 ms |
A 1,000-file repository (1,000 t3code source files, 22 MB) pays far less. All 1,000 entries were racily clean after a 44–49 ms checkout. Captures took 121–150 ms, against 71–101 ms after a refresh, and T3's status took 120–139 ms against 32–54 ms. The cost grows with the amount of tracked content that has to be re-read.
How much of a checkout ends up racily clean depends on how fast the machine writes it. On acerlinux (Ryzen 7 4700U, 8 threads, nightly 0.1.1-nightly.20261018.3315), real T3 thread worktrees still have their creation-time index days later:
- Two
internship_llm_suiteworktrees created on 2026-10-16 still have all 164 of 164 entries racily clean. - Two t3code thread worktrees,
t3-393a2d75(2026-10-14) andt3-26acb9d2(2026-10-16), still have 664 and 944 racily clean entries, and their index mtime is still the creation time. - In
t3-26acb9d2, a T3-style status run against a copy of that index took 187–212 ms, against 75–78 ms after refreshing a second copy. The real index was left alone.
Why it happens:
- Git's rule. Git only trusts an entry's cached stat data when the file is older than the index file itself. Anything modified in the index's own second or later is "racily clean" (Git's
Documentation/technical/racy-git.adoc), so every status, diff andaddre-reads and re-hashes it. Standard builds compare whole seconds, becauseUSE_NSECis off by default: here, a file 0.8 s older than the index in the same second was still re-read. - Any checkout leaves some.
git worktree addwrites the files first and the index last, so every file written in that last second is racily clean.checkout.workers=0, whichcreateWorktreepasses when the repository doesn't set it, writes in parallel, so a 28k-file checkout fits inside one second on a 16-thread machine. A sequential checkout, asgit clonedoes, left 9,688 of the 28,271 entries racily clean; captures then took 1.33 s instead of 0.21 s. - Git normally clears them right away. The next command that writes the index fixes this, and a plain
git statuswrites it whenever it finds racily clean entries. - In a T3 worktree, nothing writes the index. I checked that the real index stayed byte-identical (same inode, size, mtime and hash) through 10 captures and 3 status refreshes:
- Status polling runs with
GIT_OPTIONAL_LOCKS=0and plumbingdiff-index, which never write the index (fix(server): status polling no longer locks the git index #14718). - Captures copy the index and keep its racy timestamp on purpose (fix(server): reuse Git index metadata during checkpoint capture #10792), then delete the copy.
- The setup script doesn't touch it.
- The agents don't either. Claude Code 2.1.302 starts with
git --no-optional-locks status(twice), and Codex 0.163.0 rangit status --porcelain8 times at startup. Both left the index fully racy, and each of those calls pays the same re-hash: 1,733 ms against 68 ms on the t3code worktree.
- Status polling runs with
So the cost lasts until the agent or the user happens to run a git command that writes the index. When Claude Code ran git status through its Bash tool, that cleared it. On t3code here, a turn's two captures add about 3.6 s to the turn itself, and the status refresh after the turn adds another 3.6 s of git work.
Impact
Major degradation or frequent failure
Version or commit
origin/main d8e9fd2 (captures and status through the real layers). Nightly 0.1.1-nightly.20261018.3315 on both machines.
Environment
- lenovolinux: Ubuntu 26.04, Ryzen 7 8840HS (16 threads), ext4 on NVMe, Git 2.53.0, Node 26.12.
- acerlinux: Ubuntu 26.04, Ryzen 7 4700U (8 threads), Git 2.53.0.
- Claude Code 2.1.302 and Codex 0.163.0.
- macOS not measured: the Mac was offline.
Logs or stack traces
# git processes inside one capture in the fresh t3code worktree (ms), from a wrapper around ProcessRunner
rev-parse=5 rev-parse=5 config=7 rev-parse=5 read-tree=49 ls-files=32 add=1830 write-tree=41 commit-tree=11 update-ref=11
# the same capture after `git update-index -q --refresh`
rev-parse=4 rev-parse=3 config=3 rev-parse=3 read-tree=50 ls-files=19 add=82 write-tree=26 commit-tree=11 update-ref=11Workaround
Run git update-index -q --refresh (or a plain git status) once in a new worktree. On t3code here it takes 1.7 s once.
A possible fix, in GitVcsDriverCore.createWorktree: after git worktree add and the submodule update, run git update-index -q --refresh in the new worktree once.
- Timing. The refresh has to run after the clock has moved past the new index's mtime second. In the 1,000-file repository, an immediate refresh landed in the same second and left all 1,000 entries racily clean; one 1.2 s later cleared them. On t3code the refresh itself takes 1.7 s, so only small repositories would wait, and at most a second.
- Safety. Nothing else uses the worktree yet, so taking its index lock is safe, and a failed refresh can be ignored.
- Cost. It replaces one full re-hash per capture and per status refresh with a single pass.
- Clones. The same refresh would help after T3 clones a project.
I wouldn't make captures write the real index, or let status polling take the index lock again; both work the way they do on purpose.
Activity
juliusmarminge commentedon Oct 18, 2026
Note
Grok responding on behalf of Julius.
Accepting this as a narrow performance bug: nothing T3 runs after creating a worktree ever writes that worktree's index, so the racily clean entries from the checkout are never cleared.
What main does today:
createWorktreerunsgit worktree add, then an optional submodule update and a branch config write. It never runsgit update-index --refreshor an index-writing status in the new worktree.GIT_OPTIONAL_LOCKS=0, and local status usesgit status --porcelain=2plus plumbingdiff-index, so status never saves a refreshed index. That was deliberate in fix(server): status polling no longer locks the git index #14718: background polling must not takeindex.lockwhile the user's own git commands run. A status-time refresh is not on the table.captureCheckpointcopies the real index into a scratch index and sets its timestamp just below the source on purpose (fix(server): reuse Git index metadata during checkpoint capture #10792), so Git keeps revalidating racy entries. The scratch index is then deleted.So the gap is a single missing refresh right after the worktree is created, before anything else uses it. The timing and safety constraints you list (the refresh must land in a later second than the new index, and its failure should be ignored) are the right ones. The measured timings are yours and were not reproduced here. Clones are a separate case and are out of scope for this issue.
Related: #12924 and #3646 (fresh scratch index), #13741 (special flags), #9661 (performance tracker). #21315 is separate.
fix(server): new worktrees no longer re-hash every file on each status
fix(server): new worktrees no longer re-hash every file on each status