Skip to content

perf(list): cut warm-cache wt list re-run ~36% by killing redundant per-row git forks - #3334

Merged
max-sixty merged 1 commit into
mainfrom
bench-list-rerun
Jun 30, 2026
Merged

max-sixty merged 1 commit into
mainfrom
bench-list-rerun

Conversation

@max-sixty

Copy link
Copy Markdown
Owner

What

A warm re-run of wt list (where .git/wt/cache/ is already populated) is throughput / critical-path bound on per-worktree and per-branch git subprocesses. Two of worktrunk's three cache layers — Arc<RepoCache> and the process-global path/merge-base/commit-tree DashMaps — live in process memory and don't survive across invocations, so every re-run re-forks work that's content-addressed or already in hand. The "~1 ms recompute, not worth a disk file" assumption behind keeping merge-base and commit→tree in memory is false at scale on macOS (10–25 ms per fork × dozens of rows).

This eliminates three classes of those forks. On a 24-worktree / 120-branch mixed-state fixture, subprocess work drops 3629 ms → 1013 ms; the criterion rerun_warm benchmark goes 212 → 135 ms median (−36%, p<0.05). What remains is the irreducible working-tree floor — per-worktree git status, and add -A/diff/write-tree for dirty worktrees — which depends on live working-tree state and genuinely can't be cached.

The three changes

  1. %T primes the commit→tree cache (diff.rs). The pre-skeleton git log --no-walk batch already resolves each commit for %ct; adding %T rides along for free and primes commit_tree, so the per-row CommittedTreesMatch / WouldMergeAdd tree lookups never fork git rev-parse <sha>^{tree}.
  2. Persist merge-base (sha_cache.rs, diff.rs). New content-addressed merge-base/ sha_cache kind (in-memory front over disk back) plus a sha1 == sha2 short-circuit. The AheadBehind orphan check forked git merge-base per row even when ahead/behind counts were already cache-warm; now it's a file read on re-runs.
  3. Prime worktree root/git-dir (mod.rs, collect/mod.rs). git worktree list already gives every worktree's path; seeding WORKTREE_ROOTS/GIT_DIRS from it (git-dir read from each .git entry, mirroring prewarm_rev_parse's canonicalization) eliminates the per-worktree git rev-parse --show-toplevel / --git-dir forks. Runs post-skeleton — its values are consumed only by the worker pool, so it stays off the skeleton critical path and is skipped under WORKTRUNK_SKELETON_ONLY (measured: skeleton time unchanged at 24.4 ms).

All three are behavior-preserving: caches are content-addressed and never stale, and any value that can't be derived cleanly falls back to the original subprocess.

Benchmark

Adds cargo bench --bench list rerun_warm (24 worktrees + 120 branches in varied states: clean / unstaged / staged+untracked worktrees; merged / ahead / diverged / identical branches), a reusable wt_perf::create_mixed_repo fixture, and a wt-perf setup mixed-W-B config. The existing list benches are warm-capable but each cover worktrees-only or branches-only in a single uniform state. A TODO(bench-full-combined) marks consolidating these into one cold+warm "full" scenario with per-feature trace attribution.

Reviewer orientation

  • Cache mechanics: src/git/repository/sha_cache.rs (new kind) and the # Caching docstring in src/git/repository/mod.rs (updated to document the in-memory-front-over-disk-back decision and correct the stale cost premise).
  • The list-collect path: src/commands/list/collect/mod.rs module docstring (caching table + %T fork doc updated).
  • Tests: cache priming, merge-base roundtrip (incl. orphan), and a prime-vs-subprocess equivalence test that pins the derived git-dir against the real git rev-parse across main + linked worktrees.

This was written by Claude Code on behalf of max

… per-row git forks

A warm re-run of `wt list` (persistent `.git/wt/cache/` already populated) is
throughput/critical-path bound on per-worktree and per-branch git subprocesses.
Three in-memory-only caches die with each `wt` process, so every re-run
re-forked work that is content-addressed or already in hand — and the
"~1 ms recompute, not worth persisting" premise behind them is false on macOS
(10–25 ms per fork, times dozens of rows).

Three fixes, each removing a class of per-row forks:

- Prime the commit→tree cache from the existing pre-skeleton `git log` batch by
  reading `%T` (git already reads the commit object for `%ct`), eliminating the
  per-row `git rev-parse <sha>^{tree}` forks for zero new subprocesses.
- Persist merge-base to a new content-addressed `merge-base/` sha_cache kind
  (in-memory front over disk back), plus a `sha == sha` short-circuit, so the
  orphan check stops re-forking `git merge-base` per row on every invocation.
- Prime `WORKTREE_ROOTS`/`GIT_DIRS` from the worktree list `git worktree list`
  already fetched (root = the path; git-dir read from each `.git` entry),
  eliminating the per-worktree `git rev-parse --show-toplevel` / `--git-dir`
  forks. Runs post-skeleton so it stays off the skeleton critical path.

Adds a `rerun_warm` benchmark (combined many-worktrees-and-branches fixture in
mixed states, `wt_perf::create_mixed_repo` + a `mixed-W-B` setup config) — the
existing list benches are warm-capable but each cover worktrees-only or
branches-only in a single uniform state.

Subprocess work on the 24-worktree/120-branch fixture drops 3629 ms → 1013 ms
(the remainder is the irreducible working-tree floor: per-worktree `status`, and
`add -A`/`diff`/`write-tree` for dirty worktrees). Criterion `rerun_warm`:
212 ms → 135 ms median.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@max-sixty
max-sixty merged commit f045e0f into main Jun 30, 2026
37 checks passed
@max-sixty
max-sixty deleted the bench-list-rerun branch June 30, 2026 00:25
@worktrunk-bot worktrunk-bot mentioned this pull request Jun 30, 2026
social4hyq pushed a commit to social4hyq/homebrew-core that referenced this pull request Sep 20, 2026
worktrunk 0.64.0

Created-by: HarmonybrewBot
Commit-by: HarmonybrewBot
Merged-by: HarmonybrewBot
Description: Created by `brew bump`

---

Created with `brew bump-formula-pr`.<details>
  <summary>release notes</summary>
  <pre>## Release Notes

### Improved

- **`{{ git.branch.* }}` template namespace for custom columns**: `wt list` custom columns can now read a branch's own git config via `{{ git.branch.<key> }}` — both convention keys you set yourself (`branch.<name>.jira`) and the git-native `branch.<name>.description` — without re-storing the values through `wt config state vars set`. It complements `{{ vars.* }}`, which reads only worktrunk's own state namespace. ([#3319](max-sixty/worktrunk#3319), thanks @cazador481 for the request)

- **Faster warm-cache `wt list` re-runs**: A repeat `wt list` (with `.git/wt/cache/` already populated) is ~36% faster, by removing redundant per-row git subprocesses — priming the commit→tree cache from the `%ct` batch already issued, persisting merge-base results to a content-addressed on-disk cache, and seeding worktree roots and git-dirs from the single `git worktree list` instead of re-forking `git rev-parse` per worktree. ([#3334](max-sixty/worktrunk#3334))

### Fixed

- **Refs that look like flags can't inject git options**: `wt`'s `git diff` previews and diffstats (picker diff previews, `show_diffstat`, push diffstat) now fence user-controlled refs behind `--end-of-options`, and branch removal (`git branch -D`) passes them after `--`, so a branch literally named like a flag (`-x`, `--foo`) reaches git as a positional ref instead of being misparsed as an option. ([#3317](max-sixty/worktrunk#3317))

- **Statusline renders untruncated when `COLUMNS=0`**: `wt list statusline` treated `COLUMNS=0` as a zero-width budget and dropped every segment, rendering an empty line. A zero or unparsable `COLUMNS` is now treated as no detectable width, so the line renders everything untruncated — as the statusline sizing already documented for a missing width. ([#3318](max-sixty/worktrunk#3318))

- **Watchdog "still waiting" line uses the hint symbol**: The transient `Waiting for the commit message (Ns)` line shown during a slow `wt step commit` LLM call now uses the hint prefix (`↳`) instead of the info symbol (`○`), matching the convention for fully-dim status lines. ([#3330](max-sixty/worktrunk#3330))

## Install worktrunk 0.64.0

### Install prebuilt binaries via shell script

```sh
curl --proto '=https' --tlsv1.2 -LsSf https://github.com/max-sixty/worktrunk/releases/download/v0.64.0/worktrunk-installer.sh | sh && wt config shell install
```

### Install prebuilt binaries via powershell script

```sh
powershell -ExecutionPolicy Bypass -c "irm https://github.com/max-sixty/worktrunk/releases/download/v0.64.0/worktrunk-installer.ps1 | iex"; git-wt config shell install
```

### Install prebuilt binaries via Homebrew

```sh
brew install worktrunk && wt config shell install
```

## Download worktrunk 0.64.0

|  File  | Platform | Checksum |
|--------|----------|----------|
| [worktrunk-aarch64-apple-darwin.tar.xz](https://github.com/max-sixty/worktrunk/releases/download/v0.64.0/worktrunk-aarch64-apple-darwin.tar.xz) | Apple Silicon macOS | [checksum](https://github.com/max-sixty/worktrunk/releases/download/v0.64.0/worktrunk-aarch64-apple-darwin.tar.xz.sha256) |
| [worktrunk-x86_64-apple-darwin.tar.xz](https://github.com/max-sixty/worktrunk/releases/download/v0.64.0/worktrunk-x86_64-apple-darwin.tar.xz) | Intel macOS | [checksum](https://github.com/max-sixty/worktrunk/releases/download/v0.64.0/worktrunk-x86_64-apple-darwin.tar.xz.sha256) |
| [worktrunk-x86_64-pc-windows-msvc.zip](https://github.com/max-sixty/worktrunk/releases/download/v0.64.0/worktrunk-x86_64-pc-windows-msvc.zip) | x64 Windows | [checksum](https://github.com/max-sixty/worktrunk/releases/download/v0.64.0/worktrunk-x86_64-pc-windows-msvc.zip.sha256) |
| [worktrunk-aarch64-unknown-linux-musl.tar.xz](https://github.com/max-sixty/worktrunk/releases/download/v0.64.0/worktrunk-aarch64-unknown-linux-musl.tar.xz) | ARM64 MUSL Linux | [checksum](https://github.com/max-sixty/worktrunk/releases/download/v0.64.0/worktrunk-aarch64-unknown-linux-musl.tar.xz.sha256) |
| [worktrunk-x86_64-unknown-linux-musl.tar.xz](https://github.com/max-sixty/worktrunk/releases/download/v0.64.0/worktrunk-x86_64-unknown-linux-musl.tar.xz) | x64 MUSL Linux | [checksum](https://github.com/max-sixty/worktrunk/releases/download/v0.64.0/worktrunk-x86_64-unknown-linux-musl.tar.xz.sha256) |



### Install via Cargo

```sh
cargo install worktrunk && wt config shell install
```

### Install via Winget (Windows)

```sh
winget install max-sixty.worktrunk && git-wt config shell install
```

### Install via AUR (Arch Linux)

```sh
paru worktrunk-bin && wt config shell install
```
</pre>
  <p>View the full release notes at <a href="https://github.com/max-sixty/worktrunk/releases/tag/v0.64.0">https://github.com/max-sixty/worktrunk/releases/tag/v0.64.0</a>.</p>
</details>
<hr>

See merge request: Harmonybrew/homebrew-core!13223
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants