Skip to content

perf(list): settle picker rows top-down and fork rebase lookups concurrently - #4286

Merged
max-sixty merged 10 commits into
mainfrom
switch-refresh
Sep 27, 2026
Merged

max-sixty merged 10 commits into
mainfrom
switch-refresh

Conversation

@max-sixty

Copy link
Copy Markdown
Owner

In a repo with many worktrees, the wt switch picker took up to ~1.6s after painting its rows before the visible ones dimmed (dim marks a row whose branch is integrated and safe to remove). Two causes, one commit each:

  • Work items now run in queue order (par_bridge instead of into_par_iter). The queue is local tasks in display-row order, then network tasks. into_par_iter gave each pool worker a contiguous slice of it, so one worker ran a row's dozen tasks one after another, and the slices holding the bottom rows and the network tasks started as early as the top rows. With par_bridge, each idle worker takes the next item, so the top row's tasks run at once and rows settle from the top down.
  • Rebase branches of detached worktrees are looked up concurrently. git worktree list reports a mid-rebase worktree only as detached, so list_worktrees resolves each detached worktree's git dir to read its rebase head-name. That was one serial git rev-parse --git-dir fork per detached worktree, before the skeleton. They now fork together through a new Cmd::run_concurrently, which spawns every child before waiting on any. It deliberately avoids a thread pool: this runs inside the list_worktrees cache initializer, and a rayon thread waiting there runs other pool jobs, one of which may read the same cache (wt step prune does, and a par_iter version deadlocked it).

The module docstring of src/commands/list/collect/mod.rs now specs the collect pipeline: each phase, the thread it runs on, every blocking wait, and the rule that keeps the waits deadlock-free.

Measurements

On a real repo with ~140 worktrees (16 detached), 5 runs each:

main this PR
Picker: visible rows dimmed, after rows appear 275–1584ms 150–230ms
wt list skeleton (median) 178ms 103ms
wt list total (median), par_bridge change alone 417ms 461ms

cargo bench --bench list -- full/warm (the full fixture detaches 6 of its 24 worktrees), run alternately on the same machine at load average 3–13:

Round main median (min–max) this PR median (min–max)
1 185.0ms (161.9–315.6) 160.6ms (149.2–272.9)
2 203.2ms (175.8–317.5) 196.8ms (174.9–309.3)

Cost: CI results arrive later

par_bridge starts network tasks only once every local task has been handed out, where into_par_iter happened to start some early. On the same repo with wt list --full (CI cache cleared, 249 gh calls, 5 runs each), the last CI result arrives at a median of 6168ms against 5549ms on main, and the first CI request starts at 592ms against 438ms. The underlying limit is that network tasks share the pool workers and the 32-permit command semaphore with local work, so starting them first would stall every row; giving them their own budget would let them start immediately. This PR leaves that as it is.

The ~40ms rise in the wt list total also comes from par_bridge; I haven't attributed it further.

This was written by Claude Code on behalf of max-sixty

🤖 Generated with Claude Code

max-sixty and others added 9 commits September 25, 2026 15:50
`wt list` and the `wt switch` picker fork `git fsmonitor--daemon start`
for every worktree before the per-row tasks can start. In a repo with
128 worktrees nearly every start reported "already running", and the
forks cost 100-300ms. The pre-start now first connects to the daemon's
IPC socket, the check git's own `start` makes before it refuses, and
forks only when nothing answers.

Also adds a test for the rebase-branch backfill in a linked worktree;
the existing test named for it read no rebase state.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Work items are sorted so network tasks run last, and rows run in display
order within each class. `into_par_iter` ignored that order: it gives
each worker a contiguous slice, so the slices holding the last rows and
the `gh pr list` calls started at once. In the picker for a
128-worktree repo, the top rows could take 0.3-1.6s after first paint to
settle; handing items out through `par_bridge` brings that to ~0.2s.

Total `wt list` time rises by ~40ms (median 417 -> 461ms) on the same
repo: the rows dequeued last now finish last.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
`git worktree list` reports a worktree that is mid-rebase only as
`detached`, so each detached worktree forks `git rev-parse --git-dir` to
find its rebase state. Those forks ran one after another before the first
paint: ~100ms for the 16 detached worktrees in one 143-worktree repo.

They now run as concurrent child processes through a new
`Cmd::run_concurrently`, which spawns every command before waiting on
any. The lookup runs inside the `list_worktrees` cache initializer, where
a thread pool can deadlock: a rayon thread that waits runs other pool
jobs, and `wt step prune`'s jobs read that same cache. Waiting only on
child processes has no such cycle. The loop takes ~12ms, and the skeleton
paints at a median of 103ms instead of 178ms.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Replaces the "Post-Skeleton Operations" tree and "Measured Phase
Timings" table with a "Pipeline" section: which thread each step runs
on, what runs in parallel, every blocking wait, the rule that keeps the
waits free of cycles, and where the pipeline reads git's files directly.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
# Conflicts:
#	src/commands/list/collect/mod.rs
#	src/git/repository/mod.rs
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

@worktrunk-bot worktrunk-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

spawn_captured bypasses the finish_once_started opt-out. It registers each child through the free track_if_cancellable, but Cmd::run goes through the method, which skips registration for commands marked finish_once_started. The assert in run_concurrently rejects the other unsupported builders and lets this one through, so a later caller that batches such a command would still see its children signalled on cancellation. Today's only caller doesn't set the flag, so nothing is affected yet. Calling the method fixes it.

Comment thread src/shell_exec.rs Resolved
spawn_captured registered children through the free track_if_cancellable,
skipping Cmd's finish_once_started opt-out. It now shares one pre-spawn
check (cancellation, then preconditions) with run and pipe_into.

finalize_worktrees now takes the repository being listed, from
list_worktrees, instead of looking up Repository::current().

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@max-sixty
max-sixty merged commit d13e362 into main Sep 27, 2026
39 checks passed
@max-sixty
max-sixty deleted the switch-refresh branch September 27, 2026 01:03
max-sixty pushed a commit that referenced this pull request Sep 27, 2026
Four comments no longer matched the code after yesterday's `wt list`
perf work. This PR changes only comments, no behavior.

- **`src/commands/list/collect/mod.rs`**: the stale-default-branch note
said the picker skips the local branch inventory (#5) when branches are
hidden, so it can fork one `for-each-ref`. Since #4290, the first scan
phase primes `repo.local_branches()` unconditionally. The check reads
the cached inventory, so it never forks.
- **`src/commands/list/ci_status/mod.rs`**: `detect_uncached` said
"PR/MR detection always runs". Since #4289, `platform::detect_ci` skips
the lookup for a branch that was never pushed (`may_head_pr`).
- **`tests/integration_tests/push.rs`**: a test doc named
`finalize_worktree`, which #4286 renamed to `finalize_worktrees`.
- **`src/shell_exec.rs`**: `Cmd::run_concurrently` said at most
`max_concurrent_commands()` children run at once. A background batch
takes a single semaphore permit while other threads hold the rest, so
the cap holds per batch, not per process. The comment now says so.

No test accompanies this, since only comments change.

Co-authored-by: worktrunk-bot <254187624+worktrunk-bot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants