Repository navigation
perf(completion): overlap the git reads a Tab press waits on - #3664
Conversation
The completion benchmarks ran against fixtures carrying two remote-tracking refs, so the `for-each-ref refs/remotes/` scan — the most expensive git call on the path — was measured at a size no real clone has. A clone that has been fetched into for months accumulates an order of magnitude more remote refs than local branches, and past ~100 candidates the completer discards every remote-only branch it just scanned. Add a `remote_refs` knob to `RepoConfig` (materialized by one `update-ref --stdin`, refs spread over distinct commits so the scan pays for one object read each), reachable as `wt-perf setup remotes-L-R`, and two variants over it: `many_remote_refs` for `wt switch <Tab>` and `remove_many_remote_refs` for the completer that never offers remote-only branches and so should not pay for them.
Shell completion forks a whole `wt` per Tab press with the user's finger still on the key, and it has nowhere to hide work: nothing paints progressively, no cache survives the process, and no shell caches a dynamic completion. It was doing every git read in sequence — six forks for `wt switch <Tab>`, one after another. Three changes, none of which alter what the shell is offered: - Call `Repository::prewarm` on this path. It already exists for exactly this, filling the git-discovery, git-config, and user-config caches on three threads, but completion returns from `parse_cli` long before `main` reaches it — so the reads it would have overlapped were each a separate serial fork instead. - Run the three scans behind `branches_for_completion` concurrently. They are independent git calls the function only joins in memory, so in sequence the user waits for their sum where they could wait for the max. - Stop scanning `refs/remotes/` for completers that can never offer a remote-only branch (`wt remove`, worktree-only arguments). The completer now decides that once, up front, and it drives the scan as well as the filter; a fetched-into clone carries far more remote-tracking refs than local branches, making that the most expensive read here. `main` also returns to the completion handler before building the rayon pool — `available_parallelism() * 2` OS threads, spawned eagerly, on a path that reaches no rayon at all. `parse_cli` stops returning `Option` as a result. Measured on `remotes-80-1400` (the shape `bench --bench completion many_remote_refs` covers), M4 Max, against the same binary before the change: `wt switch <Tab>` 40.0ms → 24.4ms, `wt remove <Tab>` 40.3ms → 19.4ms, `wt <Tab>` 22.2ms → 12.8ms. Candidate sets are byte-identical across bash/fish/zsh/nu at 64 completion points, inside a repo and outside one.
…esses `branches_for_completion` groups remote-tracking refs through a `HashMap` and then sorts the result by committer timestamp, most recent first. That sort is stable and the map's iteration order varies per process, so whenever timestamps tie — branches cut from one commit, a bulk import — the surviving order is whatever that run's hashing produced. The candidate list is the same set every time, in a different order every time. It reproduces directly: three `refs/remotes/origin/*` at one commit, and three consecutive completions print three different orders. Break the tie on branch name.
The variants all completed a branch argument against differently-shaped repos, so each one's number was a startup cost and a ref-scan cost added together with no way to tell which had moved. And every one of them was shaped so that a ref scan was the slowest thing in the run, which left the other way completion gets slow — worktree enumeration — unmeasured. - `subcommand_only` completes a subcommand name, reaching no branch code at all: process spawn, prewarm, the clap tree, alias injection, the PATH scan for `wt-*` binaries. That is the floor every other variant also pays, so a regression above it is now attributable to the scans rather than to startup. - `many_worktrees` puts 80 worktrees against few enough refs that `worktree list --porcelain` is the slowest call in the run — measured on this shape it is 11.3ms against 8.5ms for `refs/heads/` and 7.2ms for `refs/remotes/`. A change that traded worktree enumeration for ref scanning previously read as a win on every variant. Measured (M4 Max, 4s samples): subcommand_only 9.2ms, branches_only 17.3ms, with_worktrees 19.9ms, many_remote_refs 18.2ms, remove_many_remote_refs 14.3ms, many_worktrees 22.7ms. Two dimensions were measured and deliberately left out. Completing `-` is the only path that builds the clap tree twice, but at 14.0ms against an 11.3ms floor it catches the same regressions `subcommand_only` does, just with a larger multiplier. Forty aliases cost 0.6ms, which is under the noise of a criterion run on a shared CI box, so a variant for them could not reliably detect anything.
The six variants were doing attribution that criterion can't do. Completion has no phases worth timing apart — it spawns, fills caches on one set of threads, scans refs and worktrees on another, prints, exits — and the calls within a wave overlap, so no split of the wall time says which one moved. Localizing a regression here is a trace's job, the same conclusion this file's `list`/`full` guidance already reaches. What the split cost was real: six variants built four fixtures, and each one re-measured the same startup over a repo deliberately small in every dimension but one. Replace them with `RepoConfig::long_lived_clone` — 80 worktrees, 80 more local branches, 1400 remote-tracking refs, sized from a real reported repo (81/80/1382) and reachable as `wt-perf setup long-lived-clone`. It takes no parameters because there is nothing to vary. One run now covers all three of `BranchCompleter`'s categories, puts both candidate slow calls (`for-each-ref refs/remotes/` at 1400 refs, `worktree list --porcelain` at 80 worktrees) in the same process at a size where each is expensive, and sits past the completer's 100-entry threshold — where the remote refs are scanned and then discarded. Bigger and cheaper: 6.8s to build one fixture against four, and `completion_switch/long_lived_clone` reports 22.4ms. History depth stays shallow on purpose. Every scan reads one commit object per ref, so deepening it costs build time and measures nothing.
worktrunk-bot
left a comment
There was a problem hiding this comment.
Reviewing as a draft — flagging anything that looks worth a quick fix. Mark ready for a full review.
The completion-path changes trace clean: prewarm joins its scoped threads before returning so the caches are warm by the time branches_for_completion reads them; the concurrent-scan re-call pattern is correct (remote_branches()? is gated on the same include_remote_only as its warm-up spawn); and the threshold refactor is output-preserving — in the worktree_only / exclude_remote_only cases the downstream filter already dropped remotes, so not scanning them now yields the same candidate set. Nice.
One thing worth a second look, in the bench fixture rather than the shipped path — the "distinct objects" rationale in add_remote_refs (tests/helpers/wt-perf/src/lib.rs):
The refs are spread round-robin over the commits already on
mainso they name distinct objects … pointing them all atHEADwould collapse that to a single object read repeated, giving a misleadingly cheap number.
long_lived_clone sets commits_on_main: 20 and remote_refs: 1400, so the round-robin (commits[i % commits.len()]) points those 1400 refs at only ~20 distinct commit objects — ~70 refs per object, not one object per ref. Git parses each object once and caches it in-process for the life of the for-each-ref, so %(committerdate) over refs/remotes/ does ~20 object parses plus 1400 cheap ref-iteration hits, not 1400 parses. A real long-lived clone's 1400 remote tips are mostly distinct commits, so the fixture may under-weight exactly the object-read cost this call is meant to expose.
This is the same assumption behind long_lived_clone's "History depth is deliberately shallow — every scan reads one commit object per ref, so deepening history … measures nothing": deepening history is precisely what would raise the distinct-target count the round-robin can reach, so the two comments lean opposite ways on whether object reads scale with ref count.
I haven't measured the magnitude — with 1-file commits the per-object parse is cheap, so the gap may be small in absolute ms. But if the fixture's job is to make for-each-ref refs/remotes/ expensive, either bumping commits_on_main toward remote_refs (at a build-time cost the PR is right to weigh) or softening the "distinct objects" wording to "spread over the available commits" would keep the rationale and the fixture in step.
`create_mixed_repo` already builds the diverse repo this bench wants — 200 commits, branches forked at depths fanned across that history in four states, worktrees in four states, packed refs and a commit-graph — and `full` already runs on it. Adding `RepoConfig::long_lived_clone` next to it was a second, thinner answer to a question that had one: it rebuilt worktrees and branches with none of the varied states, and left `RepoConfig` carrying a `remote_refs` field only it set. Move the remote-ref count onto `create_mixed_repo` instead, where it is the one dimension `mixed` was missing, and give the handle an optional third component. `RepoConfig` goes back to what it is on main. The completion bench becomes `mixed-80-80-1400`; `full` passes 0 and is untouched, so its time series has no discontinuity. Its own fidelity gap — `wt list` scans `refs/remotes/` too, and measures it at two refs — is now a one-argument change if we want it, rather than a fixture to write. `completion_switch/mixed` reports 23.1ms; the fixture builds in 9.7s.
Moving `maybe_handle_env_completion` out of `parse_cli` and up into `main` left three comments describing the old control flow — completion no longer returns *from* `parse_cli`, it answers and returns before `parse_cli` is called at all. Found while reviewing the branch, not by a test: a comment that names the wrong function still compiles, and the next reader of the startup order is the one who pays. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
#3669 consolidated the bench harness onto a shared `FixtureRepo` plus `wt_command` / `run_and_check`, touching the same four files this branch changes. Resolved toward main's harness in every case: - `benches/completion.rs` — adopt main's `run_completion` wholesale. It routes through `wt_command` and, unlike the version here, asserts the measured subprocess actually succeeded. The single `mixed` bench now takes `fixture.path()` rather than joining `repo` itself. - `create_mixed_repo` returns `FixtureRepo`, with `remote_refs` threaded through to `create_mixed_repo_at` as before. - `benches/CLAUDE.md` — main's `create_prune_repo` rename and its new `FixtureRepo` paragraph, plus this branch's `mixed-W-B[-R]` row. Kept from this branch: the `remote_refs` dimension end to end (`SetupConfig::Mixed`, `parse_config`, `add_remote_refs`) and the collapse of `completion_switch` to one id. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`add_remote_refs` said the round-robin makes its refs "name distinct objects", with the scan reading "one commit object per ref". It cannot: the round-robin spreads only over the history it has, so 1400 refs over `create_mixed_repo`'s 200 commits land 7 apiece on ~200 distinct commits, and git parses each object once and reuses it for the rest of the `for-each-ref`. The fixture is unchanged — closing the gap needs a history as deep as the ref count, and `BASE_COMMITS` is shared with `full`, so deepening it would slow that fixture and shift the repo `full` is measured on for the sake of a per-object parse this bench is not primarily about. What changes is that the comment now says so, instead of asserting a cost model the fixture doesn't have. Caught in review against `long_lived_clone`, the parallel fixture since deleted; the same overclaim had survived into the shared-fixture version. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
worktrunk 0.71.0 Created-by: HarmonybrewBot Commit-by: HarmonybrewBot Merged-by: HarmonybrewBot Description: Created by `brew bump` --- Created with `brew bump-formula-pr`.<details> <summary>release notes</summary> <pre>## Release Notes ### Improved - **Forge hosts are classified by exact DNS label**: Host detection matched the bare substring, so `github` anywhere in a hostname selected the GitHub provider — `evil-hub.lumenfield.work` and `dev.azure.com.attacker.example` included. Classification now requires `github`, `gitlab`, or `gitea` as a whole dot-separated label (Azure DevOps matches `dev.azure.com` / `visualstudio.com` as a domain suffix), and userinfo is split at the final `@`, so `https://github.com@attacker.example/owner/repo` resolves to `attacker.example` rather than GitHub. Ports and a trailing DNS root dot are normalized away first, so both transports classify identically. `github.mycompany.com` still resolves, but a host that carries the forge name only as part of a label no longer does — `gitlab-internal.company.com`, `github-mirror.example.com`, and `myhub.lumenfield.work` all become unknown, alongside the lookalikes the change targets. What is lost is forge-derived detail: CI status in `wt list --full`, `wt switch --prs`, and `repo.provider` in `wt list --format=json`, which reads `unknown`. Setting `forge.platform` in `.config/wt.toml` restores all of it, and `wt switch pr:{N}` keeps working either way. Only one case announces itself — a single-label SSH alias of the `github-personal` form, the shape used for multi-account setups: `SSH host alias github-personal is not auto-detected as a forge; enable CI status and wt switch --prs with forge.platform = "github" @ .config/wt.toml`. A self-hosted instance at a hyphenated hostname gets no such message; it simply stops reporting CI. Going the other way, a recognized Gitea or Azure DevOps remote now knows its own change-request vocabulary, so `wt switch 123` against a missing branch suggests `wt switch pr:123` instead of offering both `pr:` and `mr:` forms. (Breaking: a remote whose host merely contains a forge name is no longer detected. Set `forge.platform` in project config.) ([#3662](max-sixty/worktrunk#3662)) - **The retired single-file shell directive is no longer honored**: Wrappers written before 0.37 passed one `WORKTRUNK_DIRECTIVE_FILE`, mixing a trusted directory path with arbitrary shell for the parent to source. The split `WORKTRUNK_DIRECTIVE_CD_FILE` / `..._EXEC_FILE` pair replaced it in 0.37, the fallback has warned since 0.54, and it is now gone. A shell still running the old wrapper fails closed on both halves: `Cannot change directory — shell wrapper is out of date`, with the repair line `To update the shell wrapper, run wt config shell install`, and `--execute disabled because the shell wrapper is out of date` in place of running the payload. That makes a stale wrapper stricter than no wrapper at all, where `--execute` still runs the command directly — deliberate, since a retired wrapper is the one case where worktrunk cannot tell what the parent will do with what it writes. Child processes can no longer inherit the retired capability. (Breaking: a live shell on a pre-0.37 wrapper stops changing directory and stops running `--execute` until `wt config shell install` and a shell restart; nushell needs the reinstall because its wrapper is a static file.) ([#3662](max-sixty/worktrunk#3662)) - **Every hyperlink is underlined**: The statusline underlined its PR reference but not the dev-server port, so nothing marked the port as clickable. Both now route through one helper that emits the OSC 8 sequence and the underline together, closing with `[24m` rather than a full reset so a wrapping color or a dim survives the link. Text that is not a link stays plain: the picker cannot carry OSC 8 through skim, so it prints the same short text with no underline instead of underlining something unclickable, and on a terminal without OSC 8 support a dev-server URL still prints in full and stays copyable. ([#3643](max-sixty/worktrunk#3643), [#3651](max-sixty/worktrunk#3651)) - **Shell completion answers a Tab press in roughly half the time**: Completion forks a whole `wt` process on every Tab press, with the user's finger still on the key, and it has nowhere to hide work — nothing paints progressively, no cache survives the process, and no shell caches a dynamic completion. It was doing every git read in sequence. The reads now overlap, the repository caches prewarm on this path, `refs/remotes/` is not scanned where a remote-only branch can never be offered (`wt remove` and worktree-only arguments), and the handler returns before `main` builds a rayon pool it never reaches. `wt switch <Tab>` goes from 45.7 ms to 27.4 ms on a fixture of 80 worktrees and 1400 remote-tracking refs; on worktrunk's own checkout, 47.9 ms to 29.0 ms, and `wt <Tab>` from 22.2 ms to 12.8 ms. ([#3664](max-sixty/worktrunk#3664)) - **`wt step prune` prunes stale metadata alongside its other removals**: Stale-detached and not-yet-planned prunable candidates were held on the scan lock's write side because the repo-wide `git worktree prune` they used to call raced sibling prunes. Each call now names its own entry, so both shapes join the read-side fan-out; the write side keeps only the candidates that run arbitrary commands or own the terminal — hook-bearing ones, `--foreground` ones, and the deferred current worktree. ([#3655](max-sixty/worktrunk#3655)) ### Fixed - **Removing one worktree no longer unregisters another**: Every stale-metadata cleanup in the removal chain ran a bare `git worktree prune`, which takes no path filter — it walks every entry in `.git/worktrees/` and unregisters each one whose directory it cannot find at that instant. A worktree that is merely absent right now (an unmounted volume, a dropped network mount, a half-finished `mv`) is indistinguishable from a deleted one, so removing worktree A also unregistered bystander B. B's commits and files survive, but its admin directory does not: the index, `ORIG_HEAD`, the per-worktree reflog, `refs/worktree/*` and `refs/bisect/*`, and any in-progress rebase or merge go with it — and `git worktree repair` can relink an admin directory but cannot recreate one. Each cleanup now names the entry it means to remove, so a removal's blast radius equals its intent. ([#3650](max-sixty/worktrunk#3650)) - **A locked worktree whose directory is missing is no longer removed**: The missing-directory fallback was tested before the lock guard, so a worktree that was both locked and currently absent skipped the lock entirely — its registration was pruned and it fell through to branch deletion, which is precisely the case `git worktree lock` exists for (removable media, a network mount, a dropped VPN). The lock is now honored whether or not the directory is present, and `--force` still does not override it, matching `git worktree remove`. Fixes [#3645](max-sixty/worktrunk#3645). ([#3647](max-sixty/worktrunk#3647)) - **A branch that gains a checkout just before deletion is retained**: Branch deletion read worktree topology from cache, so a worktree registered after that read — by a hook, or a concurrent `wt switch` — was invisible, and the ref was deleted out from under it, leaving that worktree unable to resolve `HEAD`. A fresh topology read now sits immediately before the compare-and-swap delete, and a branch found checked out (live or locked) is kept and reported. Git exposes no transaction spanning worktree registration and ref updates, so the window cannot be closed entirely; it is now as narrow as two adjacent commands allow, and either one failing leaves the branch intact. ([#3662](max-sixty/worktrunk#3662)) - **Planning a removal no longer prunes metadata as a side effect**: `prepare_worktree_removal` pruned stale worktree metadata while planning the missing-directory fallback, so every "validate first" path quietly mutated — a refused `wt remove` of the default branch still pruned its entry, `wt remove` pruned before its approval prompt, and the picker pruned on skim's event loop during row validation. Planning is now a pure read, and the prune happens at execution, ahead of the branch deletion that may still be declined. Two related contracts move with it: a failing metadata prune surfaces as a removal failure and abandons the rest of `wt step prune`'s queue instead of being skipped silently, and `--dry-run` plans stale entries through the same gate as every other candidate, so a dirty prunable-but-present entry is correctly left out of the preview. ([#3658](max-sixty/worktrunk#3658)) - **`wt config shell install` previews and confirms the legacy files it reclaims**: Install reclaims its own legacy wrapper paths — the fish `conf.d/{cmd}.fish` and stranded nushell autoload copies — but the deletion never appeared in anything the user saw before it happened. `--dry-run` returned an empty cleanup list, the `?` preview showed only the files to add, and when everything was already configured the cleanup ran and returned without prompting at all. Both previews now render `Will remove <old> (deprecated; now using <new>)`, and the already-configured branch prompts before removing and preserves the file on decline. `--yes` still skips the prompt, so no non-interactive flow changes. Fixes [#3644](max-sixty/worktrunk#3644). ([#3648](max-sixty/worktrunk#3648)) - **Completion offers remote branches in a stable order**: They were grouped through a `HashMap` and sorted by timestamp with a stable sort, so on tied timestamps the order was whatever that process's hashing produced — three consecutive completions could print three different orders. Ties now break on name. ([#3664](max-sixty/worktrunk#3664 See merge request: Harmonybrew/homebrew-core!15534
Shell completion forks a whole
wtprocess on every Tab press, with the user's finger still on the key, and it has nowhere to hide work: nothing paints progressively, no cache survives the process, and no shell caches a dynamic completion. It was doing every git read in sequence.Measured on
mixed-80-80-1400, M4 Max, same binary before and after:wt switch <Tab>On this repo's own checkout,
wt switch <Tab>goes 47.9 ms → 29.0 ms;wt <Tab>(subcommand names, no branch work) goes 22.2 ms → 12.8 ms.The benchmark first
benches/completion.rsran against fixtures carrying two remote-tracking refs and no worktrees — a shape nobody has. It never reached the completer's 100-candidate threshold, where every remote-only branch is discarded after being scanned, so the most expensive call on the path was measured at a size that hid it.It now runs one repo, on the
mixedfixturefullalready uses:mixed-80-80-1400— 80 worktrees in four states, 80 more branches forked across 200 commits of history, and 1400 remote-tracking refs. That covers all three categoriesBranchCompleterdistinguishes, puts both candidate slow calls (for-each-ref refs/remotes/andworktree list --porcelain) in one process at a size where each is expensive, and lands past the 100-entry threshold.completion_switch/mixedreports 23.1 ms; the fixture builds in 9.7 s.The remote-ref count is the one dimension
mixedwas missing, so it goes oncreate_mixed_repoas an optional third component of the handle.fullpasses 0 and is unaffected, so its tracked series has no discontinuity — and its own version of the same gap (wt listscansrefs/remotes/too, at two refs) becomes a one-argument change if we want it.completion_switch's own history does break, and knowingly: its two ids (branches_only,with_worktrees) collapse into onemixed, andcriterion-to-jsonl.pykeys the nightly gist rows by<group>/<id>, so those two series stop accumulating. Worth paying — they measured the fixture that hid this cost in the first place, so carrying them forward would mean maintaining a benchmark that structurally cannot see the thing it exists to watch.Worth calling out explicitly, since #3669 landed while this was open and states that it preserves all 41 existing Criterion IDs: this PR is the one place that number changes, deliberately, and 41 becomes 40.
Deliberately not a variant matrix. Completion has no phases worth timing apart, and the calls within a wave overlap, so no split of the wall time says which one moved — localizing a regression is a trace's job, as
benches/CLAUDE.mdalready says forfull.The changes
Repository::prewarmon this path. It already exists for exactly this, filling the git-discovery, git-config, and user-config caches on three threads — but completion answers and returns long beforemainreaches it, so those reads were each a separate serial fork.branches_for_completionconcurrently. They are independent git calls the function only joins in memory; in sequence the user waits for their sum where they could wait for the max.refs/remotes/where a remote-only branch can never be offered (wt remove, worktree-only arguments). The completer decides that once, up front, and it drives the scan as well as the filter.mainbuilds the rayon pool —available_parallelism() * 2OS threads, spawned eagerly, on a path that reaches no rayon.wt switch <Tab>goes from six serial forks to five in two concurrent waves;wt remove <Tab>to four.A bug found on the way
branches_for_completiongroups remote refs through aHashMapand sorts by timestamp with a stable sort, so on tied timestamps the order is whatever that process's hashing produced — three consecutive completions print three different orders. Fixed by breaking the tie on name, in its own commit.Verification
--createsuppression, and outside a git repo.cargo run -- hook pre-merge --yesgreen on the merge commit, after merging currentmainin: 4491 tests, all lints.cargo bench --bench completion -- --testgreen — worth running explicitly now thatrun_and_checkasserts the measured subprocess succeeded, since the assert is new to this bench.Merged
maintwice while this was open. The second, #3669, consolidated the bench harness onto a sharedFixtureRepopluswt_command/run_and_checkand touched all four bench files this branch changes; every conflict was resolved toward main's harness. Notably the completion bench now uses main'srun_completion, which asserts the measured subprocess actually succeeded — the version here did not, so awtthat failed under benchmark would have been silently timed.Review feedback folded in
The draft-stage review caught the
add_remote_refsrationale overclaiming, and it was right. The comment said the round-robin makes the refs "name distinct objects", with the scan reading "one commit object per ref". It can only spread over the history it has: at 200 base commits, 1400 refs land 7 apiece on ~200 distinct commits, and git parses each object once and reuses it for the rest of thefor-each-ref. So the scan pays ~200 parses plus a cheap iteration hit per ref, not 1400 parses.That gap is now stated rather than papered over. Closing it would need a history as deep as the ref count, and
BASE_COMMITSis shared withfull— deepening it would slow that fixture and shift the repofullhas been measured on, to sharpen a per-object parse this bench isn't primarily about. The ref count is the dimension it exists to vary. (The review raised this againstlong_lived_clone, the parallel fixture since deleted; the same overclaim had survived into the shared-fixture version.)Not addressed
Two things from the report that prompted this, both left alone deliberately:
~/.gitconfigsymlink chain landing on NFS, re-resolved by every git child. That's local environment, not code — though it is exactly why overlapping the forks helps them more than the numbers above suggest.worktrunk.state.<branch>.*markers accumulate without bound: on this checkout, 79 branches have markers and 72 of those branches no longer exist. Real, but a separate subsystem with its own design question about when a marker may be dropped.