Repository navigation
agent: seed a worktree's untracked files in one batch, not four execs per file - #143
Merged
Merged
Conversation
… per file
`Worktree::create` copied each untracked file with a stat, mkdir, read and
write. In a sandbox every one of those is an exec round trip, so seeding
cost 4 sequential execs per file (4,011 for 1,000 files); locally the same
loop ran blocking `std::fs` calls on an async worker.
Copy the `ls-files` list in one batch instead: one `spawn_blocking` locally,
and remotely one fixed `sh` script per argv-sized chunk of paths (paths ride
as positional parameters, never as shell text). The script runs `mkdir` only
when the directory changes, since forks are its remaining cost.
New bench `worktree_seed` (median ms, 2 ms injected per exec, ShellFs):
files before (execs) after (execs)
10 306 (51) 76 (12)
100 2,426 (411) 187 (12)
1,000 25,355 (4,011) 1,296 (12)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0152kQKDdP8XRhoeqisJYpWr
jaredLunde
force-pushed
the
worktree-seed-batch-copy
branch
from
October 7, 2026 05:59
2f1cf07 to
db65aaa
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Worktree::seed_from_parentcopied each untracked-but-not-ignored file with four calls (is_file,create_dir_all,read_bytes,write_bytes), one file after another:std::fssyscalls inline on an async worker, againstLocalFs's own rule.This is the "batch I/O instead of one round trip per item" pattern, here with sandbox execs in place of Tokio's blocking pool.
Change
Copy the
ls-files -zlist as one batch (Git::copy_files):spawn_blockingover the whole list.shscript (COPY_FILES_SCRIPT) per argv chunk of paths (COPY_ARGV_BUDGET= 128 KiB). Paths are passed as positional parameters and never as shell text, perShellFs's rule. The script runsmkdironly when the directory changes (ls-filesoutput is sorted), because forks are now the remaining cost.The skip behavior is the same as before: both arms follow symlinks (
[ -f ]/Path::is_file), so a symlink to a file is copied as its content, and a dangling symlink or a file that vanished since listing is skipped. The removedGit::is_file/Git::copy_filehad no other callers.Perf
New bench
benches/worktree_seed.rs:Worktree::createagainst a real repo, with the remote arm onShellFs(what a replica uses) overRealRunnerplus an injected per-exec delay. Median of 3 runs, in ms.The exec count is now flat. What remains at 1,000 files is about one
cpfork per file inside the sandbox, and that cost doesn't grow with latency.Tests
seeding_copies_awkward_untracked_paths_on_both_arms: spaces, a quote with$HOMEand backticks, a leading-, a newline in a file name and in a directory name, a symlink to a file, and a dangling symlink, through bothGit::LocalandGit::Remote.remote_seeding_spans_several_argv_chunks: checks that no file is dropped or duplicated at a chunk boundary.argv_chunks_stay_within_budget_and_keep_every_item_in_order.Locally:
cargo test -p beyond-ai-agentpasses, apart from one flake.exec_endpoint_response_cap::run_applies_the_exec_max_response_bytes_flaghit its 120s deadline once under full-suite load, then passed 3/3 in isolation on this branch; it doesn't touch worktrees. Clippy (-D warnings) anddprint checkare clean.🤖 Generated with Claude Code
https://claude.ai/code/session_0152kQKDdP8XRhoeqisJYpWr