Skip to content

fix: run copy-ignored outer loop on dedicated copy pool - #1913

Merged
max-sixty merged 1 commit into
mainfrom
fix/copy-ignored-thread-pool-parallelism
Apr 5, 2026
Merged

max-sixty merged 1 commit into
mainfrom
fix/copy-ignored-thread-pool-parallelism

Conversation

@max-sixty

Copy link
Copy Markdown
Owner

The outer .par_iter() in step_copy_ignored ran on the global rayon pool (2× CPU cores), but each iteration called copy_dir_recursive which enters the 4-thread COPY_POOL via install() and blocks. This meant many global-pool threads sat idle waiting for just 4 copy workers — effectively serializing top-level entries rather than copying them in parallel.

Adds copy_pool_install() to wrap the outer loop so all parallelism (top-level entries and recursive directory walks) shares the same 4-thread pool directly. This restores the pre-0.33 behavior of 4 concurrent copy threads across all paths, without the per-folder pool creation that caused EMFILE in #1864.

This was written by Claude Code on behalf of @max-sixty

The outer `.par_iter()` in `step_copy_ignored` ran on the global rayon
pool (2× CPU cores), but each iteration called `copy_dir_recursive`
which enters the 4-thread `COPY_POOL` and blocks. This meant many
global-pool threads sat idle waiting for just 4 copy workers —
effectively serializing the work.

Wrap the outer loop in `copy_pool_install()` so all parallelism
(top-level entries and recursive directory walks) shares the same
4-thread pool directly, matching the pre-0.33 intent of 4 concurrent
copy threads across all paths.

Co-Authored-By: Claude <noreply@anthropic.com>
@max-sixty
max-sixty merged commit 9dff845 into main Apr 5, 2026
25 checks passed
@max-sixty
max-sixty deleted the fix/copy-ignored-thread-pool-parallelism branch April 5, 2026 04:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants