Feature/accelerated file copy - #228
Open
cedric-appdirect wants to merge 5 commits into
Open
Conversation
pjbgf
requested changes
Jul 20, 2026
pjbgf
left a comment
pjbgf
left a comment
Member
There was a problem hiding this comment.
@cedric-appdirect thanks for looking into this. I'm not opposed to introducing a Copy primitive, however, a few points of feedback based on the current proposal:
⚠️ The current implementation breaks the traversal resistance thatos.Rootbrought into the project. The use ofos.Open(srcAbs)inosfs/copy_linux.goweakens that satefy net and opens the osfs implementation to path traversal. We probably can get away with it by using the existing primitives and working off file descriptors.Copysounds like a primitive that likely should be combined with aCapability. So that users can check whether a given implementation supports it.- We may need to refine the API:
Instead of having:
billy.CopyFile(srcFS, "source", dstFS, "target")
We probably should move it to util instead:
util.CopyFile(srcFS, "source", dstFS, "target")
We are probably better off having Copy instead of CopyFile. And CopyFrom instead of CopyFileFrom.
cedric-appdirect
force-pushed
the
feature/accelerated-file-copy
branch
2 times, most recently
from
July 27, 2026 22:11
b9f1b6a to
afe55b8
Compare
Pair the copy feature with a Capability bit so callers can query support via CapabilityCheck, and define the Copier interface plus the UnderlyingFS and RootedWrapper helpers that util.Copy uses to unwrap wrappers to their leaf filesystems. CopyCapability is advertised in AllCapabilities only; a generic filesystem that does not implement Capable is assumed not to support copy. Assisted-by: Cursor with GLM 5.2 <noreply@anthropic.com> Signed-off-by: Cedric BAIL <cedric.bail@appdirect.com>
Copy copies a regular file from one filesystem to another, preserving the source mode. It unwraps both source and destination to their leaf filesystems, dispatches to a leaf Copier when the destination implements one, and otherwise falls back to Lstat + Open + Create + io.Copy. Symlink and directory sources are rejected on both paths. Assisted-by: Cursor with GLM 5.2 <noreply@anthropic.com> Signed-off-by: Cedric BAIL <cedric.bail@appdirect.com>
Open source and destination through each filesystem's own os.Root-backed opener, which rejects escaping symlinks and .. traversal at open time with no TOCTOU, and operate on the resulting file descriptors. Linux uses copy_file_range with a sendfile fallback; other platforms use io.Copy on the contained opens. The path-based darwin clonefile and windows CopyFileEx syscalls are dropped, as they cannot be made provably containment-safe. Symlink sources are rejected via Lstat. Assisted-by: Cursor with GLM 5.2 <noreply@anthropic.com> Signed-off-by: Cedric BAIL <cedric.bail@appdirect.com>
Add containment regression tests proving util.Copy cannot escape the source root: a symlink whose target is outside the root and a .. path must not place outside bytes in the destination. Also add conformance tests for regular-file copy, mode preservation, empty files, truncate-if-exists, and rejection of directory and symlink sources, and per-platform CopyFrom smoke tests. Assisted-by: Cursor with GLM 5.2 <noreply@anthropic.com> Signed-off-by: Cedric BAIL <cedric.bail@appdirect.com>
Benchmark util.Copy across osfs and memfs source/destination pairs. osfs to osfs may use Linux fd-based copy acceleration; pairs involving memfs use Open + Create + io.Copy. Assisted-by: Cursor with GLM 5.2 <noreply@anthropic.com> Signed-off-by: Cedric BAIL <cedric.bail@appdirect.com>
cedric-appdirect
force-pushed
the
feature/accelerated-file-copy
branch
from
August 18, 2026 20:38
afe55b8 to
ec0ee82
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is a dependency and pre work for the tier 3 of this issue: go-git/go-git#1956 . The goal is to provide an API that is clean and fast to copy file across filesystem.