Pack the bare caches' refs on the sweep

docs/cleanup.md

@@ -344,6 +344,64 @@ everything else on the machine. Deleting them is a decision with your own
 containers on the other side of it, and `docker system df` is the tool that shows
 you what it costs.
 
+### The bare caches' loose refs, and who packs them
+
+| artifact | who reclaims it | what makes that safe |
+| --- | --- | --- |
+| loose ref files under a bare cache's `refs/` | the background freshness sweep, with one `git pack-refs --all` after each fetch that succeeded | packing changes how a ref is stored and not which refs exist, so there is nothing to prove and nothing to consent to |
+
+Every ref a fetch updates is written as a file, and a file costs a whole
+filesystem block: about 4096 bytes each, against the 81 or so the same ref takes
+as a line in `packed-refs`. Nothing used to collapse them. `pack-refs --auto` is a
+documented no-op on git's `files` ref backend, and `dl` runs no `gc` on a bare, so
+`gc.auto` never gets the chance either.
+
+The cost is one block per **ref**, which means it tracks how many branches and
+tags a remote leaves open and has almost nothing to do with how big the
+repository is. Measured across ten real remotes with `git ls-remote`,
+`torvalds/linux` carries 1887 refs against `microsoft/vscode`'s 5342 and
+`rust-lang/rust`'s 334, with a median around 370. So a whole cache of 20 to 40
+repositories holds something like 30 MB to 60 MB of loose refs, a couple of
+percent of one bare's own size. **Disk is not the reason this is here.**
+
+What carries it is placement. The broad sweep is the only thing in `dl` that
+fetches every head and tag, so it is the only thing that makes loose refs in
+quantity, and it already holds the repository's lock while it does. Packing there
+costs one more bounded `git` call in a scope that just spent its whole network
+budget, and it happens only on a pass that actually fetched. Measured on git
+2.51.1 over a bare of 551 refs, 301 of them loose, those files held 1204 KiB of
+blocks against a 30 KiB `packed-refs` for all 551, and the pack itself took 23 ms.
+
+There is a second payment, and it is banked rather than collected. A guard that
+walks every ref on the bare to decide whether a clone is safe to remove reads one
+file instead of thousands once the refs are packed, and such a probe measured
+2.8 ms against 5.3 ms on that same bare. **No shipped code collects that yet.**
+The bare-side reachability guard is decided and not built, and what ships today
+asks the clone instead. So the saving is a reason to keep this once that guard
+arrives, and it is not a reason this is here now.
+
+**A pack that refuses is not a fetch that failed.** The fetch is the point of the
+sweep and the packing is the optional half, so a refusal becomes a notice
+carrying the repository and git's own words, the record's freshness stamp still
+lands, and the next sweep tries again. Withholding the stamp would make every
+later pass re-fetch the whole repository forever on account of a representation
+change that did not come off.
+
+That notice reaches nobody today, and the honest reading of why is that the sweep
+runs detached with its output discarded, so every notice it raises goes to a null
+descriptor and this is simply the first one that anybody would want to read. What
+a refusal costs while it stays unread is bounded: loose refs are one file per ref
+rewritten in place rather than appended, so a pack that keeps failing holds the
+ref count flat at what one sweep writes instead of growing it.
+
+**Packing does not change what a later prune may delete.** A ref the remote
+retracts is removed whether it was loose or packed: git rewrites `packed-refs`
+through the same ref transaction that unlinks a loose file, and a ref that was
+loose over a stale packed line loses both, so nothing comes back at an old sha.
+The only difference is cost, and it falls on the prune rather than here, since
+removing a packed ref rewrites the whole file where removing a loose one unlinks a
+single path.
+
 ### Reconciling records that disagree
 
 `dl` keeps its own record of every workspace, and devpod keeps one too. They

rust/devlaunch-core/public-api.rest.txt

@@ -2079,6 +2079,10 @@ pub devlaunch_core::flows::repo_manager::CacheNotice::RefNotFetched::branch: all
 pub devlaunch_core::flows::repo_manager::CacheNotice::RefNotFetched::owner: alloc::string::String
 pub devlaunch_core::flows::repo_manager::CacheNotice::RefNotFetched::reason: devlaunch_core::flows::repo_manager::NotRefreshed
 pub devlaunch_core::flows::repo_manager::CacheNotice::RefNotFetched::repo: alloc::string::String
+pub devlaunch_core::flows::repo_manager::CacheNotice::RefsNotPacked
+pub devlaunch_core::flows::repo_manager::CacheNotice::RefsNotPacked::owner: alloc::string::String
+pub devlaunch_core::flows::repo_manager::CacheNotice::RefsNotPacked::reason: alloc::string::String
+pub devlaunch_core::flows::repo_manager::CacheNotice::RefsNotPacked::repo: alloc::string::String
 pub devlaunch_core::flows::repo_manager::CacheNotice::TrackedFilesNotListed
 pub devlaunch_core::flows::repo_manager::CacheNotice::TrackedFilesNotListed::reason: alloc::string::String
 pub devlaunch_core::flows::repo_manager::CacheNotice::WorkspaceCloneRemoved

rust/devlaunch-core/src/clients/git.rs

@@ -586,6 +586,50 @@ impl<'r> Git<'r> {
         self.captured("fetch", &spec)
     }
 
+    /// Collapse every loose ref in the bare into `packed-refs`.
+    ///
+    /// A ref git writes as a file costs a whole filesystem block, typically 4096
+    /// bytes against the ~81 the packed line takes, and [`Git::fetch_all`] writes
+    /// one per ref it updates. Nothing else in devlaunch ever packed them:
+    /// `pack-refs --auto` is a documented no-op on the `files` backend, and no
+    /// `gc` is run on the bare, so `gc.auto` never gets the chance either.
+    /// Measured on git 2.51.1 over a bare with 301 loose refs of 551, the loose
+    /// files held 1204 KiB of blocks against a 30 KiB `packed-refs` for all 551,
+    /// and the pack took 23 ms.
+    ///
+    /// `--all` rather than the default, and the difference is the whole verb here
+    /// rather than a nicety: measured on 2.51.1 against a bare holding 301 loose
+    /// heads and 101 loose tags, a bare `pack-refs` took the tags to zero and left
+    /// all 301 heads exactly where they were. Heads are the population a broad
+    /// sweep of a real repository mostly makes.
+    ///
+    /// **Pure, in the sense that decides where this is allowed to live.** Packing
+    /// changes how a ref is stored and not which refs exist, so it can lose no
+    /// work — and it does not change what a later `--prune` may delete either.
+    /// Measured on 2.51.1: a prune of a packed ref rewrites `packed-refs` through
+    /// the same ref transaction, deletes nothing else, and a ref that was loose
+    /// over a stale packed line loses *both*, so nothing is resurrected at the old
+    /// sha. The only difference is cost, and it falls on the prune: removing a
+    /// packed ref rewrites the whole file where removing a loose one unlinks
+    /// a single path.
+    ///
+    /// Bounded at [`ABOUT_ONE_REPO`] rather than left unbounded like the sweep's
+    /// fetch: this touches no network, so a pack that has not finished in thirty
+    /// seconds is a stuck filesystem rather than a slow remote, and the caller it
+    /// runs under is a detached child whose whole point is that nobody is watching
+    /// it.
+    pub(crate) fn pack_refs(&self, bare: &Path) -> GitAnswer<String> {
+        self.captured(
+            "pack-refs",
+            &SpawnSpec::new(
+                Invocation::new(PROGRAM)
+                    .with_args(["pack-refs", "--all"])
+                    .with_cwd(bare.to_path_buf()),
+            )
+            .with_timeout(ABOUT_ONE_REPO),
+        )
+    }
+
     /// Fetch exactly one branch into the bare cache.
     ///
     /// The launch path's entire network budget, so the time it can hold the repo

rust/devlaunch-core/src/clients/git/tests.rs

@@ -503,6 +503,21 @@ fn the_background_sweep_s_bound_reaches_the_spawn() {
     assert_eq!(timeout(&fake), Some(Duration::from_secs(60)));
 }
 
+#[test]
+fn packing_collapses_every_loose_ref_in_the_bare_under_a_bound() {
+    // `--all` and not the default, which packs tags alone and would leave every
+    // head the sweep just fetched sitting loose. The bound is there because this
+    // touches no network: thirty seconds of `pack-refs` is a stuck filesystem, and
+    // the caller is a detached child nobody is watching.
+    let fake = ScriptedRunner::new();
+
+    Git::new(&fake).pack_refs(Path::new("/cache/o/r/.bare"));
+
+    assert_eq!(strs(&argv(&fake)), ["git", "pack-refs", "--all"]);
+    assert_eq!(cwd(&fake).as_deref(), Some(Path::new("/cache/o/r/.bare")));
+    assert_eq!(timeout(&fake), Some(Duration::from_secs(30)));
+}
+
 #[test]
 fn fetching_one_ref_moves_exactly_that_ref_in_the_c_locale() {
     let fake = ScriptedRunner::new();
@@ -967,6 +982,7 @@ fn nothing_here_spawns_more_than_once_per_verb() {
 
     git.clone_bare("url", Path::new("/cache/.bare"));
     git.fetch_all(Path::new("/cache/.bare"), None);
+    git.pack_refs(Path::new("/cache/.bare"));
     git.fetch_ref(Path::new("/cache/.bare"), "feature");
     git.symbolic_ref(Path::new("/cache/.bare"), "HEAD");
     git.remote_branch_listing(Path::new("/cache/.bare"));
@@ -993,7 +1009,7 @@ fn nothing_here_spawns_more_than_once_per_verb() {
     git.status_porcelain(Path::new("/ws"));
     git.unpushed_commits(Path::new("/ws"));
 
-    assert_eq!(fake.call_count(), 27, "one spawn per verb, 27 verbs");
+    assert_eq!(fake.call_count(), 28, "one spawn per verb, 28 verbs");
     assert!(
         fake.calls()
             .iter()