Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 38 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,44 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

### Added

- **`dl --prune` now reclaims the Docker volumes of workspaces devpod has already
forgotten.** Deleting a workspace through `dl` has removed its two named volumes
since devlaunch#325, and the names come from devpod's own record of what it
substituted, read at delete time and never guessed from a pattern. That leaves
one hole, and it is the one that was measured: a workspace deleted by a bare
`devpod delete` outside `dl`, which removes devpod's record and leaves the
volumes. After that there is nothing on the machine that names them.

So `dl` keeps its own copy. At the tail of every `up` that completed, it reads
the same two fields out of the same `workspace_result.json` and writes them to a
small per workspace file under its cache, beside the tool verdict markers. A
prune reclaims from those copies, and the whole of what it asks per copy is
whether any workspace `devpod list` returns carries that id. Where none does,
the workspace is gone and its volumes are leftovers.

Nothing is invented at either read, which is the point: every name that reaches
`docker volume rm` still came out of a substitution devpod performed and wrote
down. Matching `<basename>-pixi` and `dind-var-lib-docker-<id>` against `docker
volume ls` stays refused, because the `docker-in-docker` feature writes that
second name in every devcontainer tool that runs it, so the candidates `dl`
cannot attribute are exactly the ones belonging to somebody else.

A copy can still be wrong, in exactly two ways, and neither is answered by
believing the file. It can name a volume that is already gone, which `docker
volume rm --force` treats as a success. It can name one something else now
holds, which Docker refuses with `volume is in use`, reported, nothing removed,
and the copy kept so the retry is still there. A copy is dropped once, when a
removal came back removed for a workspace devpod does not list.

Two things follow. A run pointed at a scratch `XDG_CACHE_HOME` finds no copies,
so it names no volume and removes none, which is what makes the scratch
convention safe here by construction. And the 39 orphaned volumes holding 37.28
GB measured on the reference host are **out of scope and stay**: their records
died before any of this existed, and no route reaches them that is not the
pattern above. `docs/cleanup.md` carries the whole of it.

## [0.25.0] - 2026-08-28

### Fixed
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -253,7 +253,7 @@ instead. [docs/cli.md](docs/cli.md) has the full `--rm` contract, including whic
| `dl --ls` | List every workspace |
| `dl --ls --json` | The same, machine-readable, with what each workspace would lose if deleted |
| `dl --ls --size` | Add what deleting each one would free. Opt-in: it walks every file |
| `dl --prune` | Remove the clone directories no workspace opens any more |
| `dl --prune` | Remove the clone directories no workspace opens any more, and reclaim the volumes of workspaces devpod has forgotten |
| `dl --reconcile` | Re-point workspaces whose recorded source folder went missing. Deletes nothing |
| `dl --purge` | Remove devlaunch's own workspaces and caches |
| `dl --install` | Install shell completions |
Expand Down Expand Up @@ -368,7 +368,7 @@ different jobs:

| Command | Takes | Leaves |
|---|---|---|
| `dl --prune` | Clone directories no workspace opens | Every workspace, container, image and volume |
| `dl --prune` | Clone directories no workspace opens, and the volumes of workspaces devpod no longer lists | Every workspace, container and image |
| `dl --purge` | The workspaces devlaunch created, and its caches | Workspaces it did not create, named before it asks |
| `dl --reconcile` | Nothing | Repairs records that stopped matching the disk |

Expand Down
56 changes: 50 additions & 6 deletions docs/cleanup.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,48 @@ be made worse by it:
frees](#the-disk-neither-command-frees). That boundary is about images now, and
deliberately stays there.

### The volumes of a workspace devpod has already forgotten

The read above happens at delete time, immediately before `devpod delete` takes
devpod's record away with the workspace. That closes the leak for every delete
that goes through `dl`. It does nothing for a workspace deleted some other way: a
bare `devpod delete`, or a devpod home that was cleared out. devpod removes its
own record and leaves the volumes, and after that there is nothing on the machine
that names them.

So `dl` keeps its own copy. At the end of every `up` that completed, it reads the
same two fields out of the same `workspace_result.json` and writes them to a small
per workspace file under its cache, beside the tool verdict markers:
`~/.cache/devlaunch/workspace-copies/<workspace>.json`. `dl --prune` reclaims from
those copies, and the whole of what it asks per copy is: does any workspace
`devpod list` returns carry that id? Where none does, the workspace is gone and
its volumes are leftovers.

Nothing is invented at either read. Every name `dl` hands `docker volume rm` still
came out of a substitution devpod performed and wrote down, which is the rule
above and is unchanged. The alternative, matching `<basename>-pixi` and
`dind-var-lib-docker-<id>` against `docker volume ls`, stays refused: the
`docker-in-docker` feature writes that second name in every devcontainer tool that
runs it, so the candidates `dl` cannot attribute are exactly the ones that belong
to somebody else, and a volume is not an image. The wrong answer there is data
loss, not a rebuild.

A copy can still be wrong, and it can be wrong in exactly two ways. It can name a
volume that is already gone, which `docker volume rm --force` treats as a success
and says nothing about. It can name a volume something else now holds, which
Docker refuses: `volume is in use`, reported, nothing removed, and the copy kept so
the retry is still there. Neither is caught by trusting the file. A copy is dropped
once, when a removal came back removed for a workspace devpod does not list, which
is the one moment it is provably pointless.

Two consequences worth knowing. A run pointed at a scratch cache
(`XDG_CACHE_HOME=...`) finds no copies at all, so it names no volume and removes
none, which is what makes the scratch convention safe here by construction. And the
**39 orphaned volumes measured on the reference host are out of scope**: their
records died before any of this existed, no route reaches them that is not the
pattern above, and they stay. See [the disk neither command
frees](#the-disk-neither-command-frees).

### What a delete says while it does it

`dl <ws> rm` names the workspace going in and again once it has gone, both on
Expand Down Expand Up @@ -224,8 +266,9 @@ until now nothing removed them: measured on one host, **52 clone directories for
all-or-nothing: the only way to get the 4 GB back was to destroy the 7.86 GB
too, and every bare cache with it.

`dl --prune` removes exactly the clone directories no live workspace opens. It
never deletes a devpod workspace, a container, an image or a volume, never
`dl --prune` removes exactly the clone directories no live workspace opens, and
reclaims the Docker volumes of workspaces devpod no longer lists. It never
deletes a devpod workspace, a container or an image, never
touches a repo's `.bare` cache (0.08 GB for seven repos, and it is what makes
the next clone of a repo fast), and never looks outside
`<cache>/devlaunch/repos`. Every directory it finds is one of three things:
Expand Down Expand Up @@ -331,10 +374,11 @@ exception: that is where somebody is deciding what is worth deleting.
(devlaunch#325). Deleting a workspace now removes the named volumes its
devcontainer created, see [what a delete takes with
it](#what-a-delete-takes-with-it), so a disclaimer that still covered them would
be describing a leak that has been fixed. The `--prune` half of the pair still
frees no volume at all, and that is not an oversight either: it removes clone
*directories* and never deletes a workspace, so there is no workspace whose
volumes it could be taking.
be describing a leak that has been fixed. The `--prune` half of the pair frees
volumes too, and its reason changed rather than disappeared: it still never
deletes a workspace, it reclaims the volumes of one devpod has already forgotten,
from [the copy `dl` keeps](#the-volumes-of-a-workspace-devpod-has-already-forgotten).
The 39 orphans above are not among them. Nothing reaches those but a pattern.

**It is a sentence, not a measurement.** `dl` runs no `docker` command to print
it, so there is nothing to be slow and nothing to fail where Docker is absent,
Expand Down
8 changes: 8 additions & 0 deletions docs/development.md
Original file line number Diff line number Diff line change
Expand Up @@ -149,6 +149,14 @@ working tree; on a host, `./dev.sh` installs it as `dl-next`/`aid-next` beside t
print a `-dev` version so a working-tree build is never mistaken for a released one. See
[AGENTS.md](../AGENTS.md).

One thing to know about pointing a run at a scratch cache (`XDG_CACHE_HOME=/tmp/...`), which
AGENTS.md recommends for anything near storage. A run that *creates* a workspace writes its copy
of that workspace's volume names into the cache it is about to throw away, and that copy is the
only record of those names once devpod's own record goes. Remove such workspaces with `dl <ws> rm`
before discarding the scratch cache, or their volumes outlive everything that could name them. See
[the volumes of a workspace devpod has already
forgotten](cleanup.md#the-volumes-of-a-workspace-devpod-has-already-forgotten).

### The Quickstart's demo GIFs

The four GIFs in the Quickstart come from VHS tapes in `docs/demo/`:
Expand Down
2 changes: 1 addition & 1 deletion rust/devlaunch-core/public-api.api.txt
Original file line number Diff line number Diff line change
Expand Up @@ -33,5 +33,5 @@ pub fn devlaunch_core::api::identity(&str) -> core::result::Result<devlaunch_cor
pub fn devlaunch_core::api::json_document(&[devlaunch_core::flows::listing::ListedWorkspace]) -> serde_json::value::Value
pub fn devlaunch_core::api::parse(&str) -> devlaunch_core::domain::spec::WorkspaceSpec<'_>
pub fn devlaunch_core::api::resolve_devcontainer_ref(&str) -> core::result::Result<devlaunch_core::domain::spec::DevcontainerPath, devlaunch_core::domain::spec::DevcontainerRefError>
pub fn devlaunch_core::api::workspace_delete(&mut devlaunch_core::flows::listing::CommandContext<'_>, &mut devlaunch_core::flows::lifecycle::Refresh<'_>, &devlaunch_core::flows::workspace_clone::WorkspaceCloneManager<'_>, &mut devlaunch_core::domain::metadata::MetadataStorage, core::option::Option<&devlaunch_core::clients::devpod_home::DevpodHome>, &str, devlaunch_core::flows::lifecycle::Insistence, devlaunch_core::flows::lifecycle::Persistence, &mut dyn core::ops::function::FnMut(devlaunch_core::flows::lifecycle::DeleteStalled), &mut dyn devlaunch_core::notices::Notices<devlaunch_core::flows::lifecycle::LifecycleNotice>) -> core::result::Result<devlaunch_core::flows::lifecycle::DeleteOutcome, devlaunch_core::clients::devpod::NotRun>
pub fn devlaunch_core::api::workspace_delete(&mut devlaunch_core::flows::listing::CommandContext<'_>, &mut devlaunch_core::flows::lifecycle::Refresh<'_>, &devlaunch_core::flows::workspace_clone::WorkspaceCloneManager<'_>, &mut devlaunch_core::domain::metadata::MetadataStorage, core::option::Option<&devlaunch_core::clients::devpod_home::DevpodHome>, &devlaunch_core::flows::kept_copies::KeptCopies, &str, devlaunch_core::flows::lifecycle::Insistence, devlaunch_core::flows::lifecycle::Persistence, &mut dyn core::ops::function::FnMut(devlaunch_core::flows::lifecycle::DeleteStalled), &mut dyn devlaunch_core::notices::Notices<devlaunch_core::flows::lifecycle::LifecycleNotice>) -> core::result::Result<devlaunch_core::flows::lifecycle::DeleteOutcome, devlaunch_core::clients::devpod::NotRun>
pub fn devlaunch_core::api::workspace_stop(&mut devlaunch_core::flows::listing::CommandContext<'_>, &mut devlaunch_core::flows::lifecycle::Refresh<'_>, &str) -> core::result::Result<devlaunch_core::flows::lifecycle::StopOutcome, devlaunch_core::clients::devpod::NotRun>
Loading
Loading