Follow the recorded devpod workspace id, and reconcile the orphans

[blooop]

Closes #88.

The id dl handed devpod was derived from (owner, repo, ref) on every command and written down nowhere, so the derivation was the only copy of it in existence. When #81 moved that derivation, every workspace created under the old one stopped being addressable in the same instant — 36 of 39 on the reporting host. Nothing was corrupted and nothing was deleted; dl began asking devpod about ids devpod had never been given, and devpod answered, correctly, that there were no such workspaces.

Two changes, doing different jobs. One makes the failure impossible to repeat; the other repairs what already happened. Neither repairs the other's half.

1. The record gets written, and gets followed

WorktreeInfo.devpod_workspace_id has been declared since the worktree backend was written and had zero writers. It is now written when a clone is prepared — on re-registration too, which is how a record from an older build acquires one with no migration step.

Reading it is placed deliberately. A launch still derives an id and asks devpod about it first; the record is consulted only once devpod has denied the derived id. That keeps #145's warm attach path clear of the metadata lock, the parse and the id-scheme migration's version check — three things #145 removed from the path a user waits on — while paying for the lookup exactly in the case this ticket is about. The two orders differ only when devpod knows the derived id and a different recorded one for one triple, which after this change takes a workspace dl did not create.

A stored id devpod also denies is not used: metadata.json is append-mostly, so a record naming a workspace deleted months ago is ordinary, and addressing it would substitute one absent workspace for another and lose the derived id a create needs.

stop, rm, restart, recreate, reset and the attach all read the one resolution (resolve_known_workspace), so they were fixed together rather than one at a time.

2. dl --reconcile

A new global command, and the three decisions the ticket asked to be made deliberately:

• A top-level command, not prune --repair and not automatic. --prune states as its contract that it never touches a devpod workspace; repair is nothing but a devpod-workspace write, and folding it in would retract that promise for every existing --prune user to save a flag. Automatic-on-startup writes another tool's records unasked, which nothing else here does.
• It writes devpod's own workspace.json directly. devpod v0.26.1 exposes no subcommand that changes an existing workspace's source — the surface is build, delete, list, logs, ssh, status, stop, up plus config subcommands, and the only one that sets a source is a create, which needs a container daemon and would destroy the record being repaired. Only source.localFolder is replaced; the file is rewritten from what devpod itself last wrote, so uid, provider options and timestamps survive, and it goes through a temp file and a rename so a failure leaves the record whole.
• The join is by path, never by id. The id is what moved, so it connects nothing; the source folder devpod kept still names owner and repo exactly, and its leaf still names the branch in one of the three spellings dl has used for a clone directory (current hashed leaf, bare branch, pre-Derive workspace ids at one parse boundary #81 flattened branch). This is the same reasoning --prune's _site_of already runs on, reused rather than duplicated.

It deletes nothing and guesses at nothing. A clone a live workspace already opens is never taken from it; a clone two dead records both match is claimed by neither; an orphan with nothing to adopt is named and left standing. Refusing costs a line in a report, guessing costs a workspace. The plan is printed and confirmed before anything is written, -y skips the question, and the plan names the dl <workspace> recreate a repair costs before asking rather than after acting — the container was built with the dead path bind-mounted, and no record change moves a mount.

Real output, from a scratch cache and a scratch devpod home:

$ dl --reconcile
devpod workspaces sourced under .../cache/devlaunch/repos at something that is not a clone:

Re-pointing 2:
  - devlaunch-main: .../blooop/devlaunch/main -> .../blooop/devlaunch/devlaunch-main-zovomobo
  - bencher-test1: .../blooop/bencher/test1 -> .../blooop/bencher/bencher-test1-pipagito

Each of these needs `dl <workspace> recreate` afterwards: the container
still has the old source bind-mounted, and no record change moves a mount.

Leaving 1, which dl will not guess at:
  - rockerc-main (.../blooop/rockerc/main): no clone of that repository answers to this name

Nothing here is deleted. `dl <workspace> rm` is how one goes, if it should.

How this lines up with #155

--prune's Disputed arm — "#88's record-disagreement shape, never deletable" — is untouched and stays untouched. --prune keeps refusing to guess which clone a misplaced workspace needs; --reconcile is the command that resolves the disagreement, after which those clones classify as Referenced in the ordinary way. The README's prune section now points at it.

Tests

Persistence (test/test_workspace_clone.py, 2 tests) — the record carries the id dl hands devpod, for a new clone and for a re-registered existing one.

Addressing (test/unit/test_stored_workspace_id.py, 8 tests) — the stored id wins over a derived id devpod denies; a record agreeing with the derivation changes nothing; a record with no stored id and a stored id devpod also denies both fall back to the derivation; and the warm path still builds no clone manager, which is #145's guard. The scheme change is simulated at the derivation — one syllable narrower, the change the format has genuinely undergone once — never written down as a literal id, and a guard test asserts the simulation really moves the id so the rest cannot pass vacuously.

Reconciliation (test/unit/test_reconcile_orphaned_workspaces.py, 11 tests) — adoption for a missing folder, for the config-only stub, and for a branch whose old directory name was flattened; the record learning the id; refusal to take a clone a live workspace opens, to break a tie between two orphans, or to touch a workspace outside the cache; the report-and-keep path with no devpod delete reached; the confirmation gate; and a second run finding nothing to do. devpod's on-disk shape is written as devpod writes it, so the repair is asserted against devpod's format rather than against the test's idea of it.

Mutation-checked, eight for eight. Each of these was applied and the named test went red, alone: dropping the id write (2 tests red), ignoring the record in resolution (2 red), letting a claimed clone be a candidate, resolving ambiguity by listing order, not writing the id back on adoption (2 red), forgetting the legacy flattened leaf, counting only missing folders and not the stub, leaving unadoptable orphans unnamed, and skipping the confirmation.

Boundaries declared rather than decided

• Docker named volumes (What does the nested daemon cost in disk and time? #101's datum on the ticket) are not in this PR. That is a leak in devpod's own delete path — docker rm without -v, DeleteVolume defined with zero callers — not a record disagreement, and nothing in this ticket's "Done when" covers it. It wants its own ticket.
• A refused rename still advances the schema header, stranding those records forever #180 (a refused rename advancing the schema header) is adjacent and untouched. This adds no schema bump and no rename; it writes one previously-unwritten field that every existing reader already tolerates. If A refused rename still advances the schema header, stranding those records forever #180 changes how a partial write advances the header, nothing here has to move.
• The version-skew note Build: persist the devpod workspace id instead of re-deriving it, and reconcile the orphans #88 asked for is in the README rather than in code: an older dl sharing a reconciled cache derives the old directory name, misses it, clones a second directory, registers a second workspace and rewrites that branch's record with an empty id — undoing the repair for that one workspace. It is not destructive, the next --reconcile sorts it out, and the README says to upgrade or give the old build its own XDG_CACHE_HOME.

Not verifiable here, and the ticket says so

That a repaired workspace then recreates end to end needs a Docker daemon. Every claim above is from records on disk and from devpod --help on v0.26.1; the final dl blooop/devlaunch recreate is the human's, on the host.

pixi run ci green — 1645 passed, pylint 10.00/10, exit 0.

🤖 Generated with Claude Code

Summary by Sourcery

Persist the devpod workspace id in workspace records and add a CLI command to reconcile devpod workspace records with devlaunch clones after an id/naming scheme change.

New Features:

• Introduce dl --reconcile to re-point orphaned devpod workspaces to the correct clone directories based on source paths and branch naming history.
• Add a context field to workspace metadata to track the devpod context owning each workspace.

Bug Fixes:

• Ensure devlaunch writes and follows the devpod workspace id it creates, allowing workspaces created under previous id derivations to remain addressable.
• Resolve workspace selection by consulting stored workspace ids when the derived id is unknown to devpod, while preserving the fast path when devpod already recognises the derived id.

Enhancements:

• Refine devpod home handling via a shared devpod_home() helper and update config path resolution accordingly.
• Centralise workspace id resolution into resolve_known_workspace so all workspace lifecycle commands use the same logic.
• Improve CLI help, README, and changelog to document reconciliation behaviour and the impact of id-scheme changes.

Tests:

• Add integration-style tests to ensure devpod workspace ids are persisted for new and re-registered clones.
• Add unit tests covering stored-id resolution behaviour across derivation changes and ensuring the warm path does not touch metadata.
• Add unit tests for dl --reconcile, including adoption, refusal cases, idempotence, and user confirmation behaviour.

[sourcery-ai]

Sorry @blooop, you have reached your weekly rate limit of 500000 diff characters.

Please try again later or upgrade to continue using Sourcery

[sourcery-ai]

Reviewer's Guide

This PR makes devpod workspace IDs persistent and introduces dl --reconcile to repair orphaned devpod workspace records caused by an ID derivation change, wiring the new resolution logic through all workspace lifecycle commands and documenting the behavior.

Sequence diagram for resolving devpod workspace id on launch

sequenceDiagram
    actor User
    participant dl
    participant devpod
    participant Metadata

    User->>dl: dl <workspace> attach
    dl->>devpod: get_workspace_state(derived_id)
    alt devpod knows derived_id
        devpod-->>dl: state
        dl->>dl: resolve_known_workspace => KnownWorkspace(derived_id, state)
        dl->>devpod: use derived_id for subsequent calls
    else devpod denies derived_id
        devpod-->>dl: None
        dl->>Metadata: recorded_devpod_workspace_id(owner, repo, branch)
        alt no recorded id or same as derived
            Metadata-->>dl: None or derived_id
            dl->>dl: KnownWorkspace(derived_id, None)
        else recorded id differs
            Metadata-->>dl: recorded_id
            dl->>devpod: get_workspace_state(recorded_id)
            alt devpod knows recorded_id
                devpod-->>dl: state
                dl->>dl: KnownWorkspace(recorded_id, state)
                dl->>devpod: use recorded_id for subsequent calls
            else devpod denies recorded_id
                devpod-->>dl: None
                dl->>dl: KnownWorkspace(derived_id, None)
            end
        end
    end

Loading

File-Level Changes

Change	Details	Files
Persist devpod workspace IDs in worktree records and use them when resolving which devpod workspace to address.	Write devpod_workspace_id when preparing a workspace clone, including re-registration paths. Add KnownWorkspace and recorded_devpod_workspace_id helpers to encapsulate workspace ID resolution. Introduce resolve_known_workspace to prefer the stored ID when devpod does not recognize the derived ID, falling back safely when no usable record exists. Update launch and lifecycle commands (attach, stop, rm, restart, recreate, reset) to use resolve_known_workspace instead of directly using the derived ID. Add tests simulating an ID-scheme change to verify behavior when derivation and stored IDs differ.	devlaunch/worktree/workspace_clone.py devlaunch/dl.py test/unit/test_stored_workspace_id.py test/test_workspace_clone.py
Add dl --reconcile to re-point orphaned devpod workspace records to the correct clones without deleting anything or guessing across ambiguities.	Introduce Workspace.context field and parsing to track devpod contexts. Add helpers to detect orphaned workspaces under the cache and to compute legacy directory leaf spellings for branches. Implement reconcile_plan to match devpod workspaces sourced at non-checkout paths to candidate clones by owner/repo/path leaf, refusing ambiguous matches and clones already opened by live workspaces. Implement print_reconcile_plan and reconcile_command to show the plan, gate on confirmation (-y/--yes), and apply changes via apply_reconciliation. Implement _repoint_devpod_source to rewrite devpod workspace.json source.localFolder safely via temp file and rename, and update metadata records with devpod_workspace_id while invalidating caches. Add extensive unit tests that build real on-disk devpod and devlaunch layouts and assert adoption, non-adoption, idempotence, and non-deletion behavior.	devlaunch/dl.py README.md CHANGELOG.md test/unit/test_reconcile_orphaned_workspaces.py
Refactor devpod configuration path handling to expose devpod_home and adjust callers.	Replace _devpod_config_path root computation with a new devpod_home helper that respects DEVPOD_HOME and returns the devpod home directory. Update _devpod_config_path to be a thin wrapper returning devpod_home()/"config.yaml".	devlaunch/dl.py

Assessment against linked issues

Issue	Objective	Addressed	Explanation
#88	Persist the devpod workspace id when a workspace is created (and on re-registration), and use the stored id instead of re-deriving it when addressing devpod workspaces across commands (up/attach, stop, rm, restart, recreate, reset). Include tests to guard regressions, including across id-scheme changes.	✅
#88	Provide a reconciliation mechanism that repairs already orphaned devpod workspaces whose recorded source folders are missing or lack a .git checkout, by matching them to healthy clones by path (not id), updating devpod’s workspace.json and devlaunch’s records, while refusing ambiguous matches and never deleting workspaces. Make it idempotent and test-covered.	✅
#88	Document the behavior and version skew considerations around the id-scheme change and reconciliation (e.g., how older dl versions interact with reconciled metadata), and integrate the new reconciliation functionality into the CLI help/README/CHANGELOG. Ensure the test suite (pixi run ci) passes.	✅

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[codecov]

Codecov Report

❌ Patch coverage is 97.91667% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 92.76%. Comparing base (e3d4639) to head (ce08cce).

Files with missing lines	Patch %	Lines
devlaunch/dl.py	97.91%	4 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #210      +/-   ##
==========================================
+ Coverage   92.38%   92.76%   +0.37%     
==========================================
  Files          24       24              
  Lines        3467     3649     +182     
==========================================
+ Hits         3203     3385     +182     
  Misses        264      264              

Files with missing lines	Coverage Δ
devlaunch/worktree/workspace_clone.py	93.62% <ø> (ø)
devlaunch/dl.py	95.04% <97.91%> (+0.57%)	⬆️

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[blooop]

This was generated by AI during review.

Two-axis review of #210 against ticket #88 at merge-base e3d4639. Preflight: all checks green (ci ×5, e2e, prek, gate, codecov patch+project, GitGuardian), non-empty diff (7 files, +1384/−8), MERGEABLE.

Standards

• BLOCKING — reconcile_plan's candidate index resolves collisions silently, contradicting its own docstring. candidates[(owner, repo, spelling)] = resolved is last-write-wins across list_worktrees(), and the legacy spelling is known non-injective: commit 5db5c42 records five verified preimages of feature-auth under the very regex _legacy_leaf resurrects. So two records can claim one key and the winner is iteration order — a coin flip that re-points a workspace at the wrong branch's clone. The docstring says "Two candidates are refused rather than resolved, in both directions"; only the orphan side (wanted[clone] > 1) is actually refused. Either detect key collisions and route them to Unadoptable, or stop claiming both directions. devlaunch/dl.py:2506-2530.
• BLOCKING — claimed = set(locations.by_path) is a weaker guard than the sibling command's, and bypasses the accessor built for it. by_path is keyed by the source path, so a live workspace opening <clone>/subproject leaves <clone> unclaimed and adoptable — the exact mistake WorkspaceLocations.holder() exists to prevent ("Equality answered no and deleted the parent", dl.py:1847-1867). Relatedly, reconcile_command computes locations but never checks locations.unlocatable, where --prune refuses to run for the same reason (an unfollowable live workspace could be holding any candidate). Both hazards land on the invariant the PR advertises: "a clone a live workspace already opens is never taken from it."
• Non-blocking — duplication with #155's classification: _orphaned_workspaces re-walks the same source_places/_canonical/_site_of chain workspace_locations just walked, because misplaced is keyed (owner, repo) and collapses. Widening Misplaced to a tuple would remove the second traversal and the near-duplicate Unadoptable (same two fields). It also discards InARepositoryOnly(owner, repo) and recomputes it via source.relative_to(root).parts[:2].
• Non-blocking — Reconciliation = Adoptable | Unadoptable (dl.py:2500) is defined, undocumented, and referenced nowhere: speculative generality in a file where every other alias is consumed.
• Non-blocking — KnownWorkspace(id, state: Optional[str]) is the one new type here that isn't a sum type, and workspace_id changes meaning (resolved vs. derived) with state is None. Every neighbour (SourcePlaces, SourceSite, Decision) models this as arms.
• Non-blocking — unlike prune_clones, apply_reconciliation re-checks nothing between consent and write; the deliberate second pass is the house pattern, so the omission deserves at least a sentence.
• Nit — tests otherwise match test_prune_orphaned_clones.py closely (World, FakeDevpod, needs_an_unprivileged_user, fidelity rationale). Divergences: three function-local imports with pylint disables, and import subprocess twice inside one method, where the sibling imports at module top. README/CHANGELOG/help conform; anchor resolves.

Spec

Ran the new/modified suites → 89 passed; also test_devpod_spawn_counts.py + test_prune_orphaned_clones.py → 85 passed, neither in the diff.

Nothing blocks on this axis. All four "Done when" bullets are met.

• Deviation 1 (derived-first) — sound, non-blocking. The ticket's §1 says "use the stored id, falling back to derivation only when there is no record", i.e. record-first; resolve_known_workspace (dl.py:3691) is derived-first. But "Done when" only asks that paths "use the stored id in preference to a derived one", and the orders diverge only when devpod knows both ids for one triple — unreachable for a workspace dl created after this change. The warm-path claim holds: the record lookup is behind state is not None, and test_a_workspace_devpod_knows_builds_no_clone_manager asserts _CLONE_MANAGER_KEY not in dl._cache. One resolution feeds workspace_id for stop, rm, up, restart, recreate, reset and attach_workspace (→ ssh), covering the ticket's full list.
• Deviation 2 (Docker volumes) — sound. The volume datum arrived as a comment; no "Done when" bullet covers it.
• Deviation 3 (#180 / no schema bump) — sound, verified. SCHEMA_VERSION = 2 unchanged, storage.py untouched; to_dict uses asdict so the field serializes, and from_dict _drop_unknown means older readers tolerate it. Written on both new clone and re-registration (workspace_clone.py:730), pinned by two tests.
• Deviation 4 (no end-to-end recreate) — sound. The ticket itself: "That last step is the human's, on the host."
• Reconcile safety verified in code, not prose. No devpod delete and no deletion anywhere in reconcile_plan/apply_reconciliation; ties report both sides; the rebuild warning prints before the confirmation. (The claimed-clone guard's strength is the Standards axis's finding.)
• Non-blocking: the ticket-thread breadcrumb promised "orphan + no clone → named, exit code says so"; reconcile_command returns 0 when there is nothing to adopt. The ticket body only requires "say so" — breadcrumb/implementation mismatch, not a spec miss.
• Non-blocking: an orphan sourced deeper than a clone leaf joins on source.name and so can never match — it falls to the report, which is the safe arm. README version-skew section and CHANGELOG (### Added + ### Fixed, both under [Unreleased]) match what is built.

Verdict

Request changes — two blocking findings, both on the reconcile safety invariants (Standards):

1. Candidate-index collisions resolve last-write-wins; route key collisions to Unadoptable so the docstring's "both directions" is true (dl.py:2506-2530).
2. Use WorkspaceLocations.holder() (not set(locations.by_path)) for the claimed guard, and refuse to run when locations.unlocatable is non-empty, matching --prune's reasoning (dl.py:1847-1867).

Spec axis is fully clean. Non-blocking findings recorded above. GitHub refuses --request-changes from the PR author's account, so this is posted as a comment; this written verdict governs.

[blooop]

This was generated by AI during review.

Fix re-review (fresh context, commit ce08cce)

Both blocking findings RESOLVED, verified by independent mutation runs: reverting the index to last-write-wins reddens exactly the collision test (and removing the same-record dedupe reddens 8); reverting holder() to equality reddens the subdirectory test; deleting the unlocatable guard reddens 3. The exit-0→1 inversion of the unfollowable-source tests is judged correct — an unfollowable live source could hold any candidate, symmetric with prune's reason for stopping. report_unlocatable's refactor reproduces prune's literals byte-for-byte at both call sites; scope is clean (dl.py + reconcile tests + README only). Non-blocking observations for later: prune's report wording is unpinned at the refactored seam; records[resolved] remains last-write-wins (pre-existing, only reachable with two records sharing one local_path).

Verdict: clean — merging with a merge commit.