Repository navigation
Description
Where did this happen?
Desktop app (macOS)
Version
1.24.0-canary.20260821030527 (git 2.50.1, Apple Git-155)
What happened?
Deleting a workspace ("Separate session and workspace loaders", branch separate-session-workspac) failed with:
Failed to remove worktree at /Users/avipeltz/.superset/worktrees/<project-id>/thin-zone
Root-caused this to three stacked bugs in the workspace-delete saga, confirmed by reading the source and reproducing the underlying git behavior directly:
-
git worktree remove --force --forcecan partially fail while still fully unregistering the worktree. Reproduced directly (git 2.50.1): make a nested file inside a worktree undeletable (chflags uchg), then rungit worktree remove --force --force <path>. Git printserror: failed to delete '<path>': Operation not permittedand exits 255 — but immediately afterward,git worktree list --porcelainno longer lists it and the repo's.git/worktrees/<name>admin entry is gone entirely, even though the working directory (and the worktree's own.gitfile) are still sitting on disk. Unregistration and file deletion are not atomic in git's own implementation. -
The app doesn't account for that. In
packages/host-service/src/workers/tasks/git.ts(gitWorktreeRemoveTask), thegit worktree remove --force --forcecall's error is discarded (.catch(() => {})), and the task's return value (stillRegistered) is derived solely from re-runninggit worktree list --porcelainafterward:await git .raw(["worktree", "remove", "--force", "--force", target]) .catch(() => {}); const raw = await git.raw(["worktree", "list", "--porcelain"]); return { stillRegistered: parseWorktreeList(raw).some( (w) => normalizeWorktreePath(w.path) === target, ), };
In
packages/host-service/src/trpc/router/workspace-cleanup/workspace-cleanup.ts, whenstillRegisteredcomes backfalse, the code setsworktreeRemoved = trueand moves on — it never re-checksexistsSync(local.worktreePath)before trusting that. (Two other branches in the very same function — the "session" branch and the "project repo is missing" branch — already do exactly thatexistsSync+rm(path, { recursive: true, force: true })fallback; it just isn't wired into the common "project + repo present" path that actually ran here.)When
stillRegistereddoes come backtrue(as it apparently did on the attempt that produced this toast), the code throws:throw new TRPCError({ code: "INTERNAL_SERVER_ERROR", message: `Failed to remove worktree at ${local.worktreePath}`, });
— with no detail, because the real git error from step 1 was already thrown away.
-
Nothing logs the underlying git failure anywhere. Checked
host-service.log, the terminal-hostdaemon.log, and the Electronmain.log— none contain any trace of the actualgit worktree removeerror. The toast text is the entire diagnostic record. The softer warnings this same saga produces (e.g."N terminal(s) may still be running"from a failed PTY kill) are also only returned as strings in awarningsarray to the renderer — never logged server-side — so even those vanish once the toast is dismissed.
Net effect: the specific workspace above is now stuck — git has zero record of it, but its directory (a partial sparse-checkout: apps/api, apps/discord-triage, apps/marketing, apps/mobile, apps/relay, apps/relay2, plus root files) is still on disk, and its workspaces DB row is still un-archived. Clicking Delete again as-is will very likely report success (git already shows nothing registered) and archive the row, permanently leaving the orphaned folder behind — nothing targets an already-unregistered path for cleanup.
Most likely trigger (circumstantial, not provable after the fact because of bug #3): runDestroyPhases step 3a (workspace-cleanup.ts) disposes PTY sessions tied to the workspace before step 3b removes the worktree, but a failed kill only pushes a warning — it doesn't block or retry step 3b. disposeSessionsByWorkspaceId (terminal.ts) only touches sessions tracked in the terminalSessions table, so a detached grandchild process (e.g. a dev server or install step whose parent PTY died but which wasn't fully reaped) would survive invisibly. host-service.log shows repeated [workspace-fs/watch] FSEvents overflow bursts for this exact path in the surrounding window — consistent with something still actively writing into the directory while the recursive delete ran.
Steps to reproduce
Reliable repro of the underlying git-level bug (not the full app flow, which requires timing a race):
git init -q main-repo && cd main-repo
git config user.email a@b.com && git config user.name test
echo hi > file.txt && git add . && git commit -q -m init
git worktree add -q --no-checkout ../wt1 -b wtbranch1
cd ../wt1 && git checkout wtbranch1
mkdir -p sub && echo x > sub/lockedfile
chflags uchg sub/lockedfile # simulate a file the recursive delete can't remove
cd ../main-repo
git worktree remove --force --force ../wt1
# -> error: failed to delete '.../wt1': Operation not permitted (exit 255)
git worktree list --porcelain # wt1 is already gone from the list
ls .git/worktrees/ # admin dir for wt1 is gone entirely
ls ../wt1 # directory (and its .git file) still on diskIn the app, this manifests whenever git worktree remove --force --force hits any obstruction partway through its recursive delete (locked/open file, permission issue, a live process still writing into the tree) during workspace delete.
Logs, screenshots, or recordings
Full write-up with timeline, exact file/line references for every claim above, the missing-existsSync-fallback comparison, a concrete fix checklist, and immediate remediation steps for the specific stuck workspace: see docs/issues/workspace-delete-orphaned-worktree.md on the debug-workspace-delete branch.
Suggested fixes (detailed in that doc):
- Log the real git stderr in
git.ts'sgitWorktreeRemoveTaskinstead of.catch(() => {}). - After
removeWorktreereportsstillRegistered: false, still checkexistsSync(local.worktreePath)before trustingworktreeRemoved = true; fall back to the samerm(..., { recursive: true, force: true })pattern the other two branches inworkspace-cleanup.tsalready use. - Make the terminal-disposal step actually gate (or retry) the worktree-removal step instead of degrading a failed kill to a no-op warning.
- Add a regression test for "git reports not-registered after remove, but the directory still exists on disk" — the existing suite covers "still registered → throws" and "git open failure blocks delete," but not this case.
Related issues
- [bug] Failed to delete worktree #4941 — same symptom class on an older version (1.11.3):
fatal: '<path>' is not a working tree, partial deletion left a.vite/folder behind. Closed without (as far as I can tell from this investigation) the swallowed-error / missing-existsSync-fallback root cause actually being fixed — this issue looks like a fresh recurrence of the same underlying defect on the current host-service implementation. - Workspace deletion leaves orphaned worktree directories on disk #2863 — same class of report against the old desktop-side implementation (rename-to-
.superset-delete-*→git worktree prune→ detachedrm -rf), since replaced by the current host-serviceworkspace-cleanupsaga this issue is about. Worth noting the failure mode survived the rewrite in a different shape. - [feat] Reconcile partially-deleted workspaces: ghost sidebar rows, orphan worktree dirs, stale host.db registrations #5499 — the general "reconcile partially-deleted workspaces" feature request. This issue is a concrete, reproduced root cause behind that pattern (specifically the "worktree dir on disk, no registration" flavor it documents) — fixing the two items above would close one real source of the drift [feat] Reconcile partially-deleted workspaces: ghost sidebar rows, orphan worktree dirs, stale host.db registrations #5499 is asking to reconcile after the fact.
- Feature request: auto-prune orphaned worktrees / storage cleanup command #5631 — adjacent ask for an auto-prune/cleanup command; a good complement once orphans are reliably created less often, but doesn't address why they're created in the first place.
- workspace delete:
forceconflates "discard dirty changes" with "skip project teardown script" #6070, [bug] Deleting a workspace leaves its git branch behind, so reusing the workspace name creates a mismatched branch #6380 — different bugs in the sameworkspace-cleanup.tsdestroy saga (force-flag semantics, branch-delete ordering). No known interaction with this bug, but worth the same author's attention since a fix here touches adjacent code.
Activity
github-actions commentedon Aug 21, 2026
Summary
The report is that a workspace delete surfaced
Failed to remove worktree at …, and that the underlying cause is thatgit worktree remove --force --forceis not atomic: git can unregister the worktree from the repo's admin area and then fail partway through the recursive directory delete (exit 255,error: failed to delete '<path>': Operation not permitted), leaving the directory on disk whilegit worktree list --porcelainno longer mentions it. The destroy saga treats that porcelain listing as the sole authority for "removed," never re-checks the filesystem afterwards, and throws away git's actual stderr — so the orphaned directory is invisible to the app and the failure leaves no diagnostic anywhere. I read the relevant code and the claims check out against what's onmain; below are the confirmations plus two consequences the report doesn't cover.Affected code
packages/host-service/src/workers/tasks/git.ts:198-225—gitWorktreeRemoveTaskpackages/host-service/src/trpc/router/workspace-cleanup/workspace-cleanup.ts:459-541— the "project + repo present" branch of step 3bpackages/host-service/src/trpc/router/workspace-cleanup/git-ops.ts:60-71—cleanupGitOps.removeWorktreepackages/host-service/src/terminal/terminal.ts:2247-2287—disposeSessionsByWorkspaceIdpackages/host-service/src/runtime/archived-workspace-reconcile.ts:16-70— the startup reconcilerpackages/host-service/test/workspace-cleanup.test.ts:43-74, 380-417— the mockedremoveWorktreesurface and its coverageRoot cause
1. The error is discarded and the registry read is treated as authoritative.
git.ts:213-215does.raw(["worktree","remove","--force","--force",target]).catch(() => {}), then derivesstillRegisteredpurely fromworktree list --porcelainatgit.ts:218-223. The comment atgit.ts:208-212states the intent explicitly — "the registry read below is authoritative, not the command's locale- and version-dependent exit text" — and that is precisely the invariant the reported git behavior violates. Unregistration and recursive deletion are separate steps in git's implementation, sostillRegistered: falsemeans "git no longer tracks this path," not "the directory is gone."2. The consumer never re-checks the disk. In
workspace-cleanup.ts,worktreeRemovedis computed at line 460 before the removal (!existsSync(local.worktreePath)), and then at line 539 unconditionally overwritten withworktreeRemoved = true;wheneverstillRegistered === false. There is noexistsSyncbetween the removal and that assignment. The asymmetry the report flags is real: the session branch (:432-450) and the missing-project-repo branch (:460-494) both doexistsSync→rm(path, { recursive: true, force: true })under a root guard, and the integration test comment atworkspace-cleanup.integration.test.ts:352even calls out "--force --forceplus the existsSync fallback" — but that fallback is not wired into the common path that actually runs here.3. Nothing is logged server-side. The throw at
:534-537carries only the path; git's stderr was dropped atgit.ts:215. Thewarningsarray (:412,:416,:455,:481,:507,:559,:569) is returned to the renderer and only rendered as toasts (useDestroyDialogState.ts:152,useBulkWorkspaceDelete.ts:194) — never written to a log. So both the hard failure and the soft warnings are gone once the toast is dismissed.Two consequences worth adding:
existsSyncis true → repo is present →removeWorktreeruns, git fails withis not a working tree(swallowed atgit.ts:215) → the listing shows nothing →stillRegistered: false→ line 539 setsworktreeRemoved = true→ success, row stays archived, directory stays on disk.selectStranded(archived-workspace-reconcile.ts:78-86) picks up exactly this shape — archived row, no live owner, directory still exists — and callsdestroyWorkspacewithforce: true, which walks the same faulty path and reports success without deleting anything. That repeats every host-service start, loggingresumed 1 interrupted delete(s)forever while the orphan persists. The safety net designed for this state is defeated by the same bug.Note on the suspected trigger (
disposeSessionsByWorkspaceIdletting a detached grandchild survive): plausible from the code — it only iteratesterminalSessionsrows (terminal.ts:2251-2260) and a failed close only incrementsfailed, which becomes a warning atworkspace-cleanup.ts:411-413and does not gate step 3b. But it isn't the only way to obstruct the delete (an open file handle, a Finder/editor lock, an immutable flag would all do it), and bug #3 means there's no evidence either way for this particular incident. The disk-recheck defect stands independent of the trigger.Reproduction
The git-level behavior in the report is a faithful, self-contained repro of the precondition. For the app-level defect, the cleanest reproduction is a unit test — and note that today's mocked default (
workspace-cleanup.test.ts:68-71,{ stillRegistered: false }) already runs against atmpdirectory that still exists on disk, so the existing happy-path tests are unknowingly asserting the buggy behavior and will need their mocks updated to actually remove the directory:An end-to-end version is possible in the integration suite on macOS by
chflags uchg-ing a nested file beforedestroy(with anafterhook tochflags nouchg, or the temp dir leaks), though that's platform-specific.Suggested fix direction
git.ts:213-215). Capture stderr and return it alongside the verdict — e.g.{ stillRegistered, directoryExists, removeError }— and checkexistsSync(target)inside the task, right after git exits, rather than back on the event loop; that's both closer to the event and off the loop. Update thegit.ts:208-212comment, since the invariant it asserts ("the registry read is authoritative") is what's wrong here.workspace-cleanup.ts:539). Replace the bareworktreeRemoved = truewith the sameexistsSync→rm(..., { recursive: true, force: true })fallback the other two branches use, guarded byisInsideProjectWorktreesRootso a corruptworktreePathcan never aim a recursive delete at user data. Include the capturedremoveErrorin the:534-537throw message so the toast carries the real reason.killed.failed > 0before attempting the removal, since a live writer is the most likely obstruction.For the specific stuck workspace, once (2) lands, re-running Delete will take the new fallback and clear both the directory and the row. Before then it can be unstuck manually by removing the leftover worktree directory (and clearing any
chflags uchg/ permission obstruction on files inside it) and then deleting the workspace from the app.Cross-references in the report to #4941, #2863, #5499, #5631, #6070 and #6380 are plausible from the code shape but I did not verify those issues' contents.
fix(host-service): workspace delete re-checks the disk after git unre…
fix(host-service): delete worktree trees in the app, let git only unr…
fix(host-service): delete worktree trees in the app, let git only unr…