Skip to content

fix: finish a confirmed deletion when the lease release is refused as unauthenticated - #987

Merged
alongubkin merged 1 commit into
mainfrom
alon/alien-1395-release-after-deletion
Oct 11, 2026
Merged

alongubkin merged 1 commit into
mainfrom
alon/alien-1395-release-after-deletion

Conversation

@alongubkin

Copy link
Copy Markdown
Member

Problem

alien-deploy destroy can exit 1 with Failed to release sync lock > UNAUTHORIZED after a deletion has finished. By then every resource and the deployment record are gone.

Root cause

When the manager confirms Deleted, it removes the deployment record. The lease and the deployment credential the CLI authenticates with go with it. The CLI then releases its lease with that credential. A manager that validates bearer tokens upstream, with a short cache, can only answer that release from its cache. Once the cache entry has expired, the token no longer exists, so the release gets 401 instead of the 404 that release_deployment tolerates. The failure is timing dependent. It's common when teardown reconciles are slow (for example a registry-grant revoke on every reconcile, ~30 s each), because requests to one manager instance end up more than the cache TTL apart.

Fix

After the manager has confirmed the deletion, treat a release refused as 401 or 404 as completed:

  • finalize_step_loop checks for a persisted terminal Deleted checkpoint.
  • final_reconcile checks that the Deleted reconcile was accepted (or the record was already missing).

In both cases nothing remains to release. Every other release still fails on 401, including a release after a refused Deleted checkpoint.

Alternatives considered:

  • Tolerating 401 in release_deployment for every caller would hide real credential failures.
  • Skipping the release after Deleted relies on every manager dropping the lease along with the record.
  • Keeping the deleted credential valid for the release is a server-side policy change with a wider blast radius.

Proof

  • New a_confirmed_deletion_completes_when_the_release_is_refused_as_unauthenticated: on main it fails with the exact error seen in the field (Failed to release sync lock > UNAUTHORIZED: Platform API authentication failed with status 401). It passes with the fix, through both finalize_step_loop and final_reconcile.
  • New an_unauthenticated_release_still_fails_without_a_confirmed_deletion: a 401 release on a running deployment, and a 401 release after a refused deleted checkpoint, still fail.
  • cargo test -p alien-deployment --lib: 179/179 pass.
  • Live: a Lambda worker teardown completed (registry grant revoked, record removed, cloud resources gone), but destroy exited 1 at the final release. A direct release with the removed deployment credential returns 401 every time.

Rollout

CLI-side only, and it ships with the next CLI and setup binary. It doesn't change server behaviour.

ALIEN-1395

🤖 Generated with Claude Code

… unauthenticated

Once the manager confirms a deployment's deletion it removes the record,
its lease and the deployment credential. The lease release that follows
can then be refused as unauthenticated (401) as well as missing (404).
Treat both as a completed deletion instead of failing destroy after every
resource is gone. Other releases keep failing on 401.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@alongubkin

Copy link
Copy Markdown
Member Author

@greptileai review

@greptile-apps

greptile-apps Bot commented Oct 10, 2026 •

Copy link
Copy Markdown

RetriggerConfidence Score: 5/5

[Critical impact] The PR appears safe to merge.

Summary

The CLI now tolerates a lease release returning 401 or 404 after the manager confirms deletion.

  • alien-deploy destroy finishes after confirmed deletion even when lease release gets 401 or 404.

Reviews (1) · Last reviewed commit: "fix: finish a confirmed deletion when th..." · Reviewed by Greptile

@alongubkin
alongubkin merged commit 0171fc9 into main Oct 11, 2026
24 checks passed
@alongubkin
alongubkin deleted the alon/alien-1395-release-after-deletion branch October 11, 2026 05:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant