Skip to content

fix: release follow-ups for domain preflight, secret rotation, re-setup, logs and deletes - #895

Merged
alongubkin merged 16 commits into
mainfrom
alon/alien-1239-alien-release-follow-ups-custom-domain-preflight-secret
Oct 6, 2026
Merged

alongubkin merged 16 commits into
mainfrom
alon/alien-1239-alien-release-follow-ups-custom-domain-preflight-secret

Conversation

@alongubkin

@alongubkin alongubkin commented Oct 5, 2026 •

Copy link
Copy Markdown
Member

Follow-ups from the v3.3.27 release validation. One commit per fix.

Custom-domain-only stacks fail the domain metadata preflight

Cause: DomainMetadataRequiredCheck counted every public HTTP container and only checked whether domainMetadata was set. Containers listed in stackSettings.domains.customDomains bring their own domain and certificate, and the control plane sends no metadata when every public resource has one, so such stacks were rejected.
Change: resources_requiring_domain_metadata takes the config and skips custom-domain resources, in both should_run and check.
Test: custom-domain-only public container passes (fails on main); a mixed stack without metadata still fails and names only the managed container.

Deployer secret rotation is not picked up by a redeploy

Cause: vault-native deployer secrets were named, never versioned, in workload config. Overwriting one in the secret store and redeploying changed nothing the executor diffs, so workloads kept the old value until an unrelated change restarted them.
Change: Vault::secret_presence returns Present { version } from metadata only: SSM parameter Version, Secret Manager version name, newest Key Vault version id, Kubernetes resourceVersion, and a per-secret write counter for the local vault, counted before the value is saved so a failed save can only cause an extra restart (a counter instead of the file mtime: mtimes can collide between quick writes and are not per key). DeployerSecretReport and DeployerSecretEnv carry version (serialized only when present), so the next update after an overwrite changes the worker/container config and rolls it; an update without a write changes nothing. On Kubernetes the pod spec only references the Secret, so a deployer-secret-checksum pod-template annotation carries the versions. Workloads with filled slots roll once on the first update after this ships.
Test: test_redeploy_after_deployer_secret_overwrite_restarts_workloads (alien-deployment test_platform): write V1, run to Running, overwrite V2, same-release update → env version changes and the worker goes through an update; second update without a write → identical env, no worker update. Kubernetes manifest test: same version renders the same template, a new version changes only the annotation. Local vault test: reads keep the version, overwrite changes it.
Regenerated packages/core with pnpm --filter @alienplatform/core generate.

alien deploy re-run waits two minutes for a lock nobody grants

Cause: for running / update-failed / refresh-failed deployments alien deploy asked for the runtime lock with the deployment token. The runtime lock never matches a running deployment or a blocked update, and the setup lock refuses deployment tokens, so it timed out.
Change: alien deploy reads the active update from the platform (with the deployment token) and decides:

  • running with nothing pending → prints "Nothing to do" and exits;
  • an update waiting for setup while running or provisioning-failed, or update-failed / refresh-failed → gets setup authority from a deployment group token passed with --token (now kept instead of dropped for tracked deployments, checked against the deployment's group) or a first-party group session minted from the user's login, retries the failed operation when needed, takes the setup lock with it, prepares the setup update and runs setup; the installed release is loaded so step reconciliation keeps it; the deployment token keeps configuring the runtime (same split as alien-deploy up --setup-update);
  • an update waiting for setup in another status (e.g. update-pending, which the setup lock does not accept) → fails immediately and points to alien deployments get;
  • neither available → fails immediately with DEPLOYMENT_SETUP_AUTHORITY_REQUIRED, naming alien login and --token.
    Test: decision table for every status/active-update combination in platform and standalone mode; token selection against a mock platform API (supplied group token wins and no login happens; login mints a session for the deployment's group with the user's bearer; no login and no token gives the error with both hints). Not run against staging.

Push-created Lambda log groups never expire

Cause: the push-model AWS Worker controller left /aws/lambda/<name> to the Lambda runtime, which creates it on first invoke with no retention.
Change: new CloudWatch Logs client (CreateLogGroup single attempt, PutRetentionPolicy); the controller creates the group with 30-day retention before CreateFunction, sets retention when it already exists, and keeps it on delete (matches the CloudFormation log group with Retain). The management role's permissions only change when setup runs again, so an install from before this change lacks the new grants: access denied on either call logs a warning and the worker is still created (Lambda then creates the group without retention). Other errors stay fatal. The update flow runs the same idempotent step, so the first update after setup grants the permissions sets the retention. worker/provision grants logs:CreateLogGroup and logs:PutRetentionPolicy on /aws/lambda/${stackPrefix}-* (and the resource-scoped name).
Test: controller harness: log group + retention happen before the function; an existing group still gets retention; access denied on create or on retention still creates the function; other errors (e.g. throttling) fail it. Client tests against a mock endpoint (request shape, ResourceAlreadyExistsException → conflict). To verify live: deploy a push-model AWS worker, then aws logs describe-log-groups --log-group-name-prefix /aws/lambda/<prefix>-<worker> shows retentionInDays: 30 before the first invocation.

Deleting a deployment silently keeps deployer secrets

Cause: delete removes only Alien-synced vault keys; deployer secrets (values the deployer wrote) stay by design, but nothing said so.
Change: DeployerSecretLocation gains deleteCommand per store; the delete step logs kept secrets; alien destroy and alien deployments delete list every deployer secret slot with its delete command (a slot last reported missing may have been written since). There is no inputs/secrets page in this repo to update.
Test: exact delete command per store, next to the write command test; destroy output lists present/invalid secrets with their commands and includes one last reported missing.

alien deployments redeploy --wait waits silently

Change: prints the operation's status, reasons and actionRequired whenever any of them changes; the timeout error names the last of them.
Test: progress lines for queued → queued with action → applying → succeeded (each change once); timeout error includes the action.

🤖 Generated with Claude Code

alongubkin and others added 6 commits October 5, 2026 18:31
Containers listed in stackSettings.domains.customDomains bring their own
domain and certificate, so a stack whose public containers all use custom
domains no longer fails the domain metadata prerequisite.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Vault-native deployer secrets were named, never versioned, in workload
config, so overwriting one in the secret store and redeploying changed
nothing the executor diffs. The vault's secret_presence now returns the
store's version of the value (SSM parameter version, Secret Manager
version name, Key Vault version id, Kubernetes resourceVersion, a
per-secret write counter for the local vault). The deployer secret
report and the workload's deployer secret entry carry it, so the next
update after an overwrite changes the workload config and rolls it, and
an update without a new write changes nothing. Kubernetes pods
reference the Secret by name, so the version also goes into a
pod-template annotation.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Re-running `alien deploy` on an installed deployment asked for the
runtime lock with the deployment token. Nobody grants that lock to a
running deployment or to one whose update waits for setup, so the
command waited two minutes and timed out. It now reads the deployment's
active update from the platform and:

- exits with "nothing to do" when it runs with nothing pending;
- when an update waits for setup, or the last update/refresh failed,
  takes setup authority from a deployment group token passed with
  `--token` or mints a first-party group session from the user's login,
  retries the failed operation if needed, and runs setup with the setup
  lock while the deployment token keeps configuring the runtime (the
  same split `alien-deploy up --setup-update` uses);
- fails right away, naming `alien login` and `--token`, when it has
  neither.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The push-model AWS worker controller left `/aws/lambda/<name>` to the
Lambda runtime, which creates it on first invoke with no retention, and
nothing deleted it on teardown. The controller now creates the log
group with 30-day retention before creating the function (or sets the
retention on an existing group) and keeps it on delete, matching the
CloudFormation template's log group. Adds a CloudWatch Logs client and
grants logs:CreateLogGroup/PutRetentionPolicy on the stack's Lambda log
groups in worker/provision.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Deleting a deployment removes the secrets Alien synced but keeps the
deployer secrets, whose values the deployer wrote in their own secret
store. That was silent, so the secrets looked like leftovers. Each
deployer secret location now carries the command that deletes it, the
deletion step logs the kept secrets, and `alien destroy` and
`alien deployments delete` list them with that command.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The wait printed only status changes, so an operation waiting on
something stayed silent. It now prints the operation's status, reasons
and required action whenever any of them changes, and the timeout error
names the last of them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@greptile-apps

greptile-apps Bot commented Oct 5, 2026 •

Copy link
Copy Markdown

RetriggerConfidence Score: 5/5

[High risk] Modifies secret storage and retrieval across multiple vault providers.

The PR appears safe to merge; no outstanding blocking finding or new actionable issue was established.

Summary

The PR addresses release-validation follow-ups for domain preflight, deployer-secret rotation, re-setup, Lambda log retention, deletion notices, and redeploy progress. The changes since the previous review add an installed-release fallback for refresh retries and metadata-only Azure Key Vault version selection with pagination.

Reviews (11) · Last reviewed commit: "fix: retry failed refreshes and order sa..."

Comment thread crates/alien-cli/src/commands/deploy.rs Outdated
Comment thread crates/alien-cli/src/commands/deploy.rs
Comment thread crates/alien-bindings/src/providers/vault/local.rs Outdated
Comment thread crates/alien-cli/src/commands/destroy.rs Outdated
- alien deploy runs a setup update only in statuses the setup lock
  accepts (running, provisioning/update/refresh failed); an update
  waiting for setup on an update-pending deployment fails fast instead
  of waiting for a lock.
- A setup update loads the installed release, so reconciling setup
  state no longer clears it.
- The local vault counts a write before saving the value, so a failed
  save can only cause one extra restart, never a missed one.
- Deletion lists every deployer secret slot with its delete command: a
  slot last reported missing may have been written since.
- Update the CloudFormation snapshot for the Lambda log group grant.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@alongubkin

Copy link
Copy Markdown
Member Author

@greptileai review

Comment thread crates/alien-bindings/src/providers/vault/local.rs
Counting the write before saving the value meant the first write to a
new local vault tried to write versions.json into a directory that did
not exist yet. The local vault tests now start from a missing directory.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@alongubkin

Copy link
Copy Markdown
Member Author

@greptileai review

Comment thread crates/alien-bindings/src/providers/vault/local.rs
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@alongubkin

Copy link
Copy Markdown
Member Author

@greptileai review

The management role's permissions are fixed at setup. Setup creates the
management role and its policies (CloudFormation, Terraform, or the CLI
running the role's create flow), and nothing at runtime re-applies them:
the role's config is only its id, so a release never plans an update for
it. An install that predates the new log group grants keeps a role
without them.

Access denied on CreateLogGroup or PutRetentionPolicy now logs a warning
and the worker is still created; Lambda then creates the group itself,
without a retention period. Other errors stay fatal.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@alongubkin

Copy link
Copy Markdown
Member Author

@greptileai review

Comment thread crates/alien-infra/src/worker/aws.rs
A worker created while the management role could not manage its log
group reached Running without a retention period, and nothing set it
later. The update flow now runs the same idempotent log group step, so
the next update after setup grants the permissions sets it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@alongubkin

Copy link
Copy Markdown
Member Author

@greptileai review

Deployer secret reports gained deleteCommand and version, but the vendored
platform API spec still had the old report schema, so the generated client
dropped both fields when it decoded a deployment's runtime metadata and a
consumer that re-checks the state for losses refused it.

Refresh the three spec copies and add a test that sends a fully populated
report through the generated Deployment type and requires it back intact,
so the next field added without a spec refresh fails CI.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@alongubkin

Copy link
Copy Markdown
Member Author

@greptileai review

alongubkin and others added 2 commits October 6, 2026 08:20
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Adds the deployer secret report fields (deleteCommand, version) on top of
main's spec, which also carries the volume restore API.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@alongubkin

Copy link
Copy Markdown
Member Author

@greptileai review

@alongubkin

Copy link
Copy Markdown
Member Author

@greptileai review

The last review covered the volume backup and restore work, which came in only through the merge of main (it landed there separately). It isn't part of this PR's diff: the 39 changed files are the release follow-ups (custom-domain preflight, deployer secret versions and delete notes, alien deploy re-run, worker log group retention, redeploy --wait reasons) plus the platform API spec refresh, whose only change against main's spec is deleteCommand and version on deployer secret reports. Please review the diff against main.

Comment thread crates/alien-cli/src/commands/deploy.rs
Comment thread crates/alien-bindings/src/providers/vault/azure_key_vault.rs Outdated
A failed refresh has no update in flight, so the deployment carries no
desired release; the setup retry now applies the installed release
again instead of failing for want of a target.

Key Vault records version creation in whole seconds and version ids
are random, so versions written in the same second cannot be ordered.
Secret presence now reports every version in the newest second, which
any later write changes, and requires each of them to be usable.
Version listing also follows nextLink pages, since Key Vault returns
at most 25 versions per page.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@alongubkin

Copy link
Copy Markdown
Member Author

@greptileai review

Comment thread crates/alien-bindings/src/providers/vault/azure_key_vault.rs
@alongubkin

Copy link
Copy Markdown
Member Author

@greptileai review

Resolves deployer_secrets.rs (main's delivery filter plus the report version
on DeployerSecretEnv), adds the new report fields to main's new test, and
keeps the platform API spec as main's plus deleteCommand and version on
deployer secret reports.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@alongubkin
alongubkin merged commit 501ef42 into main Oct 6, 2026
30 checks passed
@alongubkin
alongubkin deleted the alon/alien-1239-alien-release-follow-ups-custom-domain-preflight-secret branch October 6, 2026 08:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant