Repository navigation
Conversation
…nt VPC origin A failed VPC origin create leaves the resource tainted. Because the origin uses create_before_destroy and a fixed, account-unique name, the replacement collides with the failed origin on every re-run. Document how to delete the unreferenced failed origin and re-run the deployment until the code fix lands. Refs aws-samples#648
…re deleting it
The recovery steps deleted the failed VPC origin after a prose-only
condition ("when no distribution uses the failed origin"). Look up the
distributions that reference the origin first and delete it only when
there are none, then re-run the deployment as a separate step once the
delete has finished.
Refs aws-samples#648
|
Update from the review on #645: 6679d83 adds a guard before the delete. The steps now look up the distributions that reference the failed origin and delete it only when there are none, and the re-run is a separate step once the delete has finished. I checked the lookup and the guard against a live stack, where they report the referencing distribution and refuse to delete. #645's Block Public Access entry now points readers here, so it would help to merge this one first. I'll rebase #645 on top right after. |
jeromevdl
left a comment
There was a problem hiding this comment.
Do we need a separate documentation PR for this workaround? Could we put the recovery steps in #648, then include any recovery guidance still needed alongside the Terraform fix?
If we publish these steps, the distribution guard needs a correction: when an account has no CloudFront distributions, the query returns None, so [ -n "$USED_BY" ] incorrectly refuses deletion. That blocks the first-deployment case described here. Normalize the query result to an empty array and check command success before proceeding.
| If the check reports a distribution, the origin is in use and deleting it would break that distribution, so leave it in place. Otherwise, wait until `aws cloudfront get-vpc-origin --id "$VPC_ORIGIN_ID"` returns `EntityNotFound` (several minutes), then re-run the deployment: | ||
|
|
||
| ```bash | ||
| ./scripts/deploy-terraform.sh dev |
There was a problem hiding this comment.
[P2] Recovery can redeploy a different environment
If the failed deployment used prod or staging, the origin lookup above reads that environment’s initialized state, but this command switches Terraform to dev’s backend and variables. That can apply changes to dev while leaving the failed environment unrecovered. Using <environment> here and specifying that the retry must retain the original AWS profile, configuration paths and command-line overrides would keep recovery scoped to the affected installation. This also applies if these steps move into #645.
Issue #, if available: Refs #648 (workaround; the code fix is tracked there)
Description of changes:
Problem
If a CloudFront VPC origin create fails (for example in an account with VPC Block Public Access, #644), every later deployment fails with:
The failed origin is tainted,
create_before_destroymakes Terraform create the replacement first, and VPC origin names are unique per account. Until someone deletes the failed origin by hand, the deployment stays stuck. #648 discusses a code fix.Solution
A troubleshooting entry in
docs/getting-started/setup.mdwith the recovery steps: read the origin ID from state, check that no CloudFront distribution references it, delete it with its ETag only if none does, wait forEntityNotFound, and re-run the deployment. Terraform then drops the missing origin from state and creates a fresh one. On a first deployment no distribution uses the failed origin yet. If one does, the steps refuse to delete it.Testing
uv run zensical build: no issues.npm run format,npm run lintandnpm run testpass.Note
#645 adds a troubleshooting entry at the same spot in
setup.md(just above "Frontend shows authentication errors") and points readers to this entry, so please merge this PR first. I'll rebase #645 right after.By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.