Repository navigation
fix(sandbox): read and delete an AWS MicroVM image by ARN, not by name - #712
Merged
ItamarZand88 merged 2 commits intoSep 25, 2026
Conversation
Three tests asserted the bare image name, and one matched with `ends_with`, which passes for a name and an ARN alike. They fail against the current code.
The pre-create probe passed the derived image name where the API takes an ARN, so AWS answered 400 Invalid ARN format. Only a 404 counts as absence, so the create was unreachable and every Live sandbox retried until it gave up. The two delete paths passed the same bare name on their fallback, which left an unrecorded image behind instead of sweeping it. The name is account-unique, so its ARN is derivable before the image exists.
 |
ItamarZand88
deleted the
itamar/alien-954-aws-sandbox-never-provisions-getmicrovmimage-is-called-with
branch
September 25, 2026 07:46
ItamarZand88
added a commit
that referenced
this pull request
Sep 25, 2026
## Summary AWS sandboxes can now be deployed with `alien-deploy`, the direct path with no Terraform or CloudFormation. Setup creates everything the runtime identity is never allowed to create itself, and the runtime builds only what it is granted. Step-by-step flow when `alien-deploy deploy` sets up a stack with a sandbox: 1. **Setup, using the deployer's credentials, creates the sandbox's build role. For `egress: deny` it also creates an operator role, a loopback-only security group and a network connector. It records each object and passes them to the sandbox's controller.** ← the heart 2. A Frozen sandbox's image is built during setup and rolled once when setup runs again with a new bundle. A Live sandbox's image is built later by the runtime, which passes the setup-made role. 3. Destroy removes the images, then everything setup created, in dependency order. This PR changes AWS sandboxes from template-only to deployable directly, with the same objects the templates create. ## What was broken `alien-deploy` could not deploy an AWS sandbox. The build role, the deny egress objects and the remote-access grant exist only in the Terraform and CloudFormation setups. The runtime identity is deliberately never given `iam:CreateRole`, so it cannot make them. ## What I did - **Setup scaffolding.** A setup step creates the build role, and for deny egress an operator role, a loopback-only security group and a network connector (through Cloud Control). It also writes a published sandbox's remote-access grant onto the Remote Bindings role. Each object is tagged, recorded in the deployment's runtime metadata and verified before it is adopted. - **Frozen sandboxes on the direct path.** Setup builds the image. A setup re-run with a new bundle re-grants the role and rolls the image once. The runtime never builds or deletes a Frozen image. - **Guards.** Two Deny statements on the management identity block it from writing to setup's sandbox roles and to its own role. The drift check lets an update add either guard, since a Deny only narrows the identity, but not remove one. An earlier version must accept stacks carrying them, and a separate PR (#718) adds that tolerance on main. It has to land first. - **Updates that need setup are refused.** A runtime update that would change scaffolding (a new sandbox, a different egress, network or build policy, or publishing and unpublishing) is refused with the reason, until setup runs again. - **Teardown.** It deletes the image, then the connector, the security group (after the connector's network interfaces are released), the roles and the grant. Anything already gone is skipped. - **Destroy after a failed setup.** When setup fails before the management identity exists, and nothing runtime cleanup acts on has come up, the manager skips runtime cleanup. The destroy goes straight to setup teardown. Before this, it ended in `delete-failed`, and the build role stayed in the account. ## Files touched - `crates/alien-infra/src/setup_scaffolding/`: the scaffolding step, AWS sandbox roles and egress, seeding and teardown. - `crates/alien-infra/src/sandbox/aws.rs`: Frozen build, roll and ownership rules on the direct path. - `crates/alien-deployment/src/`: runs the scaffolding in initial setup, refuses updates that need setup, and tears scaffolding down. - `crates/alien-core/src/`: the runtime-metadata record and the build role naming. - `crates/alien-permissions/permission-sets/`: the two management guards. - `crates/alien-preflights/src/`: the build role name budget and the guard exemption. - `crates/alien-aws-clients/src/`: the Cloud Control, EC2 and IAM calls the scaffolding needs. - CloudFormation and Terraform generator snapshots, updated for the guards. ## How I tested - **Manually (us-west-2):** did a direct deployment with the local control plane, using a Live and a Frozen sandbox with open egress. - Setup created both build roles and built the Frozen image under setup tags. - A remote-access session on the Live sandbox started in 2.4s and ran a probe. - Setup re-run with a new Frozen bundle rolled that image exactly once. - Added Live and Frozen `deny` sandboxes. Setup created their operator roles, loopback-only groups and ACTIVE connectors. Sessions started and TCP out was blocked. - `alien-deploy destroy` removed every object setup and the runtime created. The account inventory matched its pre-run baseline. - With a standalone manager, `alien deploy` failed in setup after creating the build role. `alien destroy` then reached `deleted` and removed the role. - **Unit tests:** `alien-infra` 741, `alien-deployment` 141, `alien-manager` 263, plus preflights and core. Tests use strict mocks for every IAM, EC2, Cloud Control and MicroVM call, and the behavioural fixes are mutation-tested. - **Anything I couldn't test:** - Deny sessions were started with admin credentials; the compute grant comes from `sandbox/management` and was checked by configuration only. - Under `egress: deny`, DNS still resolves, because the security group doesn't cover the VPC resolver. That gap also exists in the template setups and is tracked separately. I also ran a security review on the diff. What it checked: - **The runtime minting a role:** `sandbox/provision` grants `iam:PassRole` on the setup-made role only and never `iam:CreateRole`. The management guards deny writes to setup's sandbox roles and to the management role (`aws_management_role_guards` snapshot). - **Adopting a role someone else made:** a same-named role is adopted only if it carries setup's tags and its trust, inline and attached policies match exactly. Otherwise setup refuses (`aws_sandbox.rs` adoption tests). - **Leaving a remote grant behind:** unpublishing requires setup, and setup revokes the grant even when the stack no longer declares the Remote Bindings role. - **A destroy that skips cleanup it owes:** the rule returns nothing once a Live resource, a synced secrets vault or a Frozen type with runtime cleanup has come up. It also returns nothing on any resource without a lifecycle (`deleting.rs` tests). - **Deny egress holes:** the group has no ingress and egress only to `127.0.0.1/32`, and a tagged group that allows more is repaired or refused (`aws_sandbox_egress.rs` tests). Nothing turned up. Depends on #700 (base), #715 (image tagging) and the #712 → #713 → #714 chain (image read, build inputs, teardown). This branch merges #714 and #715 and will rebase once they land.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
An AWS Live sandbox never provisioned: the controller looked up and deleted its MicroVM image by name, but the MicroVM API only accepts ARNs.
Step-by-step flow when a Live sandbox is created:
400 Invalid ARN format, and only a 404 counts as "absent", so the create never ran. ← the bugThis PR changes those calls from the image name to its ARN.
What was broken
Every Live AWS sandbox retried its create until it gave up, because the pre-create read was refused before the create could happen. Teardown could also leave an image behind.
What I did
ends_with, which passed for a name and an ARN alike.Files touched
crates/alien-infra/src/sandbox/aws.rs: ARN-based read and delete, plus tests that pin the identifier.How I tested
GetMicrovmImagewith a bare name returns 400 and with the ARN returns the image. A direct deployment then built, served and deleted Live sandbox images with this code.cargo test -p alien-infra --features all-platforms,test-utils --libpasses.