Skip to content

feat(preflights): allow same-architecture AWS machine changes at runtime - #823

Merged
ItamarZand88 merged 6 commits into
mainfrom
itamar/alien-1167-change-a-running-deployments-machine-type-without-re-running
Oct 2, 2026
Merged

ItamarZand88 merged 6 commits into
mainfrom
itamar/alien-1167-change-a-running-deployments-machine-type-without-re-running

Conversation

@ItamarZand88

@ItamarZand88 ItamarZand88 commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

Summary

On AWS, a ComputeCluster capacity group can now move to another machine of the same CPU architecture (for example t4g.micro to t4g.small) as an ordinary update instead of a setup rerun. Cross-architecture changes, unknown machines, and GCP and Azure still require setup.

When a deployment update changes a capacity group's machine type:

  1. The Frozen compatibility check compares the installed ComputeCluster with the new one.
  2. On AWS it accepts a machine change when both machines are in the catalog with the same architecture, and the new profile is one the compute mutation could produce for that machine. ← the change
  3. The executor's ComputeCluster::validate_update allows the same pair, so the update is planned and the runtime controller rolls the fleet.

This PR changes a same-architecture AWS machine change from "rerun setup" to a runtime update.

What was broken

ComputeCluster is Frozen: setup owns it, and runtime may only change capacity (counts and autoscale bounds). Any machine-type change failed the Frozen check with DEPLOYMENT_SETUP_REQUIRED, and validate_update rejected it as immutable, even though setup creates no launch template or autoscaling group. The fleet is the runtime's to manage, and an arm64-to-arm64 change needs no new identity, network or image.

What I did

  • frozen_resources_unchanged.rs: runtime_managed_frozen_change takes the platform. On AWS it also normalizes a group's instance_type and profile when:
    • both machines are in the AWS catalog with the same architecture;
    • the profile matches the catalog (CPU, memory, GPU, architecture) and its disk is what the compute mutation produces with the workload's request unchanged: a configurable disk keeps the installed size; moving off a fixed disk, the catalog disk grown to the containers' request, no larger than the installed disk; a fixed-disk target at its catalog size, covering the request;
    • nested virtualization is only kept on a machine that supports it.
      Any other group, profile, placement or networking change still needs setup.
  • instance_catalog.rs: is_same_architecture_aws_machine(old, new).
  • compute_cluster.rs: validate_update accepts an unchanged name or a same-architecture AWS pair. Its error says what is allowed instead of "immutable". The doc comment says setup owns identity and network and runtime manages the fleet. Generated schemas updated with pnpm -C packages/core generate.
  • updating.rs: comment on which Frozen resources the runtime may reconcile.

Files touched

  • crates/alien-preflights/src/compatibility/frozen_resources_unchanged.rs
  • crates/alien-core/src/instance_catalog.rs
  • crates/alien-core/src/resources/compute_cluster.rs
  • crates/alien-deployment/src/updating.rs
  • packages/core/src/generated/schemas/computeCluster.json, packages/core/src/generated/zod/compute-cluster-schema.ts (generated)
  • Tests: crates/alien-preflights/tests/compute_machine_change.rs, crates/alien-preflights/src/runner.rs, crates/alien-infra/src/core/executor_tests/update_tests.rs, plus unit tests in the files above.

How I tested

  • Manually, real AWS: I deployed a container app on t4g.micro, changed the pool to t4g.small through a deployment update, and watched the fleet roll in about 4.5 minutes with the endpoint answering 200 throughout; the update succeeded. Then I tore it down.
  • Unit and integration tests:
    • Frozen check: a same-architecture change passes, with new bounds too; cross-architecture, unknown or removed machines, GCP and Azure, a profile change without a machine change, and forged profiles (architecture, CPU, memory, GPU, disk) need setup; a disk grown or shrunk with the machine change needs setup; fixed-disk moves follow the containers' request within the installed disk, with or without a recorded profile; a fixed-disk target must cover the request; a nested-virtualization group moves only to a capable machine.
    • The preflight runner clears a resize and a same-architecture change on the installed platform and refuses arm64 to x86_64.
    • The executor plans a same-architecture update and refuses a cross-architecture one with RESOURCE_CONFIG_INVALID.
    • cargo test for alien-core (616), alien-preflights with runtime-checks (393 + integration, including real compute-mutation output for an i4i.xlarge to m7i.large move), alien-deployment (147+) and the alien-infra executor tests (123) pass.
  • Anything I couldn't test: whether the generated TypeScript schema change matters to any consumer beyond the description text (it is description-only).

A ComputeCluster capacity group on AWS may move to another catalog machine
of the same CPU architecture as a runtime update. The Frozen check accepts it
when the new profile is one the compute mutation could produce for that
machine, and ComputeCluster::validate_update allows the same pair. A
cross-architecture change, an unknown machine, GCP and Azure still require
setup.
@greptile-apps

greptile-apps Bot commented Oct 2, 2026 •

Copy link
Copy Markdown

RetriggerConfidence Score: 5/5

[High risk] Allows runtime machine type changes on AWS within same CPU architecture.

The PR appears safe to merge based on the reviewed changes.

Summary

The PR permits same-architecture AWS ComputeCluster machine changes through the Frozen compatibility check and update validation, while retaining setup requirements for other machine changes. The latest revision derives an installed group’s disk from its catalog machine when its profile is missing and checks that a fixed-disk target covers the matched workload request.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A[Machine change] --> B{AWS catalog machines<br/>same architecture?}
  B -- No --> C[Setup required]
  B -- Yes --> D{Target profile and<br/>storage checks pass?}
  D -- No --> C
  D -- Yes --> E[Runtime fleet update]
Loading

Reviews (6) · Last reviewed commit: "fix(preflights): read a fixed disk from ..."

Comment thread crates/alien-preflights/src/compatibility/frozen_resources_unchanged.rs Outdated
Comment thread crates/alien-preflights/src/compatibility/frozen_resources_unchanged.rs Outdated
Comment thread crates/alien-preflights/src/compatibility/frozen_resources_unchanged.rs Outdated
Comment thread crates/alien-preflights/src/compatibility/frozen_resources_unchanged.rs Outdated
Comment thread crates/alien-preflights/src/compatibility/frozen_resources_unchanged.rs Outdated
@ItamarZand88
ItamarZand88 merged commit dc3cb25 into main Oct 2, 2026
22 checks passed
@ItamarZand88
ItamarZand88 deleted the itamar/alien-1167-change-a-running-deployments-machine-type-without-re-running branch October 2, 2026 14:01
islo-labs Bot pushed a commit that referenced this pull request Oct 3, 2026
…at runtime

A machine change on AWS between catalog machines of the same CPU
architecture is now an ordinary update instead of a setup rerun: the
compute mutation already validates the new machine against the workloads
and derives its profile, so the Frozen check accepts the new machine and
profile together, and ComputeCluster::validate_update allows the pair.
Any other machine change still needs setup, and the error now names the
pool and both machines.

This replaces the reverted #823 without re-deriving disk sizing in the
compatibility check.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
alongubkin added a commit that referenced this pull request Oct 3, 2026
…833)

## Summary

On AWS, a capacity group can move to another catalog machine of the same
CPU architecture (for example `t4g.small` → `t4g.medium`) as an ordinary
deployment update. Any other machine change still needs setup.

This reverts #823 and keeps only its small core:

- `instance_catalog::is_same_architecture_aws_machine(old, new)`.
- The Frozen compatibility check accepts a same-architecture AWS machine
change and takes the new group's `profile` with it. The compute mutation
has already validated the selected machine against the workloads
(`validate_compute_pool_selection`) and derived the profile from it, so
the check does not re-derive or compare disk sizing. #823's
`InstalledDisk` / `requested_disk` / `profile_matches_catalog` (~130
lines that duplicated the mutation's rules) are gone.
- `ComputeCluster::validate_update` allows the same pair, so the
executor plans the update and the runtime controller rolls the fleet.
- When a machine change needs setup, the error names the pool and both
machines: `Frozen resource 'compute' was modified (capacity group
'general' changes machine from 't4g.medium' to 'm7i.large', but without
setup a machine can change only to an AWS machine of the same CPU
architecture)…`.

What remains: counts, bounds and same-architecture AWS machine changes
are runtime-managed; pools, placement, networking, cross-architecture,
unknown machines, and GCP/Azure machine changes need setup. Limitation
accepted: a disk size that changes together with the machine follows the
mutation's output instead of being checked separately.

## Validation

Real AWS (us-east-2), a running container deployment on `t4g.small`,
using these preflights in the manager:

- Changed the pool to `t4g.medium`: the update passed preflights, the
controller wrote a new launch template version, scaled out, drained and
terminated the old machine. The fleet was on `t4g.medium` after ~3
minutes, and the app's `/health` answered 200 on all 41 probes (every 5
s) during the roll.
- Changed it to `m7i.large` (x86_64): the update was refused before any
infra change with the message above.
- Changed it to `t4g.nope`: the update failed in preflights with
`Unknown aws machine 't4g.nope' for pool 'general'`.
- Changed it back to `t4g.medium`: the deployment recovered.

Tests: `cargo test -p alien-preflights -p alien-core` (new: same-arch
change passes, cross-arch / unknown / GCP need setup with the named
message, a nested-virtualization change stays frozen; `validate_update`
allows only the same-arch pair). Generated schema descriptions updated
with `pnpm -C packages/core generate`.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Alon Gubkin <alon@alien.dev>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: islo <sandbox@islo.dev>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant