Skip to content

feat: change an AWS machine type within one architecture at runtime - #833

Merged
alongubkin merged 7 commits into
mainfrom
alon/alien-1199-simplify-compute-updates
Oct 3, 2026
Merged

alongubkin merged 7 commits into
mainfrom
alon/alien-1199-simplify-compute-updates

Conversation

@islo-labs

@islo-labs islo-labs Bot commented Oct 3, 2026

Copy link
Copy Markdown
Contributor

Summary

On AWS, a capacity group can move to another catalog machine of the same CPU architecture (for example t4g.small → t4g.medium) as an ordinary deployment update. Any other machine change still needs setup.

This reverts #823 and keeps only its small core:

  • instance_catalog::is_same_architecture_aws_machine(old, new).
  • The Frozen compatibility check accepts a same-architecture AWS machine change and takes the new group's profile with it. The compute mutation has already validated the selected machine against the workloads (validate_compute_pool_selection) and derived the profile from it, so the check does not re-derive or compare disk sizing. feat(preflights): allow same-architecture AWS machine changes at runtime #823's InstalledDisk / requested_disk / profile_matches_catalog (~130 lines that duplicated the mutation's rules) are gone.
  • ComputeCluster::validate_update allows the same pair, so the executor plans the update and the runtime controller rolls the fleet.
  • When a machine change needs setup, the error names the pool and both machines: Frozen resource 'compute' was modified (capacity group 'general' changes machine from 't4g.medium' to 'm7i.large', but without setup a machine can change only to an AWS machine of the same CPU architecture)….

What remains: counts, bounds and same-architecture AWS machine changes are runtime-managed; pools, placement, networking, cross-architecture, unknown machines, and GCP/Azure machine changes need setup. Limitation accepted: a disk size that changes together with the machine follows the mutation's output instead of being checked separately.

Validation

Real AWS (us-east-2), a running container deployment on t4g.small, using these preflights in the manager:

  • Changed the pool to t4g.medium: the update passed preflights, the controller wrote a new launch template version, scaled out, drained and terminated the old machine. The fleet was on t4g.medium after ~3 minutes, and the app's /health answered 200 on all 41 probes (every 5 s) during the roll.
  • Changed it to m7i.large (x86_64): the update was refused before any infra change with the message above.
  • Changed it to t4g.nope: the update failed in preflights with Unknown aws machine 't4g.nope' for pool 'general'.
  • Changed it back to t4g.medium: the deployment recovered.

Tests: cargo test -p alien-preflights -p alien-core (new: same-arch change passes, cross-arch / unknown / GCP need setup with the named message, a nested-virtualization change stays frozen; validate_update allows only the same-arch pair). Generated schema descriptions updated with pnpm -C packages/core generate.

🤖 Generated with Claude Code

alongubkin and others added 2 commits October 3, 2026 04:12
…at runtime

A machine change on AWS between catalog machines of the same CPU
architecture is now an ordinary update instead of a setup rerun: the
compute mutation already validates the new machine against the workloads
and derives its profile, so the Frozen check accepts the new machine and
profile together, and ComputeCluster::validate_update allows the pair.
Any other machine change still needs setup, and the error now names the
pool and both machines.

This replaces the reverted #823 without re-deriving disk sizing in the
compatibility check.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@greptile-apps

greptile-apps Bot commented Oct 3, 2026 •

Copy link
Copy Markdown

RetriggerConfidence Score: 5/5

[High risk] Changes how machine type updates are validated and applied to compute clusters.

The PR appears safe to merge based on the reviewed changes and regression coverage.

Summary

The PR permits same-architecture AWS machine changes during ordinary deployment updates while retaining setup requirements for other machine changes. The latest change adds a regression that passes mutation-derived workload profiles through compatibility and update planning.

  • The regression checks a 40 GiB workload profile, one planned update with no create or delete, and rejection of a cross-architecture change.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A[Compute selection] --> B[Mutation derives machine profile]
  B --> C[Frozen compatibility check]
  C -->|Same-architecture AWS change| D[Executor plans update]
  C -->|Other machine change| E[Setup required]
Loading

Reviews (5) · Last reviewed commit: "test: cover derived compute profiles thr..."

@greptile-apps

greptile-apps Bot commented Oct 3, 2026

Copy link
Copy Markdown

Comments Outside Diff

These findings could not be posted inline.

  • P2 Machine-change integration coverage removed crates/alien-preflights/tests/compute_machine_change.rs:1 ▶

    Deleting this test, along with the deployment-preflight and executor-planning tests, leaves machine changes covered by separate tests with hand-built profiles. Please retain a test that passes actual mutation output through frozen compatibility and update planning. Without it, a mismatch between profile derivation, setup authorization, and planning could go undetected.

    Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

@islo-labs

islo-labs Bot commented Oct 3, 2026

Copy link
Copy Markdown
Contributor Author

@greptileai

Comment thread crates/alien-preflights/src/mutations/compute_cluster.rs Outdated
@islo-labs

islo-labs Bot commented Oct 3, 2026

Copy link
Copy Markdown
Contributor Author

Addressed the second review: unchanged idle clusters keep their existing mutation behavior. Changed machine profiles now reuse materialize_selected_group, the same selection validation and disk/profile derivation used by the compute mutation. This also checks nested virtualization, without reimplementing catalog or disk rules. All 378 preflight tests pass, including invalid-profile and nested-virtualization regressions. @greptileai

Comment thread crates/alien-preflights/src/compatibility/frozen_resources_unchanged.rs Outdated
@islo-labs

islo-labs Bot commented Oct 3, 2026

Copy link
Copy Markdown
Contributor Author

@greptileai

@islo-labs

islo-labs Bot commented Oct 3, 2026

Copy link
Copy Markdown
Contributor Author

Addressed the outside-diff machine-change coverage finding in 392055c. The regression passes actual ComputeClusterMutation output for a container requesting 40Gi through PreflightRunner compatibility and StackExecutor planning. It verifies that an AWS t4g.micro → t4g.small change preserves the derived workload disk profile, passes compatibility without a frozen-change override, and plans exactly one update with no create/delete. An m7i.large change fails compatibility with the architecture explanation and fails planning with RESOURCE_CONFIG_INVALID.

Validation: cargo test -p alien-infra --all-features --lib core::executor_tests -- --nocapture passes all 123 executor tests, including the new regression. The planner fixture uses the existing built-in Kubernetes controller registration solely to invoke the platform-independent ComputeCluster validation; no provider is contacted. This does not rerun the AWS fleet rollout already recorded in the PR.

@greptileai

@alongubkin
alongubkin merged commit caa6947 into main Oct 3, 2026
22 checks passed
@alongubkin
alongubkin deleted the alon/alien-1199-simplify-compute-updates branch October 3, 2026 20:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant