Skip to content

ci: serve Terraform providers from a read-only mirror in the Rust job - #887

Merged
alongubkin merged 2 commits into
mainfrom
alon/alien-1216-terraform-provider-mirror
Oct 5, 2026
Merged

alongubkin merged 2 commits into
mainfrom
alon/alien-1216-terraform-provider-mirror

Conversation

@alongubkin

@alongubkin alongubkin commented Oct 5, 2026 •

Copy link
Copy Markdown
Member

Problem

terraform_validate (and terraform_test / terraform_plan_with_vars) in crates/alien-terraform/src/test_utils.rs run terraform init in a fresh temp directory for each stack, so every test downloads its providers again. In CI the generator suite takes ~920 test-seconds.

A shared TF_PLUGIN_CACHE_DIR isn't an option. Parallel inits write to the cache and fail with "the cached package … does not match any of the checksums recorded in the dependency lock file" (#853, and again when pre-warmed).

Change

  • scripts/terraform-provider-mirror.sh runs one terraform init with every provider the generator declares (aws, awscc, tls, google, google-beta, azurerm, Azure/azapi, time, kubernetes, helm, random, same constraints as generator.rs). It copies them into a local directory and writes a CLI config:
    • a filesystem_mirror for exactly the mirrored providers
    • direct for everything else, so a provider added later still installs from the registry
  • The Rust job runs it after installing Terraform and exports TF_CLI_CONFIG_FILE. Tests and test_utils.rs are unchanged.
  • The generator's provider constraints now come from one PROVIDER_REQUIREMENTS table in generator.rs. The unit test provider_mirror_matches_generator_requirements fails if the script's list drifts from it.

Measured

  • CI benchmark, 2 runs (with the other test changes being measured at the time): the generator suite went from 920 to ~440 test-seconds, and all 5,518 tests passed.
  • Locally on macOS: the script took 37 s for 11 providers, and all 196 generator tests pass against the mirror.

🤖 Generated with Claude Code

The Terraform generator tests run `terraform init` in a fresh directory for
each stack, downloading every provider again (~920 test-seconds per run).
A shared plugin cache breaks under parallel inits; a filesystem mirror is
only read. Providers not in the mirror still install from the registry.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@greptile-apps

greptile-apps Bot commented Oct 5, 2026 •

Copy link
Copy Markdown

RetriggerConfidence Score: 5/5

[Medium risk] Changes how Terraform providers are sourced in CI tests.

The PR appears safe to merge; no new actionable issue or outstanding previous finding remains.

Summary

The Rust CI job now prepares a local Terraform provider mirror before running tests. Generator provider requirements were moved into a table, with a test that checks the mirror script against it.

  • The previously reported constraint-drift finding is resolved by the new consistency test.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A[Install Terraform in Rust CI job] --> B[Initialize providers once]
  B --> C[Populate local provider mirror]
  C --> D[Export Terraform CLI configuration]
  D --> E[Generator tests initialize stacks from mirror]
Loading

Reviews (2) · Last reviewed commit: "test: keep the Terraform provider mirror..."

Comment thread scripts/terraform-provider-mirror.sh
The generator's provider constraints now come from one PROVIDER_REQUIREMENTS
table, and a unit test fails when scripts/terraform-provider-mirror.sh
declares different providers or constraints.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@alongubkin

Copy link
Copy Markdown
Member Author

@greptileai review

@alongubkin
alongubkin merged commit 43edeef into main Oct 5, 2026
22 checks passed
@alongubkin
alongubkin deleted the alon/alien-1216-terraform-provider-mirror branch October 5, 2026 13:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant