Skip to content

perf: compile Azure models in parallel - #652

Merged
alongubkin merged 2 commits into
mainfrom
alon/alien-868-split-azure-models
Sep 18, 2026
Merged

alongubkin merged 2 commits into
mainfrom
alon/alien-868-split-azure-models

Conversation

@alongubkin

Copy link
Copy Markdown
Member

Summary

  • split the 147,886-line generated Azure model unit into five line-balanced crates that rustc can compile in parallel
  • preserve alien_azure_clients::models::<service> paths through re-exports and forward filtered/full model modes
  • centralize shard ownership in one manifest used by build scripts, tests, and OpenAPI regeneration
  • publish the generator and model shards before alien-azure-clients in the explicit release order

This PR is intentionally stacked on #646, which supplies reachable-model filtering.

Measurements

Controlled empty-target cargo check -p alien-azure-clients --no-default-features --timings:

  • before: 41.72s; monolithic Azure unit 26.5s
  • after: 22.75s; five parallel model units 4.0-4.7s, parent 6.5s
  • improvement: 45.5%

A real retained OpenAPI schema edit rebuilt one shard plus the parent in 2.65s.

A local-only empty-target branch containing every proven compile-time PR completed cargo check -p alien-cli --timings in 1m38s / 99.00s real, down from the original 7m13s baseline (77.1%, 4.37x faster). The five Azure shards ran concurrently in 6.4-7.5s.

Validation

  • shard manifest uniquely and exactly covers all 25 checked-in Azure specifications
  • cargo test -p alien-azure-clients --no-default-features --test schema_filter (3 passed)
  • cargo test -p alien-azure-clients --no-default-features --lib (57 passed)
  • cargo test -p alien-azure-clients --lib (57 passed)
  • cargo check -p alien-infra --features all-platforms,test-utils
  • package file lists generated for the generator, five model crates, and parent client
  • git diff --check

All isolated targets were deleted after evidence capture; the largest reached 13GB.

Linear: ALIEN-868

@greptile-apps

greptile-apps Bot commented Sep 17, 2026 •

Copy link
Copy Markdown

RetriggerConfidence Score: 4/5

The PR appears safe to merge, with a non-blocking maintainability issue around enforcing authoritative and unique shard ownership.

Fix All in CodexFindings

  1. P2 Shard mappings can drift ▶
Fix with agent prompt
### Issue 1
crates/alien-azure-model-generator/src/lib.rs:113-119
The shard manifest is not fully authoritative. `generate_azure_models` silently ignores selected filenames that are absent from its separate hardcoded `specs` array, while duplicate ownership resolves to the first matching shard. The download script also accepts multiple matching shards and can form an invalid output path. A future specification addition or reassignment could therefore omit a generated module and cause its shard to fail at `include!`. Add strict validation that the manifest, generator mappings, and checked-in specification files have unique, exact coverage.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

This PR partitions Azure model generation into five independently compiled crates and preserves the existing client-facing module paths through re-exports.

  • Moves shared generation and filtering logic into alien-azure-model-generator.
  • Assigns 25 OpenAPI specifications across five model shards.
  • Forwards filtered and full-model feature modes through the parent client.
  • Adds the generator and shards to workspace and crates.io release ordering.
  • Leaves a non-blocking integrity gap because shard ownership is not strictly validated against the generator’s separate specification mapping.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  M[model_shards.json] --> G[alien-azure-model-generator]
  O1[Shard 1 OpenAPI] --> S1[alien-azure-models-1]
  O2[Shard 2 OpenAPI] --> S2[alien-azure-models-2]
  O3[Shard 3 OpenAPI] --> S3[alien-azure-models-3]
  O4[Shard 4 OpenAPI] --> S4[alien-azure-models-4]
  O5[Shard 5 OpenAPI] --> S5[alien-azure-models-5]
  G --> S1
  G --> S2
  G --> S3
  G --> S4
  G --> S5
  S1 --> C[alien-azure-clients model re-exports]
  S2 --> C
  S3 --> C
  S4 --> C
  S5 --> C
Loading

Reviews (1) · Last reviewed commit: "perf: compile Azure models in parallel"

Comment thread crates/alien-azure-model-generator/src/lib.rs Outdated
Base automatically changed from alon/alien-862-prune-azure-models to main September 17, 2026 18:46
@alongubkin
alongubkin force-pushed the alon/alien-868-split-azure-models branch 6 times, most recently from 9d4fa91 to 08aff1b Compare September 18, 2026 02:54
@alongubkin

Copy link
Copy Markdown
Member Author

Final cold qualification found a regression inherited from current main: the filtered Platform client omits the getCommand endpoint required by the CLI. The exact ordinary CLI check failed after 93.50s, while all-features Rust CI masks it. Fix and CI coverage are in #667; I’ll rebase this PR after that lands, then rerun the cold benchmark before merge.

@alongubkin
alongubkin force-pushed the alon/alien-868-split-azure-models branch from 08aff1b to cc9a883 Compare September 18, 2026 03:11
@alongubkin
alongubkin merged commit b5682e0 into main Sep 18, 2026
21 checks passed
@alongubkin
alongubkin deleted the alon/alien-868-split-azure-models branch September 18, 2026 03:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant