Skip to content

Add incremental builds to the hybrid API architecture - #1698

Closed
David Pine (IEvangelist) wants to merge 41 commits into
ievangelist-hybrid-api-renderingfrom
ievangelist-astro-incremental-builds
Closed

David Pine (IEvangelist) wants to merge 41 commits into
ievangelist-hybrid-api-renderingfrom
ievangelist-astro-incremental-builds

Conversation

@IEvangelist

@IEvangelist David Pine (IEvangelist) commented Sep 18, 2026 •

Copy link
Copy Markdown
Member

Summary

Important

This is the top layer of native stack #1911. #1874 targets release/17.0; this PR targets #1874's ievangelist-hybrid-api-rendering branch. The incremental enhancement depends on that hybrid architecture rather than maintaining a separate static API build path.

Add an opt-in hybrid incremental build for the routes that #1874 still prerenders. pnpm build:incremental enables both ASPIRE_HYBRID_API_DOCS=1 and ASPIRE_INCREMENTAL_BUILD=1: API detail HTML and Markdown remain on-demand in the hybrid renderer, while Astro can restore unchanged documentation, landing-page, endpoint, and owned OG outputs in dist-hybrid/client.

The incremental wrapper supplies stable cache keys for prerendered routes. Its compatibility partition covers ordinary source, configuration, public assets, patches, package resolution, platform, Node version, and public build inputs. Generated API data is hashed separately so API landing pages expire when their data changes without invalidating unrelated static routes; the on-demand route catalog and server artifact are always rebuilt.

Deployment identity substitution is limited to Astro's prerender environment. Cached static HTML receives the current commit through equal-width byte stamping, while the on-demand server bundle retains the real commit value. Pagefind continues to run normally and is not cached or reused.

Reduced scope

  • The PR is now 22 files relative to Add hybrid API docs with on-demand rendering #1874, down from the earlier 40-file pilot.
  • Removed the six-scenario qualification workflow, repeated-build comparison scripts, cache diagnostics, timing additions, and evidence artifacts.
  • Removed changes made only to force byte-identical clean builds: Pagefind manifest ordering, RSS clock injection, random component ID normalization, and Twoslash tokenization timing changes.
  • Retained only the hybrid incremental implementation, required Astro/hybrid compatibility fixes, focused correctness tests, and successful-build-only artifact upload gating.
  • No contributor-guide changes are included.

Safety and rollout

  • Ordinary production and ordinary hybrid builds remain unchanged unless the incremental opt-in is set.
  • Incremental mode requires hybrid mode; the unsupported standalone static-API incremental combination is rejected.
  • Cache manifests and identity indexes are written atomically. Missing or damaged cached outputs rebuild rather than producing success-shaped fallbacks.
  • Astro's Rolldown asset handles are normalized so unchanged route dependency hashes remain reusable.
  • Generated Starlight icon metadata is captured after setup so cold-checkout inputs are included correctly.
  • The development-center catalog uses Astro 7's canonical content entry IDs instead of the deprecated data.slug getter.
  • Bundled prerender configuration resolves the generated Twoslash SDK declaration from the frontend working directory.
  • Build artifacts upload only after the build step succeeds.
  • This PR makes no performance claim and does not add ordinary CI cache integration. The retained feature is an opt-in local path until a smaller CI design is intentionally introduced.

Third-party links and affiliations

None.

Validation

Current stacked head: e89a7af88d304139cbd4e2ccf0a10d1e7ce8c396.

  • pnpm lint from src/frontend - passed.
  • pnpm test:unit from src/frontend - passed: 89 files, 1,256 passed, one expected skip.
  • git diff --check - passed.
  • dotnet test tests\StaticHost.Tests\StaticHost.Tests.csproj --filter "FullyQualifiedName~HybridApiDocs" --nologo - previously passed: 39 tests.
  • dotnet test tests\Aspire.Dev.AppHost.Tests\Aspire.Dev.AppHost.Tests.csproj --filter "FullyQualifiedName~HybridApiDocs" --nologo - previously passed: 7 tests.
  • No local production build was run, per repository guidance.

@aspire-repo-bot

Copy link
Copy Markdown
Contributor

Frontend HTML artifact ready

The latest frontend build uploaded the frontend-dist artifact for PR #1698. Use the VS Code button below to open this PR with GitHub Artifacts Explorer and browse the built HTML locally.

VS Code: Open PR #1698 artifacts

This comment updates automatically when a new frontend build artifact is uploaded.

@IEvangelist David Pine (IEvangelist) added the ci:incremental-pilot Opt into the non-deploying Astro API cache qualification matrix label Sep 18, 2026
Comment thread src/frontend/tests/unit/qualification-inputs.vitest.test.ts Fixed
Comment thread src/frontend/tests/unit/qualification-inputs.vitest.test.ts Fixed
@IEvangelist David Pine (IEvangelist) added ci:incremental-pilot:identity Opt into focused incremental build identity qualification ci:incremental-pilot Opt into the non-deploying Astro API cache qualification matrix and removed ci:incremental-pilot Opt into the non-deploying Astro API cache qualification matrix ci:incremental-pilot:identity Opt into focused incremental build identity qualification labels Sep 19, 2026
@IEvangelist David Pine (IEvangelist) changed the title Upgrade age-qualified dependencies and pilot Astro API incremental builds Upgrade age-qualified dependencies and expand Astro incremental-build pilot Sep 21, 2026
@IEvangelist David Pine (IEvangelist) added ci:incremental-pilot Opt into the non-deploying Astro API cache qualification matrix and removed ci:incremental-pilot Opt into the non-deploying Astro API cache qualification matrix labels Sep 21, 2026
@IEvangelist David Pine (IEvangelist) added ci:incremental-pilot Opt into the non-deploying Astro API cache qualification matrix and removed ci:incremental-pilot Opt into the non-deploying Astro API cache qualification matrix labels Oct 14, 2026
@IEvangelist
David Pine (IEvangelist) force-pushed the ievangelist-astro-incremental-builds branch from 4996f45 to 8aae54a Compare October 19, 2026 13:37
@IEvangelist
David Pine (IEvangelist) changed the base branch from main to ievangelist-hybrid-api-rendering October 19, 2026 13:37
@IEvangelist
David Pine (IEvangelist) added this pull request to stack #1911 October 19, 2026 13:37
@IEvangelist David Pine (IEvangelist) changed the title Upgrade age-qualified dependencies and expand Astro incremental-build pilot Add incremental builds to the hybrid API architecture Oct 19, 2026
@IEvangelist David Pine (IEvangelist) added ci:incremental-pilot Opt into the non-deploying Astro API cache qualification matrix and removed ci:incremental-pilot Opt into the non-deploying Astro API cache qualification matrix labels Oct 19, 2026
@IEvangelist
David Pine (IEvangelist) marked this pull request as ready for review October 19, 2026 20:09
Copilot AI balanced review requested due to automatic review settings October 19, 2026 20:09

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Cache-key generation can persist degraded external-service outputs across subsequent builds.

2 open issues
1 file not reviewed
  • src/frontend/pnpm-lock.yaml: Generated file
What changed in this PR

Adds opt-in incremental prerendering on top of #1874’s hybrid architecture, while API detail routes remain on demand.

Changes:

  • Adds cache partitioning, route keys, and deployment-identity stamping.
  • Adds compatibility fixes and focused regression tests.
  • Restricts artifact uploads to successful builds.
File Description
src/​frontend/​tests/​unit/​twoslash-environment.vitest.test.ts Tests SDK declaration resolution.
src/​frontend/​tests/​unit/​og-build-cache.vitest.test.ts Tests OG cache keys and exclusions.
src/​frontend/​tests/​unit/​incremental-build.vitest.test.ts Tests caching, invalidation, and identity stamping.
src/​frontend/​tests/​unit/​icon-safelist.vitest.test.ts Tests deferred icon safelist loading.
src/​frontend/​tests/​unit/​dev-resource-catalog.vitest.test.ts Updates canonical-ID tests.
src/​frontend/​tests/​unit/​build-performance.vitest.test.ts Updates upload-condition assertions.
src/​frontend/​src/​utils/​og-build-cache.ts Generates OG cache keys.
src/​frontend/​src/​utils/​dev-center/​catalog-normalization.ts Uses canonical content IDs.
src/​frontend/​src/​types/​incremental-build.d.ts Types the generated icon digest.
src/​frontend/​src/​pages/​og/​reference/​samples/​[sample].png.ts Adds sample-image cache keys.
src/​frontend/​src/​pages/​og/​[...slug].png.ts Adds documentation-image cache keys.
src/​frontend/​scripts/​build-incremental.mjs Launches opt-in incremental builds.
src/​frontend/​pnpm-workspace.yaml Registers Astro compatibility patch.
src/​frontend/​pnpm-lock.yaml Locks dependencies and patch hashes.
src/​frontend/​patches/​starlight-plugin-icons@1.1.6.patch Defers safelist evaluation.
src/​frontend/​patches/​astro@7.3.2.patch Normalizes Rolldown asset handles.
src/​frontend/​package.json Adds build command and dependencies.
src/​frontend/​config/​twoslash.config.mjs Resolves declarations from frontend working directory.
src/​frontend/​config/​incremental-build.mjs Implements caching and identity stamping.
src/​frontend/​astro.config.mjs Wires incremental mode into Astro.
package.json Exposes root incremental command.
.github/​workflows/​frontend-build.yml Gates artifact uploads on build success.

Review effort: Balanced

async getStaticPaths() {
const paths = await prerenderer.getStaticPaths();
cacheableComponents.clear();
const cacheablePaths = paths.map((path) => {
const paths = await prerenderer.getStaticPaths();
cacheableComponents.clear();
const cacheablePaths = paths.map((path) => {
const component = path.route.component.replaceAll('\\', '/');
@IEvangelist
David Pine (IEvangelist) force-pushed the ievangelist-hybrid-api-rendering branch from 2f75c44 to 586d45e Compare October 19, 2026 20:27
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Wrap the default prerenderer using Astro's public hook, retaining metadata for keyed components while avoiding render scopes and response buffering for uncached docs. Preserve lifecycle receivers, image collection and response bytes.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Add ordinary cache-warm full-build controls and qualify OG asset invalidation and search cache recovery. Keep emoji-dependent cards uncached and production opt-out. Remove internal pilot details from the contributor guide.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Gate normal and pilot uploads on successful build steps, preserving reports after later test failures but not failed or cancelled builds. Recognize Astro's trailing slash on restored OG endpoints without broadening the allowed route families.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Print bounded mismatch counts and paths in the job log so failures remain diagnosable when build-gated artifact uploads are skipped.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Resolve Starlight through import.meta.resolve so Playwright can load the search suite when the package export has no CommonJS require condition.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Keep the Starlight ESM resolution fix surgical by restoring the file's existing formatting.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Use Starlight's canonical content entry ID directly so forced clean builds do not trip Astro's deprecated data.slug getter while generating the resource catalog.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Resolve the generated declaration bundle from the frontend project root so Astro's relocated prerender config continues to load it during forced-clean builds.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Capture Community Toolkit contributor responses under the lowercase repository URL used by the contributor cache so deterministic replay does not reject a case-only mismatch.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Port the custom mobile sidebar lifecycle to Starlight 0.42's native Popover API, preserve focus and responsive behavior, and restore homepage reveal contrast under Axe 4.13.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Record normalized client and prerender module hashes during the CI pilot so unexpected route invalidation identifies the changing dependency without weakening reuse or output checks.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Backport Astro's Rolldown asset placeholder normalization so unchanged font modules keep stable dependency hashes during incremental builds while 7.4.0 remains inside the seven-day release buffer.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Raise the isolated incremental qualification timeout to 240 minutes so the longest API-data scenario can complete the existing browser matrix after build, lint, and unit validation.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Restore merge-resolved qualification behavior, remove the custom Pagefind cache, retain the upgraded page-actions patch, and adapt deterministic Pagefind finalization to hybrid output.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Keep the opt-in hybrid incremental implementation and focused correctness coverage while dropping benchmark workflows, comparison scripts, and determinism-only patches.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@IEvangelist
David Pine (IEvangelist) force-pushed the ievangelist-astro-incremental-builds branch from 2fe42e3 to e89a7af Compare October 19, 2026 20:29
@IEvangelist
David Pine (IEvangelist) removed this pull request from stack #1911 October 19, 2026 20:37
@IEvangelist

Copy link
Copy Markdown
Member Author

Consolidated into #1874 as commit 8e3038f. The combined branch has the same final tree as this PR's head, so the separate stacked PR is no longer needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants