Repository navigation
test(web): async app generator — generated async apps, liveness oracle, equivalence pairs, regression gate - #4371
Merged
Merged
Conversation
… over async memos, with safety and liveness oracles An opt-in campaign (ASYNC_GEN=1, pnpm async-gen) that generates small apps (signals, sync and async memos with manual flights, latest()/isPending() reads, Loading / Errored / Show / leaves with grouped or split attribute effects) and scripts (writes, actions, settling flights in any order, flush / microtask / task). Oracles: nothing throws or logs, no unhandled rejection, no hole shows undefined/NaN, and once every flight has settled the screen equals a fresh synchronous render of the final inputs. Survey, shrink and minimize modes; the minimizer reduces the app as well as the script. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0117BoWKJDzwaZPgaGWE181b
…t Errored, store twin, flush mode, owned memos) ASYNC_GEN_PAIRS=E1,E2,E2e,E3,E6,E7 runs each scenario a second time rewritten without changing its meaning and compares the screens at every settle point (after the first render, each task step and the end); differences are grouped by what each side shows. Scenarios also choose whether memos are created at module level or inside the root component. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0117BoWKJDzwaZPgaGWE181b
1,500 generated scenarios (seed 3289, ~15 s) run with the web suite; each case's safety and liveness laws are compared to baseline.json. A case that newly breaks a law fails the suite (a change broke a neighbouring shape); a recorded failure that no longer reproduces is reported, and ASYNC_GEN_UPDATE=1 records it. The baseline holds today's 20 failing cases (#4362-#4367 shapes). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0117BoWKJDzwaZPgaGWE181b
|
Size (brotli, eager entry chunk)
|
Coverage Report for CI Build 40734544338Warning Build has drifted: This PR's base is out of sync with its target branch, so coverage data may include unrelated changes. Coverage remained the same at 78.651%Details
Uncovered ChangesNo uncovered changes found. Coverage RegressionsNo coverage regressions found. Coverage Stats
💛 - Coveralls |
Each generated scenario is emitted as a TSX module, rendered through renderToStream (server half), then hydrated chunk by chunk (client half). Oracles: no refetch of a read memo before the first write, no console.error or hydration warning, hydrated screen equals a fresh CSR render, then the scenario's steps and final liveness. Opt-in via ASYNC_GEN_HY=1 (pnpm async-gen:hydrate); minimize-hydration.mts reduces a failing case through both halves. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0117BoWKJDzwaZPgaGWE181b
ASYNC_GEN_RECORD on a reference build writes every case's screens, final screen and broken laws; ASYNC_GEN_DIFF on a candidate build replays the same seed and fails on any difference. Both the client campaign and the hydration campaign take it: the gate for the reader-seat carve. Same-build record/diff is clean (client 500 cases, hydration 300). minimize-hydration.mts loads minimize.ts through a computed specifier so tsc (no allowImportingTsExtensions) accepts it; runCase's return type names screens/final. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0117BoWKJDzwaZPgaGWE181b
This was referenced Oct 28, 2026
Merging this PR will improve performance by 8.72%2 warnings
Performance Changes
Tip Curious why performance improved? Comment Comparing 
Footnotes
|
This was referenced Oct 28, 2026
E8 re-creates the tree's readers in the flush that applies each write and compares screens where nothing is in flight; E9 reads each latest(x) the tree makes through a memo of it. Both from the carve audit's seat pairs. First run (1,000 cases): #4377, #4378. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0117BoWKJDzwaZPgaGWE181b
Member
Author
|
CodSpeed Performance Analysis is red on two commits, with a different benchmark each time:
These failures are not caused by this PR. It only adds files under Generated by Claude Code |
This was referenced Oct 28, 2026
ryansolid
pushed a commit
that referenced
this pull request
Oct 30, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Requested by Ryan · project thread
Summary
This adds a generative test harness for async edge cases in
packages/web/test/async-gen. It is test code only, so client bytes are unchanged.It generates small apps: 1–2 signals; 0–3 sync or async memos whose dependencies are read plainly or through
latest(), each async memo settled by hand by the script; and trees up to 3 deep ofLoading,Errored,Showand leaf spans. Each leaf has text holes and attributes readingx(),latest(x)orisPending(x), with grouped or split effects. Memos are created either at module level or inside the root component. Each app runs under scripts of writes, actions, settling flights in any order, and flush, microtask or task steps.The oracles need no per-case ruling:
console.error(a halt or the loop guard), no unhandled rejection, and no hole showsundefinedorNaN.ASYNC_GEN_PAIRS): grouped vs split effects (E1), reads through an identity memo (E2), an extra inert rootErrored(E2e), reads through a derived store (E3), flush after every step (E6, final screen only), and owned vs module-level memos (E7). Screens are compared at settle points.Modes:
surveytallies failures by signature,shrinkreduces a case with fast-check, andminimizeapplies a structural minimizer that also reduces the app.Regression gate:
regression.spec.tsruns 1,500 fixed scenarios (seed 3289, about 15 s) with the web suite and compares each case's laws tobaseline.json. A case that newly breaks a law fails the suite, which catches fixes that regress a neighbouring shape (as #4280→#4305 and #4307→#4342 did). A recorded failure that stops reproducing is only reported;ASYNC_GEN_UPDATE=1records it.The first campaigns found and minimized #4362, #4363, #4364, #4365, #4366 and #4367. The baseline currently records 20 failing cases from those shapes.
How did you test this change?
npx vitest run test/async-genpasses: the smoke test plus the regression gate. The gate gives the same result on 3 repeated runs.ASYNC_GEN=1across seeds 3289, 91501 and 7. Each filed issue was then reproduced by hand outside the harness.tsc -p tsconfig.test.jsonis clean.Generated by Claude Code