Repository navigation
Web UI check: screenshot baselines for four Darling pages (#5744) - #5765
Conversation
…er collection stale (#5744)
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
| - name: Cache Playwright Chromium | ||
| uses: actions/cache@v6 | ||
| with: | ||
| path: ${{ env.LOCALAPPDATA }}\ms-playwright |
There was a problem hiding this comment.
The Chromium cache is inert: env.LOCALAPPDATA is empty here.
The env expression context is populated only from env: maps declared at the workflow, job or step level (plus anything a prior step wrote to $GITHUB_ENV). It is not the runner process's environment — the runner's own OS variables (LOCALAPPDATA, USERPROFILE, TEMP) and the default GITHUB_*/RUNNER_* variables are reachable from a run: shell ($env:LOCALAPPDATA) and, for the defaults, from the github/runner contexts, never from env.. This workflow declares no env: at workflow or job level, so ${{ env.LOCALAPPDATA }} renders as the empty string and the step asks to cache \ms-playwright.
Consequence: every run is a cache miss, nothing is ever saved (the post step reports Path Validation Error: Path(s) specified in the action for caching do(es) not exist), and npx --no-install playwright-core install chromium at run-ui-smoke.ps1:124 re-downloads Chromium on every PR run. The step's own comment — "pinned by the playwright-core version in package-lock.json and cached by it" — and the PR body's "Playwright Chromium cache" bullet are both describing something that does not happen.
Two ways out. Either name the path without the expression:
| path: ${{ env.LOCALAPPDATA }}\ms-playwright | |
| path: ~\AppData\Local\ms-playwright |
or, better, stop depending on Playwright's default location at all — put PLAYWRIGHT_BROWSERS_PATH: ${{ github.workspace }}\.playwright-browsers in a job-level env: block and point both this step's path and the install at it, which also makes the cached location explicit rather than inherited from Playwright's platform defaults.
Worth confirming on the next run: the step log should say Cache restored from key: playwright-chromium-Windows-<hash> rather than Cache not found for input keys.
There was a problem hiding this comment.
Claude posting for Erik Darling. Fixed in c09be6b: PLAYWRIGHT_BROWSERS_PATH is now a job-level env (github.workspace.playwright-browsers), and the cache step and the install both use it. The run on 7172cc1 shows 'Cache restored from key: playwright-chromium-Windows-...' with no Path Validation Error.
|
|
||
| writeFileSync(join(outDir, "report.json"), JSON.stringify(results, null, 2)); | ||
| /* Screenshot baselines (#5744): the pages marked "screenshot" in pages.json, in Playwright's own pinned Chromium. */ | ||
| const shotProblems = await runShots({ config, baseUrl, outDir, only, settle }); |
There was a problem hiding this comment.
A throw out of the screenshot stage now destroys the walk's report.json.
Before this change report.json was written on the line immediately after browser.close(), so the 35-page result survived anything that happened afterwards. It is now written after await runShots(...), and runShots has unguarded work in front of its own try: chromium.launch() (shots.mjs:599) and the mkdirSync pair (shots.mjs:572-573). None of that is wrapped, so a Chromium that was not installed (the cache/install step at run-ui-smoke.ps1:122-126 failing, or a -NoShots local run where the operator set UI_SMOKE_SHOTS=compare by hand) rejects out of runShots, kills the process on an unhandled rejection, and takes the in-memory page results with it.
The failure is quiet in exactly the wrong way: the job still goes red, but ${{ runner.temp }}\ui-smoke\out\* uploads without report.json, so the artifact says nothing about which of the 35 pages were clean — and the panel dump is the thing you actually read when the smoke check fails.
Writing the pages half first costs one line and keeps the two stages independent:
| const shotProblems = await runShots({ config, baseUrl, outDir, only, settle }); | |
| writeFileSync(join(outDir, "report.json"), JSON.stringify({ pages: results, screenshotProblems: [] }, null, 2)); | |
| /* Screenshot baselines (#5744): the pages marked "screenshot" in pages.json, in Playwright's own pinned Chromium. */ | |
| const shotProblems = await runShots({ config, baseUrl, outDir, only, settle }); |
(then let the existing writeFileSync below rewrite it with the real shotProblems). Catching inside runShots and returning the message as a problem string would work too, and would match how every other failure in this file is reported.
There was a problem hiding this comment.
Claude posting for Erik Darling. Fixed in c09be6b: report.json is written with the walk's pages before the shots start, runShots is wrapped so a throw becomes a screenshot problem that still fails the run, and the file is rewritten with the real screenshotProblems.
| const staleDeadline = Date.now() + 12 * 60 * 1000; | ||
| for (;;) { | ||
| let allStale = false; | ||
| try { | ||
| const fleet = await (await fetch(baseUrl + "/api/fleet")).json(); | ||
| const cards = fleet.cards || []; | ||
| allStale = cards.length > 0 && cards.every((c) => c.collection_stale === true); | ||
| } catch { | ||
| /* the service answers again on the next poll */ | ||
| } | ||
| if (allStale) break; | ||
| if (Date.now() > staleDeadline) return ["screenshots: the fleet never reported every server as collection stale in 12 minutes"]; | ||
| await new Promise((r) => setTimeout(r, 5000)); | ||
| } |
There was a problem hiding this comment.
The stale gate waits for a state that expires, and has no exit for the far side of it.
collection_stale is true only inside a band, not after a threshold. ServerCollectionStatusRules.FlagsFor (PerformanceMonitor.Common/ServerHealthBands.cs:150-157) sets CollectionStale on exactly one arm, ServerFreshness.Stale, and that band runs from StaleThreshold = 2 × the 1-minute collector cadence (ServerHealthBands.cs:257) to OfflineThreshold = 30 minutes (ServerHealthBands.cs:281). Past 30 minutes from the seed anchor the cards band Offline, collection_stale goes back to false, and it never becomes true again for the life of the run.
So this loop has two outcomes, not one. Inside the band it breaks in seconds — the normal case today, since the 35-page walk already costs 2-3 minutes and the band opens at 2. Past the band it can never succeed, and it does not notice: it spends its full 12 minutes polling /api/fleet every 5 s and then reports the fleet never reported every server as collection stale in 12 minutes, which names a symptom an operator will read as "the service stopped answering" when the real cause is "the walk ran past the half-hour Offline threshold". Added to the 12 minutes it just burned, the job is also past timeout-minutes: 20 (darling-ui-smoke.yml:48) — and a timed-out job is cancelled, not failed, so the if: failure() diff-artifact step at darling-ui-smoke.yml:163 does not run either.
Cheap to make self-explaining: the same /api/fleet body already carries the discriminant, so an arm on cards.every((c) => c.is_online === false) can break out early and return a problem that says the seed anchor aged past the 30-minute Offline threshold. Worth doing even though it is not reachable at today's walk length — the whole value of this gate is that it tells you which clock went wrong.
There was a problem hiding this comment.
Claude posting for Erik Darling. Fixed in c09be6b: the wait now ends after 4 minutes (stale arrived after 20 to 30 seconds of waiting in CI) and has two distinct failures, 'already past collection stale' when any card is Offline (is_online false) and 'never reached collection stale' on the deadline, each naming its cause.
Notes from the blind read (non-blocking)Four things I worked out reading the diff before the body, none of them blocking, all of them worth a sentence somewhere so the next person does not have to re-derive them. 1. The as_of injection at
Worth one line in the README's "How the page is kept the same on every run" list: the Fleet page's own window is the service's clock, not the anchor; its numbers are chosen to be flat across it. 2. Masking hides glyphs, not geometry.
3. A baseline refresh is an expensive commit.
4. The baselines can go red on a pull request that did not touch them. This workflow triggers on Also: the job header comment still reads "Expected 8 to 10 minutes warm" ( |
There was a problem hiding this comment.
Verdict: CHANGES REQUESTED
Claims
- VERIFIED — Four pages compared with baseline PNGs (Fleet, ui-sql Overview, ui-sql Queries, Web Admin servers) —
pages.json:20,27,85,100carry thescreenshotkey andbaselines/holds exactly those four, each 1280x800 and visibly the named page. - VERIFIED — Screenshot options, 8-try byte-identical stability, pixelmatch threshold 0.1, zero-pixel limit —
shots.mjs:27-29,stableShotat:66-83,pixelmatch(...)at:196; no@playwright/testinpackage.json, as claimed. - VERIFIED —
screenshotDefaults.maskcovers the status bar stamp, database size, refresh hint and chart tooltips, and Fleet masks its own.metastamp —pages.json:9,20againstapp.js:427,437,438,charts.js:388, andfleet.js:489(the only.metaon that page,"Updated " + localTime(d.generated_at)). - VERIFIED —
pixelmatch7.1.0 /pngjs7.0.0, exact —package.json:10-12and the matching lockfile entries. - VERIFIED —
run-ui-smoke.ps1takes the anchor, passes-v anchor=andUI_SMOKE_ANCHOR, installs Chromium, adds the two switches —run-ui-smoke.ps1:104-106,120-126,42-43. - VERIFIED — With no caller anchor the seed falls back to the store clock, so
UiSmokeSeedAppliesLiveTestsand the WPF smoke are unchanged —seed.sql:27-31;UiSmokeSeedAppliesLiveTests.cs:75-77passes no-v anchor, andwpf-smoke.yml:157uses-SeedOnly, which returns atrun-ui-smoke.ps1:108-111before any new code. - VERIFIED — The shots use Playwright's own Chromium while the walk keeps the runner's chrome channel —
shots.mjs:161has no channel,smoke.mjs:194does. - VERIFIED — A fixed
as_ofcould not ride the URL (the range is module state,util.js:1077), and the injected anchor is one the service accepts —McpHelpers.AsOfFormatsincludesyyyy-MM-ddTHH:mm:ss.FFFFFFFK, whichtoISOString()satisfies, andResolveAsOfrefuses only a future anchor past five minutes. - UNVERIFIED — Two independent update runs gave byte-identical PNGs, and
ui-sql-queriesdiffered by 35 pixels before the stale-wait fix — live CI runs; nothing in a read-only tree records them. - UNVERIFIED — Compare runs 38428140107, 38430535486 and 38426516589 green, twelve page shots at 0 pixels — same.
- UNVERIFIED — The red proof (38426539677, 38428143988, 38429177928, 38431713958: 238 pixels passing at 1% and failing at zero) — same; the throwaway branches are deleted.
- UNVERIFIED — The baselines were rendered by the CI image only (run 38425601813) and include #5705 — the PNGs are consistent with the seed, but a tree cannot say which image rendered them.
- UNVERIFIED — "node --check and a pure-JS check of the timestamp shift only" — this environment would not execute node either, so the shift was reviewed by reading: offset handling at
shots.mjs:83-86is correct both ways, and every emitted field is fixed-width, so response lengths are preserved. - REFUTED — "Workflow: ... Playwright Chromium cache" — the step caches nothing.
${{ env.LOCALAPPDATA }}resolves against theenvexpression context, which holds only variables declared in a workflow/job/stepenv:map; this workflow declares none, sodarling-ui-smoke.yml:117asks to cache\ms-playwrightand Chromium is re-downloaded on every run. Inline there, with two fixes.
Riskiest lines
.github/workflows/darling-ui-smoke.yml:117— the one step meant to stop a per-run Chromium download is inert, and fails as a post-step warning — UNPINNED (CiClusterWorkerSizingTestsparses only this workflow's clusterAdd-Contentlines;DarlingPathFilterGateTestsonly its filter patterns).tools/ui-smoke/shots.mjs:144-157— the gate waits forcollection_stale, a band of 2 to 30 minutes (ServerHealthBands.cs:257,281), with no exit for the closed side; past it the step burns 12 minutes inside a 20-minute job and names the wrong cause — UNPINNED.tools/ui-smoke/smoke.mjs:220—report.jsonnow writes afterrunShots, whosechromium.launch()(shots.mjs:161) andmkdirSyncpair (:134-135) sit outside its owntry, so a screenshot-stage throw discards the 35-page walk result — UNPINNED.
Traps near this diff
- "dorny/paths-filter v4 evaluates each pattern as an INDEPENDENT predicate ... Keep the carve-out INSIDE the include:
dir/**/!(*.md)." — does not apply — this diff adds no filter patterns; the gate already uses the compliant form at.github/darling-paths-filter.yml:65. - "
.gitattributessaystext=auto eol=crlf... A diff that touches every line of a file is a line-ending conversion until proven otherwise." — does not apply — nothing is rewritten wholesale;package.json's three-lineenginesreflow is a formatter, and the PNGs are declaredbinaryin the same commit at.gitattributes:36. - "Lite and Darling ship in parity ... a fix to one SKU's reader, tool, alert or description without the counterpart's is a finding, not a nit." — does not apply —
tools/ui-smoke/README.md:3scopes the harness to the Darling web pages and Web Admin; Lite ships no web dashboard, and nothing in the diff touches a shared reader or tool. - "
claude-code-actionrefuses to run when.github/workflows/claude-review.ymldiffers from the DEFAULT branch's copy." — does not apply — the only workflow in the diff isdarling-ui-smoke.yml. - "
hours_back,limit,top,days_backare refused when out of range, never clamped." — does not apply —shots.mjs:100adds anas_ofthe page never sent rather than narrowing one it did, and the service validates it through the sameResolveAsOfas any caller. - "A test that reads the clock twice measures the CI scheduler ... One anchor owns both ends of an interval." — applies in shape, and this diff is the fix direction (
seed.sql:27-31,run-ui-smoke.ps1:104). One second clock survives:/api/fleettakes noas_ofand windows onDateTime.UtcNow(DarlingWebEndpoints.cs:363). I checked every number the Fleet baseline renders against it and all are flat for the hour after the anchor, so it is not a finding today — see the top-level note for why the pg deadlock delta is flat only because of where integer division lands.
Divergence (blind read vs PR body)
Three places.
The cache bullet is the refuted claim above: the body and the step's own comment both describe a cache that does not exist.
The stale wait is described as a one-way fix. "The step now waits until /api/fleet reports every card collection_stale" reads as a threshold crossed once; it is a band that closes again at 30 minutes, and the loop has no arm for the closed side, so its failure message names the wrong cause and arrives after the job's own timeout.
One change the body does not mention. report.json changed shape ({ pages, screenshotProblems } rather than a bare array) and moved behind the screenshot stage. The shape change is safe — nothing outside smoke.mjs reads the file — but the move means a screenshot-stage throw takes the walk's result with it.
Everything else matched, including the parts I expected to argue with: the as_of injection is keyed on hours, which is what readTool actually sends rather than the tools' own hours_back; the format the service accepts; and every mask selector resolving against a real element in wwwroot.
Reviewer checklist
- On the next run, read the "Cache Playwright Chromium" step log: it should say
Cache restored from key: playwright-chromium-Windows-<hash>and the post-job step should not warnPath Validation Error. I expect both to show the opposite today. - Time a full green run against
timeout-minutes: 20now that the Chromium download, the stale wait and four renders are in it; the header comment still claims 8 to 10 minutes warm. - Check the throwaway store's
pg_database_sizeis comfortably inside one digit count — thesb-store-sizemask hides the glyphs but not the element width, so a 99 to 100 MB crossing is a hard red at zero pixels. - Re-run
UiSmokeSeedAppliesLiveTestsagainst a live store: it is the only exercise ofseed.sql's new\if :{?anchor}fallback branch, and nothing here could run psql.
…efore shots, fixed-width store size (#5744)
There was a problem hiding this comment.
Verdict: LGTM
Claims
- VERIFIED — Chromium cache now works:
PLAYWRIGHT_BROWSERS_PATHis set at job level (darling-ui-smoke.yml:51-52) and the cache step reads${{ env.PLAYWRIGHT_BROWSERS_PATH }}(:122) — a job-levelenv:map is exactly what theenvexpression context sees, unlike the priorenv.LOCALAPPDATA(refuted in the last review).run-ui-smoke.ps1'splaywright-core install chromium(:124) andshots.mjs'schromium.launch()(:185) both inherit the job env and honour this var by Playwright convention. - VERIFIED — The stale wait now has two distinct exits instead of one:
offline > 0returns a "ran past the band" message, the deadline returns a "never got there" message, each naming its own cause (shots.mjs:171-179), fixing the prior review's finding that the loop "has no arm for the closed side." - VERIFIED —
STALE_WAIT_MSdropped from the old 12 minutes to 4 (shots.mjs:31), consistent withServerHealthThresholds.StaleThreshold= 2 min andOfflineThreshold= 30 min (PerformanceMonitor.Common/ServerHealthBands.cs:258,282) — the window the comment describes. - VERIFIED —
report.jsonis now written immediately after the page walk withscreenshotProblems: [], thenrunShotsis wrapped in its own try/catch at the call site (smoke.mjs:219-229), so a throw out of the screenshot stage (e.g.chromium.launch()or themkdirSyncpair insiderunShots, previously outside any try) can no longer discard the walk's result — fixes the third riskiest-line finding from the prior review. - VERIFIED — The status-bar width fix:
#statusbarisdisplay: flex(app.css:945-950), and the injected style (flex: none; width: 14em; overflow: hidden,shots.mjs:34) pins the flex item's width regardless of text length, addressing the non-blocking "masking hides glyphs, not geometry" note from the prior review's.sb-store-sizeobservation. - VERIFIED —
bypassCSP: true(shots.mjs:98) is necessary foraddStyleTagto work: the service's CSP isstyle-src 'self'with nounsafe-inlinefor style blocks (DarlingWebSecurityHeaders.cs:65), and the hash-based allowance only covers styles present at render time (:137), not a tag injected afterward by Playwright. This context registers no console/CSP listeners, so bypassing CSP here doesn't mask anything the smoke walk (a separate context insmoke.mjs) checks.
Riskiest lines
tools/ui-smoke/shots.mjs:167-179— the fleet-stale polling loop's two exit conditions are the fix for last review's "closed side" gap — pinned by manual read againstServerHealthBands.cs's band constants; no automated test exercises this loop (it needs a live service) | UNPINNED.github/workflows/darling-ui-smoke.yml:51-52,122— job-levelenv:actually resolving in theenvcontext for both the cache step and the install step — UNPINNED by any repo test (same class of gap the previous review's finding came from; only a live run proves the cache hits)tools/ui-smoke/smoke.mjs:219-229—report.json's two-write sequence and therunShotstry/catch — UNPINNED; no test exercisessmoke.mjs/shots.mjs(JS, outside both C# test projects)
Traps near this diff
- "dorny/paths-filter v4 evaluates each pattern as an INDEPENDENT predicate ... Keep the carve-out INSIDE the include." — does not apply — no filter pattern is touched in this range.
- "Lite and Darling ship in parity ... a fix to one SKU's reader, tool, alert or description without the counterpart's is a finding." — does not apply — this is Darling-only CI tooling; Lite ships no web dashboard to keep in parity with.
- "
claude-code-actionrefuses to run when.github/workflows/claude-review.ymldiffers from the DEFAULT branch's copy." — does not apply — the only workflow touched isdarling-ui-smoke.yml. - "A test that reads the clock twice measures the CI scheduler: ... One anchor owns both ends of an interval." — does not apply here — the stale-wait loop reads
/api/fleetrepeatedly but measures against a singlestaleStarted/staleDeadlinepair taken once (shots.mjs:157-158), not two independent clock reads.
Divergence (blind read vs PR body)
None — this follow-up range directly implements the three blocking fixes and the one non-blocking note the prior review raised (cache path, two-sided stale wait, report.json ordering, store-size width), and the diff matches the HEAD commit's own title exactly. Nothing in the new commits contradicts it.
Reviewer checklist
- On the next CI run, confirm the "Cache Playwright Chromium" step actually reports a hit on a second run (
Cache restored from key: playwright-chromium-...) and that the post-job save step does not warn. - Confirm a real run's stale wait settles in roughly 0-2 minutes as predicted, not near the new 4-minute deadline.
- Visually confirm the fixed-width
.sb-store-sizebox in a rendered screenshot still reads cleanly (no visible clipping of "Database: NN MB").
Claude posting for Erik Darling. Superseded: the follow-up review at c09be6b is LGTM, and every round-1 finding has a fix commit.
1 participant
Claude posting for Erik Darling.
Fixes #5744
SECTION: None
ENTRY: - None: CI-only screenshot baselines for four Darling web pages (#5701 phase 2); nothing ships.
What changed
Phase 2 of #5701. The web UI check now also compares four pages with baseline PNGs: Fleet, the
ui-sqlOverview,ui-sqlQueries, and Web Admin servers. All undertools/ui-smoke/, plus.github/workflows/darling-ui-smoke.yml.shots.mjs(new):page.screenshot({ mask, animations: 'disabled', caret: 'hide', scale: 'css' }), retaken until two shots in a row are byte-identical (up to 8 tries, then a clear failure), compared withpixelmatch(threshold 0.1) andpngjs, limit zero differing pixels (MAX_DIFF_PIXELS = 0, Playwright's owntoHaveScreenshotdefault; the failure message still prints the ratio).playwright-corehas notoHaveScreenshot(that is the test runner's), so I did not add the runner.pages.json: ascreenshotkey marks each page, with its own mask selectors.screenshotDefaults.maskcovers the status bar "Updated" stamp, the database size, the refresh hint and chart tooltips. The Fleet page also masks its.meta"Updated" stamp (it carries the real server seconds).package.json/package-lock.json:pixelmatch7.1.0 andpngjs7.0.0, exact.run-ui-smoke.ps1: takes the seed anchor itself, passes it toseed.sql(-v anchor=) and to node, installs Playwright's Chromium,-UpdateBaselines/-NoShots.seed.sql: every time now hangs off one anchor (the psql variableanchor; if the caller sets none it is taken from the store clock as before, soUiSmokeSeedAppliesLiveTestsand the WPF smoke are unchanged).workflow_dispatchbooleanupdate_baselines; Playwright Chromium cache (job-levelPLAYWRIGHT_BROWSERS_PATH=${{ github.workspace }}\.playwright-browsers, used by the cache step, the install andchromium.launch(); round 1 replaced a${{ env.LOCALAPPDATA }}path that was empty and cached nothing); artifactdarling-ui-smoke-new-baselines(update run) anddarling-ui-smoke-screenshot-diffs(a failed compare: baseline, actual and diff for each page that differs)..gitattributes:tools/ui-smoke/baselines/*.png binary. README: "Screenshot baselines" section.The browser is Playwright's own Chromium, pinned by the
playwright-core1.64.0 version. Phase 1 launched the runner'schromechannel, which updates with the image; that is still used for the smoke walk, but not for the screenshots.Determinism measures
document.fonts.readyawaited.as_ofcould not go on the page URL: the page's range is module state, not a URL parameter. And a seed dated at a fixed past instant would rot (as_ofis refused past the read's reach), so the service keeps its real clock and the BROWSER is moved. Every/apiJSON answer has its ISO timestamps (within 60 days of the seed anchor) shifted to a fixed date (2026-03-15 12:00 UTC). Every windowed/api/read/*request without anas_ofgets one (seed anchor + 1 minute) throughcontext.route.page.clock.setFixedTimepins the page clock to the fixed date + 1 minute. So relative times ("1m ago", "last collect 12:00 PM") and chart axes are the same on every run./api/fleetreports every cardcollection_stalebefore shooting. "Stale" is a band (2 to 30 minutes after the last collection), so the wait has a 4 minute deadline and two distinct failures: "already past collection stale" (a card is Offline,is_onlinefalse) and "never reached collection stale". Measured in CI: stale arrived after 0 to 30 seconds of waiting.ui-sql-overviewandadmin-servers.ui-sql-queriesdiffered by 35 pixels in a panel border (0.003%; that was before the stale-wait fix in e974241, and the compares since show 0 pixels on every page).State of the baselines
The baselines are FINAL and include #5705 (already on
dev, 3f3b9c1). They were rendered by the CI image only (update run 38425601813), reviewed by eye (masks cover the status bar stamps and the Fleet stamp, no spinner or half-loaded panel, seeded data: ui-sql 25% CPU, 25.2 GB, SQL Server 2022 / 16.0.4135.4, two servers).Compare runs on the first baselines commit dc05eda: 38426516589 attempts 1 and 2 green. Compare runs on the final head e974241 (with the wait for the stale state): 38428140107 attempts 1 and 2 (
gh run rerun) both green.Red proof
fix/5744-redproof(label "Needs attention" changed): red, uploadeddarling-ui-smoke-screenshot-diffswith baseline, actual and diff for each failing page. It also failed Overview and Queries, which exposed the Online / stale timing above (fixed in e974241).fix/5744-redproof2(same one-label change, zero-pixel limit): RED,fleet238 pixels (0.023%) differ, the other three pages 0 pixels. The artifactdarling-ui-smoke-screenshot-diffsheldfleet.baseline.png,fleet.actual.pngandfleet.diff.png. The branch is deleted.app.css): RED,fleet1.92% differs, the job uploadeddarling-ui-smoke-screenshot-diffs(baseline, actual and diff). The other three pages stayed at 0 pixels.The throwaway branch is not part of this PR and is deleted after the proof.
Limit
Zero differing pixels after the 0.1 per-pixel color threshold. Measured: twelve page shots across dispatch runs 38428140107 (attempts 1 and 2) and 38428143988 showed 0 pixels differ on every page that was not edited, and compare run 38430535486 on this head showed 0 pixels on all four pages. A one-label Fleet change was 238 pixels (0.023%): it passed at 1% (run 38428143988) and fails at zero (run 38431713958).
How to refresh baselines
gh workflow run darling-ui-smoke.yml --ref <branch> -f update_baselines=true, download the artifactdarling-ui-smoke-new-baselines, commit the PNGs intotools/ui-smoke/baselines/, review GitHub's image diff. Never commit a PNG rendered on a laptop. README has the same steps.Round 1 fixes (review of eb07e98)
PLAYWRIGHT_BROWSERS_PATHis set at job level and used by the cache step, the install and the launch. Proof: the first run saved the cache ("Cache saved with key: playwright-chromium-Windows-72cbd548..."), and later runs print "Cache restored from key: playwright-chromium-Windows-72cbd548..." (dispatch 38436978173, pull request run 38437783077, dispatch 38439935446, pull request run 38439943731) with no "Path Validation Error" in the post step.{ pages, screenshotProblems }(it was a bare array before this PR; nothing outsidesmoke.mjsreads it). The order changed too: the walk's part is written first withscreenshotProblems: [], the screenshot stage runs inside atry(a throw becomes ascreenshots: the stage threw: ...problem and still fails the run), and the file is rewritten with the real problems.timeout-minutes: 20stays (3 times the slowest run).get_store_hostread had not finished), a 4719 pixel difference on all four pages. Final fix: the screenshot context adds a style (page.addStyleTag, withbypassCSPbecause the page's CSP isstyle-src 'self') that hides.sb-store-sizeand the separator before it (.sb-sep:has(+ .sb-store-size)), so the bar is the same with or without the item. The item left the default mask list. The baselines were re-rendered from anupdate_baselinesdispatch (run 38439329533), each PNG read by eye.pg-runtime.zip(stopped afterwards):UiSmokeSeedAppliesLiveTestspassed (1 of 1, not skipped; it passes no-v anchor, so it takes the\if :{?anchor}fallback). A second run with the test temporarily passing-v anchor=2026-10-13 17:00:00also passed (test change reverted, not committed). The\ifblock alone prints the passed anchor with-v anchor=and the store clock without it.Green on the final head 0b2b304: compare dispatch 38439935446 (0 pixels on all four pages) and Darling UI smoke pull request run 38439943731 (0 pixels on all four, cache restored).
Notes
workflow_dispatchruns (the pull request was a draft then). Since round 1 the pull request runs go too.node --checkand a pure-JS check of the timestamp shift only.