Skip to content

Web UI smoke: refresh the four screenshot baselines (#5929) - #5930

Merged
erikdarlingdata merged 3 commits into
devfrom
fix/5929-ui-smoke-baselines
Oct 16, 2026
Merged

erikdarlingdata merged 3 commits into
devfrom
fix/5929-ui-smoke-baselines

Conversation

@erikdarlingdata

@erikdarlingdata erikdarlingdata commented Oct 16, 2026 •

Copy link
Copy Markdown
Owner

Claude posting for Erik Darling.

Fixes #5929.

Cause

Two intended UI changes landed on dev after the baselines were taken in #5765. Neither PR touched the baselines. The Darling UI smoke allows no changed pixels, so every PR went red.

  1. Investigation requests: a viewer panel for open, claimed and reported requests (#5904) #5907 (fabbc64, "Investigations: a viewer panel", merged 05:40Z on 2026-10-16) added <a data-route="investigations" href="#/investigations">Investigations</a> to the sidebar in wwwroot/index.html. The sidebar has one more row, so everything under "Alert History" moves down 32 px: the nav list, the SERVERS block, the VIEWS block and the footer. That region is shared by all four pages, which is why the two ui-sql pages show equal counts (5494). This is the change behind the first red at 06:11Z.
  2. Investigation notes: approve or reject pending agent notes in the viewer (#5903) #5908 (6c26226, "Investigation notes: approve or reject pending agent notes", merged 09:47Z) added a "Pending Notes" tab to the Admin page in admin.js. It changes only admin-servers. It landed after the first red, but it is on dev now and would have kept the compare red.

Intended change, not a regression

I read the baseline, actual and diff PNGs from run 38883569424 (#5926). The diffs show the sidebar rows shifted below "Alert History" and nothing in #main (the fleet cards, the ui-sql panels and the Admin grid are unchanged). The new sidebar link and the new Admin tab render correctly. No code changes: baselines only.

How the baselines were produced

On the CI runner, by the documented procedure in tools/ui-smoke/README.md: gh workflow run darling-ui-smoke.yml --ref fix/5929-ui-smoke-baselines -f update_baselines=true on this branch (dev plus nothing else). Run 38885604418, artifact darling-ui-smoke-new-baselines. The four PNGs were copied unchanged into tools/ui-smoke/baselines/. Nothing was rendered on a laptop.

I viewed the new PNGs. Against the old baselines, the only differences are the "Investigations" row in the sidebar (and the rows below it moved down) and, on admin-servers, the "Pending Notes" tab. The new fleet.png is byte-identical to the fleet image #5926's run produced on its own, a second check that it carries no extra change. #5926's other three images differ from these because that PR adds its own Admin tab, so they were not used.

Verification

  • Run 38886286663 (first render, workflow_dispatch on head f6d9496, update_baselines false, so a real compare): success. The log shows ok shot fleet: 0 pixels differ (0.000%) and ok shot ui-sql-overview: 0 pixels differ (0.000%); the job passed, so the other two pages passed too.
  • The workflow skips draft pull requests, so the PR itself will not trigger the smoke until it is ready. The dispatch run above is the proof on this exact head.

Re-render after #5926

#5926 (67835f0) landed on dev while this was open and adds a "Report Feedback" tab to the Admin page. The baselines were rendered again on the CI runner: update_baselines=true dispatch, run 38887061667. Against the first render, the only changes are the Report Feedback tab on admin-servers (pixel bbox 578,69 to 765,80, the tab strip) and 58 scattered pixels on ui-sql-overview (the same bytes #5926's own run rendered, so a stable render of current dev, not noise). fleet and ui-sql-queries are byte-identical to the first render. The baselines now match dev 67835f0 including #5926's tab. Compare run 38887741790 on head 48ebd90 (update_baselines false): success, 0 pixels differ on all four pages.

Not changed

No admin.js, no README, no tests, no CHANGELOG. This is a baseline refresh only.

CHANGELOG entry

SECTION: Fixed
ENTRY: - The Darling web UI smoke check passes again ([#5930]) - The four screenshot baselines were out of date after the Investigations sidebar link (#5907) and the Admin "Pending Notes" tab (#5908), so every pull request failed the check. The baselines are refreshed from a CI render.
REF: [#5930]: #5930

@erikdarlingdata
erikdarlingdata marked this pull request as ready for review October 16, 2026 10:35

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: LGTM

Claims

  • VERIFIED — "No code changes: baselines only" — gh pr diff 5930 --name-only lists exactly the 4 PNGs under tools/ui-smoke/baselines/; no .js, .json, .ps1, README or test file is touched.
  • VERIFIED — #5907 added an "Investigations" sidebar link right after "Alert History", pushing the rows below it down — wwwroot/index.html:39 (<a data-route="investigations" ...>Investigations</a> sits between the alerts and alert-rules anchors); the new fleet.png baseline shows that exact row between "Alert History" and "Alert Rules" in the sidebar.
  • VERIFIED — #5908 added a "Pending Notes" admin tab, and dev now also carries #5926's "Report Feedback" tab — admin.js:659-665 TABS array lists Servers, Notification Routes, Alert Settings, Report Feedback, Pending Notes in that order; the new admin-servers.png baseline shows exactly those five tabs in that order, "Servers" active, with a 2-row grid (matches pages.json's exactRows: 2).
  • VERIFIED — the four PNGs are well-formed, 1280×800, 8-bit RGB, non-interlaced — file/header check on all four; matches shots.mjs's viewport: {width:1280, height:800}. Masked regions (e.g. the status bar items, .meta on fleet) render as the expected magenta (#ff00ff) boxes from shots.mjs's maskColor, consistent with a genuine Playwright-masked render rather than a hand-edited image.
  • UNVERIFIED — "produced ... on the CI runner ... by the documented procedure" / "Nothing was rendered on a laptop" (workflow dispatch runs 38885604418 and 38887061667) — a read-only tree has no way to inspect a GitHub Actions run or its artifact, and this session's allowed gh commands don't include run/artifact inspection.
  • UNVERIFIED — "Compare run 38887741790 on head 48ebd90 ...: success, 0 pixels differ on all four pages" — same restriction; would need gh run view/the Actions UI, not an allowed command here.
  • UNVERIFIED — "The new fleet.png is byte-identical to the fleet image #5926's run produced on its own" — would require the other PR's artifact bytes, not available from this tree.

Riskiest lines

  1. tools/ui-smoke/baselines/admin-servers.png (whole file) — a baseline is pure raster data; nothing in the repo can statically prove it came from the pinned CI Chromium rather than a local render that would drift the exact-match (MAX_DIFF_PIXELS = 0) compare on every future PR — pinned by: none (no test asserts PNG provenance); the real check is the workflow's own compare step on the next run — UNPINNED from this tree.
  2. tools/ui-smoke/baselines/fleet.png (whole file) — the fleet page's "collection stale" banner is a timing-sensitive state (shots.mjs's STALE_WAIT_MS wait); a baseline captured mid-transition would be unstable long-term, though that wait logic itself is unchanged here — pinned by: the ok fleet is collection stale ... wait in shots.mjs, not a named test — UNPINNED by a test, but by design intent.
  3. tools/ui-smoke/baselines/ui-sql-overview.png (whole file) — pages.json masks nothing extra on this page beyond the shared defaults, so any future text that varies run-to-run (e.g. a new "last collect" style string) would need its own mask before a baseline refresh, same as this one — not a defect here, just the sharpest edge the next refresh will hit — UNPINNED (no test; relies on the compare job itself).

Traps near this diff

  • ".gitattributes says text=auto eol=crlf: ... A diff that touches every line of a file is a line-ending conversion until proven otherwise" — does not apply: .gitattributes:1,… marks *.png binary and specifically tools/ui-smoke/baselines/*.png binary, so these are binary diffs (Binary files ... differ), not text/eol churn.
  • "Lite and Darling ship in parity ... a fix to one SKU's reader, tool, alert or description without the counterpart's is a finding" — does not apply: this is Darling-only UI-smoke test tooling (tools/ui-smoke/) with no Lite web UI or reader/tool/alert counterpart; tools/ui-smoke-wpf (the closest Lite-side tooling) has no baseline-image concept to keep in parity.

Divergence (blind read vs PR body)

None — the body's account matches the diff and the rendered images on every claim gradable from a read-only tree: the diff is 4 PNGs only, the sidebar and Admin-tab changes it attributes to #5907/#5908/#5926 are visible in the images and match the current index.html/admin.js source, and the body is explicit (rather than silent) about the parts only a live CI run could prove.

Reviewer checklist

  • Confirm the latest "Darling web UI smoke" run on this PR's head (or the dispatch runs 38886286663 / 38887741790 cited in the body) actually reports 0 pixels differing on all four pages, since a read-only tree can't execute or inspect that workflow run.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant