[bkudiess]

What Problem This Solves

Fixes: the "Mock Gateway application and compatibility tests" CI job fails intermittently on main and on unrelated PRs, so first-attempt runs go red and need reruns.

User Impact

User impact: there is no product change. Contributors get fewer false CI failures from five mock Gateway UI tests.

Why This Change Was Made

Each test checked the UI once, or waited for the wrong condition, while the UI was still changing. The tests now wait for a state they can observe, then make the same assertions:

• Automations, weekly editor: CI receipts show the editor reached the requested scroll position (49.4%), then something else moved it to 22.4% and it stayed there. The test only accepted "field visible" or "at the requested position", so it timed out. It now waits until the scroll position stops changing, measures again, and keeps the same final full-visibility check.
• Automations, empty agent (Dark): reading the whole UI tree while the list was re-rendering threw "Pattern not found". The test now asserts on the first complete read.
• Sessions gallery, toast: after a row near the bottom of a 650 px window expands, its Archive command can sit below the window edge (UIA dump: y=740–768, window bottom 676). The test now waits for the expansion to settle and scrolls the command into view before asserting.
• Usage pinned (and the same pattern in the pinned Automations host): the hub window was looked up once, before the chat composer was visible to UI Automation. The test now waits for it.
• New agent: the assistant selector was looked up once and could be null. The test now waits for an enabled selector.

The changes add no retries, sleeps, fixed settle counts, skips, or weaker assertions.

Not fixed in this PR:

• SelectedAssistantAndConversationSurviveTransientRosterLossButNotAuthoritativeRemoval. The draft is empty after the test sets it, and the cause is not established. This PR only adds a failure receipt (draft length right after the set, composer identity and count). The receipt cannot change the test result. A separate session owns the root cause.
• SidebarSwitchesRealHistoriesAndShowsMessage240AtBothWidths, which is left for a follow-up.
• SystemsDaily_InsetAndExpandedNativeFrames, a known flake that this PR does not touch.

Evidence

• Root causes from CI artifacts: editor-scroll receipts, UIA dumps, failure screenshots and stack traces from runs 43251801019, 43189282662, 43253198365, 43262896070 and 43306025916.
• Deterministic reproduction for the editor scroll: a local injection of the CI-observed external scroll move.
  • With the old wait, it reproduced the exact CI failure ("Timed out waiting for editor scroll reaches computed position") in both themes.
  • With the new wait, the same injection passes in both themes.
• Reproduction for the empty-agent read: the "Pattern not found" failure reproduced locally under load. A further case (COMException 0x8000FFFF from a live AutomationId read) was found and fixed during validation.
• Review: three dual-model Hanselman review rounds (Claude Opus 5.5 and GPT-6 Astra, high effort). All consensus findings are fixed; the open one-model LOW notes are listed under Real Behavior Proof.

Change Type

• Bug fix
• Feature
• Refactor
• Docs or instructions
• Tests or validation
• Security hardening
• Chore or infrastructure

Scope

• Tray or WinUI UX
• Windows node capability
• Local MCP or winnode
• Gateway, connection, or pairing
• Setup or onboarding
• Permissions, privacy, or security
• Tests, CI, or docs

Required proof pools

• none: test-only change to the mock Gateway fixture UI tests; no product behavior changed, and CI runs these tests directly.

Validation

All runs were local on Windows x64 at 175% DPI, with isolated profiles. Metadata for the real %APPDATA%/%LOCALAPPDATA%\OpenClawTray was unchanged after every run.

Repeat runs on the final test source (rebased onto 3d5f142):

Test or set	Result
Automations_ScopesToAgentsAndCoversMainStates (Light + Dark)	5/5 consecutive passes
SessionsDesignNativeWindowGallery, UsagePinned_ToggleOwnsHitPointAndTabOrderWithVisibleOperationFeedback, NewAgentUsesGatewayCreationAndRefreshesWithoutReplacingConversation	5/5 consecutive passes
SelectedAssistantAndConversationSurviveTransientRosterLossButNotAuthoritativeRemoval	5/5 consecutive passes (diagnostics only)

Full suites at the same source:

• Fixture profile and app tests: test-gateway-fixture.ps1 -Screenshots -NativeInputProof passed 55/55.
• Fixture UI tests, filter Category=GatewayFixture&KeyboardProof!=Required with the native-input opt-in: 124/124 passed (30m04s). These ran directly, because the script caps each project at 1800 s, and this run takes about 30 minutes locally at 175% DPI with screenshots.

After rebasing onto origin/main 64c96ac:

• No upstream changes under tests/OpenClaw.Tray.UITests, tests/OpenClaw.GatewayFixtureHost or tests/OpenClaw.TestSupport.
• The app and UI tests were rebuilt. The affected tests (Automations ×2 themes, Sessions gallery, UsagePinned, NewAgent, Continuity) passed 6/6.

Closeout on the final tree:

• ./build.ps1: all builds succeeded.
• dotnet test ./tests/OpenClaw.Shared.Tests/OpenClaw.Shared.Tests.csproj: 5847 passed, 0 failed, 29 skipped.
• dotnet test ./tests/OpenClaw.Tray.Tests/OpenClaw.Tray.Tests.csproj: 6347 passed, 0 failed.

Real Behavior Proof

• Environment tested: local Windows x64 desktop, 175% DPI, mock Gateway fixture, isolated app and test profiles.
• PR head or commit tested: 36093e2 (the same test source was validated before the rebase on 3d5f142, then rerun after the rebase on 64c96ac).
• Exact steps or command run: dotnet test tests/OpenClaw.Tray.UITests -c Debug -r win-x64 -p:DevBuild=false --filter <test>, with the fixture environment from scripts/test-gateway-fixture.ps1.
• Evidence after fix: the pass counts listed under Validation, plus the deterministic before/after for the editor-scroll failure.
• Observed result: all targeted tests passed in every final repeat run.
• Screenshot or artifact links verified? (Yes/No/N/A): N/A (local artifacts only).
• Not verified or blocked:
  • The first-attempt CI result for this PR is pending.
  • The Continuity root cause is open.
  • Sidebar240 is not addressed here.
  • Two local-only signatures appeared during the earlier repeat runs, both outside the changed code and never seen in the 347-run CI history:
    • An Axe BoundingRectangleSizeReasonable violation on a transient button ("snapshot peer unresolved", 175% DPI).
    • A foreground window stolen by another process on the shared desktop.
  • Open one-model LOW review notes, not applied:
    • The editor settle check does not include the viewport bounds.
    • The empty-agent timeout does not report the last HRESULT.
    • The New Agent test reuses the selector element after the agent list refreshes (pre-existing code).

Security Impact

• New permissions or capabilities? (Yes/No): No
• Secrets or tokens handling changed? (Yes/No): No
• New or changed network calls? (Yes/No): No
• Command or tool execution surface changed? (Yes/No): No
• Data access scope changed? (Yes/No): No
• If any answer is Yes, explain the risk and mitigation: N/A. The new failure receipt records lengths, counts and flags only, not message content.

Compatibility and Migration

• Backward compatible? (Yes/No): Yes
• Config or environment changes? (Yes/No): No
• Migration needed? (Yes/No): No
• If yes, list the exact upgrade steps: N/A

Review Conversations

• I replied to or resolved every bot review conversation addressed by this PR.
• I left unresolved only conversations that still need maintainer judgment.

[clawsweeper]

🦞👀
ClawSweeper picked this up.

Pull request received. I will update this pull request when review starts.

ClawSweeper review complete

ClawSweeper finished reviewing this revision. The review result is being finalized.

View the workflow run.

[clawsweeper]

Codex review: needs maintainer review before merge.

What this changes

This PR stabilizes mock Gateway UI tests with observable readiness waits and adds diagnostics for an unresolved composer timeout.

Example: The Automations editor reaches the requested 49.4% scroll position, then layout moves it to 22.4%.

• Before: The test waits for the obsolete target and times out.
• After: The test observes settled geometry, recalculates scrolling, and still requires the field to be fully visible.

Review scores

Measure	Result	What it means
Overall readiness	🦐 gold shrimp (3/6)	Focused, source-supported test repairs preserve strict outcomes; independent validation is limited to source and CI metadata, with local execution results reported in the PR.
Proof confidence	🦐 gold shrimp (3/6)	No shipped behavior changes; the PR reports native Windows fixture repetitions and deterministic editor fault injection, with current-head CI still running.
Patch quality	🐚 platinum hermit (4/6)	No actionable review findings were identified.

Product

Kind: Test only · Worth it: Yes · Fix scope: Complete
User problem: Contributors receive false CI failures while native controls are still appearing, moving, or being replaced.
Reason: The maintainer-authored direction is established, the repairs target evidenced test failures, and product behavior and final assertions remain unchanged.

Merge readiness

✅ Ready for maintainer review

Keep open: this collaborator-authored PR makes useful test repairs absent from main, preserves the final assertions, and introduces no actionable defect found in this review.

Priority: P2
Reviewed head: 36093e21bc9e41fb16ca2978b57da4baea34a41c

Before merge

None.

Findings

None.

Agent review details

How this fits together

The fixture UI suite launches an isolated Windows app against a synthetic Gateway, drives native controls through UI Automation, and produces assertions and diagnostic artifacts for CI.

flowchart LR
  A[Fixture CI job] --> B[Gateway fixture UI tests]
  B --> C[Isolated Windows app]
  C <--> D[Synthetic Gateway]
  B --> E[UI Automation observations]
  E --> F[Assertions and diagnostic artifacts]

Loading

Technical review

Best possible solution:

Land the focused readiness repairs with their strict assertions intact and investigate the explicitly excluded failures separately.

Do we have a high-confidence way to reproduce the issue?

The base wait cannot finish when scrolling settles at 22.4% after a 49.4% request while the field remains clipped; the PR reports matching before/after fault injection, but this read-only review ran no tests.

Is this the best way to solve the issue?

The smallest complete repair is to observe readiness through the existing polling and scrolling owners while retaining final assertions; the diff follows that approach without production hooks or new test cases.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning medium; reviewed against 64c96ac92d10.

Provenance checked

• Automations editor scrolling and inventory reads keeps the original intent (994539b: feat(automations): native Automations page with web parity #1893 established strict native editing and inventory coverage; the changed waits preserve those outcomes.)
• Sessions archive-command visibility keeps the original intent (3f47e83: feat: manage sessions and worktrees in native Windows UI #1897 added the native Sessions gallery; the repair uses its existing scrolling owner and retains visible-command verification.)
• Hub-window discovery keeps the original intent (3e74082: feat: explore and test the app without a live Gateway #1439 established isolated real-app fixture observations; the original composer-based identity remains, with a bounded readiness wait.)
• Assistant-selector readiness keeps the original intent (3a58bf3: feat(workspace): add native Home and Sessions workspace #1541 added native workspace and agent-creation coverage; waiting for the enabled selector preserves its creation and draft-retention assertions.)

Testing

Proof path: in-process harness.

Security

None.

Evidence

What I checked:

• Pinned scope and current main: The reviewed head changes only five existing UI-test files; GitHub main still resolves to the pinned base, so these repairs are not implemented there. (36093e21bc9e)
• Editor convergence retains strict visibility: The changed wait accepts settled observations so the existing bounded loop can recalculate; its final full-visibility assertion remains intact. (tests/OpenClaw.Tray.UITests/AutomationsUiTests.cs:1770, 36093e21bc9e)
• Complete inventory read preserves absence assertion: Cached AutomationId reads avoid subsequent live-peer reads; only the observed E_UNEXPECTED traversal failure is handled locally, and the first complete traversal still fails immediately if a matching automation exists. (tests/OpenClaw.Tray.UITests/AutomationsUiTests.cs:1912, 36093e21bc9e)
• Existing owners reused: Sessions uses the existing ScrollIntoView owner before its retained visibility assertion; both pinned hosts share WaitForHubAsync, which retains the original composer-based window identity. (tests/OpenClaw.Tray.UITests/GatewayFixtureUiTests.cs:939, 36093e21bc9e)
• Continuity failure remains visible: The timeout handler writes a best-effort receipt and rethrows; successful chat.send outcome and request-count assertions are unchanged. (tests/OpenClaw.Tray.UITests/GatewayFixtureUiTests.Continuity.cs:103, 36093e21bc9e)
• Reported validation and CI boundary: The PR reports deterministic editor before/after injection, repeated native Windows passes, 124 passing fixture UI cases, and required build/Shared/Tray closeout. Current-head fixture, UI, and C# security checks remain in progress. Earlier fixture-job failures were verified through job metadata, but raw logs redirected to a blocked storage host. (36093e21bc9e)

Likely related people:

• Barbara Kudiess: Raw commit 994539b adds tests/OpenClaw.Tray.UITests/AutomationsUiTests.cs:1758 relative to its recorded parents. This identifies author metadata, not feature responsibility or a PR merger. (role: source-line author; confidence: high; commits: 994539b8b6f5; files: tests/OpenClaw.Tray.UITests/AutomationsUiTests.cs)
• Scott Hanselman: Raw commit 3e74082 adds tests/OpenClaw.Tray.UITests/GatewayFixtureUiTests.cs:318 relative to its recorded parents. This identifies author metadata, not feature responsibility or a PR merger. (role: source-line author; confidence: high; commits: 3e740827aeb1; files: tests/OpenClaw.Tray.UITests/GatewayFixtureUiTests.cs)
• Karen: Raw commit 3a58bf3 adds tests/OpenClaw.Tray.UITests/GatewayFixtureUiTests.cs:5 relative to its recorded parents. This identifies author metadata, not feature responsibility or a PR merger. (role: source-line author; confidence: medium; commits: 3a58bf34902f; files: tests/OpenClaw.Tray.UITests/GatewayFixtureUiTests.cs)

Preflight source

Source unknown. Source comparison at the observed PR snapshot only. Completion is self-reported; body, policy, tests, execution, and report bytes are not verified.

Review metrics

Metric	Value	Why it matters
Change scope	Production +0/-0; tests +168/-21	Growth stays within existing UI tests and supports distinct readiness failures and timeout diagnostics.

Labels

Label changes:

• add P2: Recurring fixture-test failures consume contributor and maintainer effort without establishing an urgent product regression.
• add rating: 🦐 gold shrimp: Overall readiness is 🦐 gold shrimp; proof is 🦐 gold shrimp and patch quality is 🐚 platinum hermit.
• add status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR.

Label justifications:

• P2: Recurring fixture-test failures consume contributor and maintainer effort without establishing an urgent product regression.
• rating: 🦐 gold shrimp: Overall readiness is 🦐 gold shrimp; proof is 🦐 gold shrimp and patch quality is 🐚 platinum hermit.
• status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR.

Rating scale

6/6 🦀 challenger crab · 5/6 🦞 diamond lobster · 4/6 🐚 platinum hermit · 3/6 🦐 gold shrimp · 2/6 🦪 silver shellfish · 1/6 🧂 unranked krab. Overall follows the weaker of proof and patch quality; ✨ marks media proof (a screenshot, video, or linked artifact) that directly shows the changed behavior.

Workflow

ClawSweeper edits this one comment on every review. Comment @clawsweeper re-review for a fresh review only. Repair and merge execution lanes are retired.

Reviewed November 12, 2026, 6:58 PM ET / 23:58 UTC.