test(remote): investigate intermittent live transcript bootstrap failure #847

Activity

1. Juliusolsson05 commented on Sep 11, 2026

   

   Juliusolsson05

   on Sep 11, 2026

   OwnerAuthor

   More actions

   Two more occurrences today, on commits that don't touch src/remote-client or src/main/remote:

   1. main @ 48ea6912 (merge of feat(codex): add Remove Cybersecurity Block command #849): quality-gate "Enforce coverage baseline" failed a different case in the same file, "remote reconnect after bounded output overflow > backfills a previously loaded session after overflow and retains access to every missed page", with expected [] to have a length of 10 but got +0 at 1017ms. The same case passed earlier in that job's "Test system" step. https://github.com/Juliusolsson05/agent-code/actions/runs/34539496773
   2. PR fix(dispatch): give every Grid Dispatch lane its own selector strip #859 @ 6ee33cc0 (Dispatch renderer files only): quality-gate "Test system" failed this case again, "TranscriptStore folds live jsonl + semantic + backfill…", with expected [] to deeply equal [ 'u-live' ] at 1012ms. The PR's previous head 0ef2b17e passed the whole file. https://github.com/Juliusolsson05/agent-code/actions/runs/34545514427

   One pattern that may help the investigation, though it isn't a diagnosis: both failures land at about 1s, which is vi.waitFor's default timeout, and both assertions saw an empty result, not a partial one. That fits the snapshot never being populated in time, whether because of a slow runner or the readiness/ordering race this issue describes. I haven't changed any timeout. It's now three failures across two cases, including one on main.
2. Juliusolsson05 commented on Sep 11, 2026

   

   Juliusolsson05

   on Sep 11, 2026

   OwnerAuthor

   More actions

   Another sighting, different case in the same file: PR #872 (head c1c9e21), quality-gate → Test system failed in WebSocketSessionFeed.integration.test.ts > remote reconnect after bounded output overflow > backfills a previously loaded session after overflow and retains access to every missed page with AssertionError: expected [] to have a length of 10 but got +0 (same "entries empty" shape as the original report).

   Why this is not the PR's change: #872 adds a terminal-kind gate to SessionFeedSource.emit (getSessionKind(id) ?? getSpawnKind(id)), but this test's fake manager returns 'claude' from getSessionKind for every id, so the gate never drops a frame and the getSpawnKind fallback is never evaluated. The file passed 13/13 three consecutive times locally on Node 24 at the same commit.

   Run: https://github.com/Juliusolsson05/agent-code/actions/runs/34624833635 — failed job rerun on the identical commit with no test/source/timeout changes.
3. Juliusolsson05 commented on Sep 11, 2026

   

   Juliusolsson05

   on Sep 11, 2026

   OwnerAuthor

   More actions

   Second sighting on PR #872 (merged-with-main head 6cd99ec, run https://github.com/Juliusolsson05/agent-code/actions/runs/34628010752): same case, backfills a previously loaded session after overflow and retains access to every missed page, expected [] to have a length of 10 but got +0.

   Likely root cause for this whole class: the failing assertion is the test's FIRST wait —
   await vi.waitFor(() => expect(store.getSnapshot('s1').entries).toHaveLength(10)) right after manager.emit('started', …) — which uses vitest's DEFAULT 1000 ms vi.waitFor timeout (the first failure timed out at 1015 ms), while every later wait in the same test passes { timeout: 6000 }. The initial backfill is a full WebSocket round trip (started → client get-history → server resolveTranscriptFile + file read → reply → TranscriptStore fold); on a loaded macOS runner that can exceed 1 s. The original report's case (TranscriptStore folds live jsonl + semantic + backfill) shows the same "entries empty" shape.

   Not changing the test in #872 (standing rule: don't raise timeouts to silence flakes; the fix belongs to this issue with its own decision). #872's remote change is a terminal-kind gate in SessionFeedSource.emit; this test's fake manager returns 'claude' for every id, so the gate is a pass-through here. Failed job rerun on the identical commit.
4. Juliusolsson05 commented on Sep 12, 2026

   

   Juliusolsson05

   on Sep 12, 2026

   OwnerAuthor

   More actions

   Another occurrence on an unrelated PR, during the coverage re-run of the quality gate (core, system and renderer tiers had all passed on the same commit):

   • PR feat(control): add Root Agent Code Management command with confirmation gate #907 (Root Agent Code Management), commit e697f30b
   • Run: https://github.com/Juliusolsson05/agent-code/actions/runs/34662138975/job/103466662199
   • WebSocketSessionFeed against a live RemoteServer > TranscriptStore folds live jsonl + semantic + backfill into a feed-ready snapshot
   • AssertionError: expected [] to deeply equal [ 'u-live' ], 477 of 478 files passing

   The PR touches the built-in MCP domain list, the control executor's caller kinds, the operator tool projection and a renderer command; the remote feed path and its fake manager are unreachable from that change. Re-running the failed job rather than changing any timeout.
5. Juliusolsson05 commented on Sep 12, 2026

   

   Juliusolsson05

   on Sep 12, 2026

   OwnerAuthor

   More actions

   Second consecutive occurrence on the same commit (e697f30b, PR #907), this time in the system tier of the re-run job: https://github.com/Juliusolsson05/agent-code/actions/runs/34662138975/job/103468427654 — same assertion (expected [] to deeply equal [ 'u-live' ]). The same tier passed on this exact commit in the previous attempt, so the failure is not deterministic for the branch.
6. Juliusolsson05 commented on Sep 12, 2026

   

   Juliusolsson05

   on Sep 12, 2026

   OwnerAuthor

   More actions

   Another occurrence, PR #909 commit 31321855 (two renderer modals and a pure helper; remote feed unreachable): system tier, https://github.com/Juliusolsson05/agent-code/actions/runs/34669154994/job/103487110726. Different case in the same file this time: remote reconnect after bounded output overflow > backfills a previously loaded session after overflow and retains access to every missed page, AssertionError: expected [] to have a length of 10 but got +0. 18 of 19 system files passed. Re-running the failed job; no timeouts changed.
7. Juliusolsson05

   added a commit that references this issue on Sep 12, 2026

   docs(remote): record the session-list eviction pull request in the plan

   ...

   b4f3501
8. Juliusolsson05 commented on Sep 12, 2026

   

   Juliusolsson05

   on Sep 12, 2026

   OwnerAuthor

   More actions

   Root cause established; fix in #911.

   The failure is in the phone client's TranscriptStore, not in the test. Every session-list frame was treated as authoritative: any session absent from the list lost its state and its listener set. The server computes the handshake list at connection time, before a session that starts a moment later, and the client feed reports open on the socket event before the handshake frames are parsed. Whether that stale list is processed before or after a view subscribes depends on TCP chunking of the upgrade response:

   • same chunk: the stream drains on process.nextTick, before the promise continuation that subscribes; nothing evicted; the started patch backfills (passing order);
   • later chunk: the list is parsed after the view subscribed and read its first snapshot; state and listeners deleted; the started patch finds no state; later live entries dropped as "unviewed". The view stays mounted and empty. That is both signatures seen here (expected [] to have a length of 10, expected [] to deeply equal ['u-live']).

   CI load widens the gap between the upgrade response and the handshake frames, which is why it clustered there and rarely reproduced locally. On the phone the same race hits any reconnect and any tap on a just-started session.

   #911 makes lists evict only unviewed state, never touch listeners, retries backfill over views rather than states, and drops a forgotten session's state when its last view leaves (retention guarantee kept). A new unit suite replays the order deterministically; all four cases fail on the pre-fix store. No timeout or assertion was changed in the integration test.
9. Juliusolsson05

   closed this as completedin #911on Sep 12, 2026