You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Terminal scrollback is shown twice after a replay timeout #9451
When a terminal subscribe times out waiting for its replay and the renderer retries, the new subscription also receives the replay of the dropped one. Every line of scrollback then appears twice.
Expected: a new subscription gets only its own replay.
Steps to reproduce
Make the pty-daemon slow to answer (busy host), so waitForSessionReplay hits its 5 s timeout.
The renderer retries (DAEMON_UNAVAILABLE is transient) and subscribes again on the same client.
See the scrollback twice.
Logs, screenshots, or recordings
Found by model checking every order of events, then reproduced as a failing test against the real code. A fix with that regression test is ready.
Implementation notes
Files
packages/host-service/src/terminal/DaemonClient/DaemonClient.tssubscribe and onData: the protocol has no subscription token. onData routes output and replay-complete by session id only, so the answers owed to the dropped subscribe go to the new one.
Approach
Count subscribe frames for each id that are not answered yet (replay-complete, or an error with that id). A new subscriber skips output and answers owed to subscribes sent before it registered.
Gotchas
Live output sent in the one round trip between the first replay-complete and the daemon handling the unsubscribe can still duplicate. A full fix needs a token echoed in replay-complete.
I read the code and the report holds up. The client routes subscribe answers by session id only. If a subscribe times out and gets torn down, the daemon still answers it later, and those answers reach the next subscriber on the same DaemonClient for that id. The dropped subscribe's replay is then treated as the new subscription's replay. Its stale replay-complete also marks the new subscription as ready too early, so when the new subscription's own replay arrives, it is handled as live output. The result is that the whole scrollback is written to the terminal twice.
Affected code
packages/host-service/src/terminal/DaemonClient/DaemonClient.ts:236-296: subscribe(). After an unsubscribe, the callbacks entry is deleted (:292), so the retry takes the wasFirst branch again (:242). It creates a new ReplayCompletion (:262) and sends a second { type: "subscribe", replay: true, modeSnapshot: true } (:268-273). The frame has no token to tell it apart from the first one.
DaemonClient.ts:494-531: onData(). replay-complete resolves whatever ReplayCompletion is pending for msg.id (:499-508). output goes to the current callbacks for msg.id (:511-524). Error frames with an id reject the current replay (:496-498). Nothing checks which subscribe a frame answers.
packages/host-service/src/terminal/terminal.ts:1069-1097: waitForSessionReplay(). When the 5 s ReplayCompletion timeout fires (DaemonClient.ts:262-265), this calls session.unsubscribeDaemon() and returns DAEMON_UNAVAILABLE with transient: true. That is the signal for the caller to retry.
terminal.ts:3366 / :3490: these lines create the subscription and set adoptionReplaySettled from daemon.waitForReplay(terminalId).
packages/pty-daemon/src/handlers/handlers.ts:173-202: the daemon processes frames in order. handleSubscribe sends the ring snapshot as output and then replay-complete. handleUnsubscribe only removes the id from conn.subscriptions. Neither one sends anything that identifies the subscribe.
Suspected root cause
When the daemon is slow, the client sends subscribe#1 → (timeout) unsubscribe → subscribe#2. The daemon handles them in order and replies with:
The same mixing affects errors. An ENOENT error frame owed to subscribe #1 would reject #2's replay (:496-498).
Reproduction
You can add a *.node-test.ts next to DaemonClient.node-test.ts:
Start a fake Unix-socket server. It answers hello with hello-ack (supportsModeSnapshots: true) and records the subscribe / unsubscribe frames without replying.
client.subscribe("s1", { replay: true }, cbA). Inject a short replay timeout so the test does no real 5 s wait. Today the 5000 is hardcoded at DaemonClient.ts:263, so it needs to become a parameter. After the timeout, call the returned unsubscribe.
client.subscribe("s1", { replay: true }, cbB).
The server now replies in daemon order: output("SCROLLBACK"), replay-complete, then output("SCROLLBACK"), replay-complete.
Assert that cbB.onOutput received "SCROLLBACK" once. Today it receives it twice. waitForReplay("s1") also resolves after the first replay-complete, which is too early.
Suggested fix direction
Client-only change (works with released daemons): count the subscribe frames sent for each id that have no answer yet. An answer is a replay-complete, or an error with that id. When a subscribe is sent while that count is non-zero, record how many earlier answers are still owed. Drop output frames and answers for that id until those owed answers have arrived, then start delivering to the new subscriber. This depends on every modeSnapshot subscribe getting exactly one terminal answer, which handleSubscribe does (handlers.ts:178-197). As the issue says, live output sent between the stale replay-complete and the daemon handling unsubscribe can still be duplicated.
Complete fix: add a subscription token (e.g. subId) to the subscribe frame and echo it in output and replay-complete, behind a hello-ack capability flag. Old daemons would keep the client-only behaviour. This is a protocol change in @superset/pty-daemon/protocol, so it needs the compatibility handling for released host-service and daemon builds.
Unrelated to the timeout path: daemons without supportsModeSnapshots send no replay-complete, so neither approach covers them. That is probably acceptable, but worth noting in the fix.
Activity
github-actions commentedon Nov 7, 2026
Summary
I read the code and the report holds up. The client routes subscribe answers by session id only. If a
subscribetimes out and gets torn down, the daemon still answers it later, and those answers reach the next subscriber on the sameDaemonClientfor that id. The dropped subscribe's replay is then treated as the new subscription's replay. Its stalereplay-completealso marks the new subscription as ready too early, so when the new subscription's own replay arrives, it is handled as live output. The result is that the whole scrollback is written to the terminal twice.Affected code
packages/host-service/src/terminal/DaemonClient/DaemonClient.ts:236-296:subscribe(). After an unsubscribe, thecallbacksentry is deleted (:292), so the retry takes thewasFirstbranch again (:242). It creates a newReplayCompletion(:262) and sends a second{ type: "subscribe", replay: true, modeSnapshot: true }(:268-273). The frame has no token to tell it apart from the first one.DaemonClient.ts:494-531:onData().replay-completeresolves whateverReplayCompletionis pending formsg.id(:499-508).outputgoes to the current callbacks formsg.id(:511-524). Error frames with an id reject the current replay (:496-498). Nothing checks which subscribe a frame answers.packages/host-service/src/terminal/terminal.ts:1069-1097:waitForSessionReplay(). When the 5 sReplayCompletiontimeout fires (DaemonClient.ts:262-265), this callssession.unsubscribeDaemon()and returnsDAEMON_UNAVAILABLEwithtransient: true. That is the signal for the caller to retry.terminal.ts:3366/:3490: these lines create the subscription and setadoptionReplaySettledfromdaemon.waitForReplay(terminalId).packages/pty-daemon/src/handlers/handlers.ts:173-202: the daemon processes frames in order.handleSubscribesends the ring snapshot asoutputand thenreplay-complete.handleUnsubscribeonly removes the id fromconn.subscriptions. Neither one sends anything that identifies the subscribe.Suspected root cause
When the daemon is slow, the client sends
subscribe#1→ (timeout)unsubscribe→subscribe#2. The daemon handles them in order and replies with:output(ring snapshot Bump react-dom from 18.3.1 to 19.2.0 #1). The client has already registered subscriber Bump lucide-react from 0.468.0 to 0.546.0 #2, so this goes to Bump lucide-react from 0.468.0 to 0.546.0 #2.replay-completeBump react-dom from 18.3.1 to 19.2.0 #1. This resolves Bump lucide-react from 0.468.0 to 0.546.0 #2'sReplayCompletionand callsrestoreModeswith Bump react-dom from 18.3.1 to 19.2.0 #1's snapshot.unsubscribeis handled. It produces no reply.)output(ring snapshot Bump lucide-react from 0.468.0 to 0.546.0 #2). Bump lucide-react from 0.468.0 to 0.546.0 #2 is already "ready", so this is handled as live output. This is the duplicate.replay-completeBump lucide-react from 0.468.0 to 0.546.0 #2. This is ignored becausestatus !== "pending"(:501).The same mixing affects errors. An
ENOENTerror frame owed to subscribe #1 would reject #2's replay (:496-498).Reproduction
You can add a
*.node-test.tsnext toDaemonClient.node-test.ts:hellowithhello-ack(supportsModeSnapshots: true) and records thesubscribe/unsubscribeframes without replying.client.subscribe("s1", { replay: true }, cbA). Inject a short replay timeout so the test does no real 5 s wait. Today the 5000 is hardcoded atDaemonClient.ts:263, so it needs to become a parameter. After the timeout, call the returned unsubscribe.client.subscribe("s1", { replay: true }, cbB).output("SCROLLBACK"),replay-complete, thenoutput("SCROLLBACK"),replay-complete.cbB.onOutputreceived"SCROLLBACK"once. Today it receives it twice.waitForReplay("s1")also resolves after the firstreplay-complete, which is too early.Suggested fix direction
replay-complete, or anerrorwith that id. When a subscribe is sent while that count is non-zero, record how many earlier answers are still owed. Dropoutputframes and answers for that id until those owed answers have arrived, then start delivering to the new subscriber. This depends on everymodeSnapshotsubscribe getting exactly one terminal answer, whichhandleSubscribedoes (handlers.ts:178-197). As the issue says, live output sent between the stalereplay-completeand the daemon handlingunsubscribecan still be duplicated.subId) to thesubscribeframe and echo it inoutputandreplay-complete, behind ahello-ackcapability flag. Old daemons would keep the client-only behaviour. This is a protocol change in@superset/pty-daemon/protocol, so it needs the compatibility handling for released host-service and daemon builds.supportsModeSnapshotssend noreplay-complete, so neither approach covers them. That is probably acceptable, but worth noting in the fix.