Skip to content

fix(desktop): diagnose and recover Multiverse receiver initialization - #7364

Draft
loganj wants to merge 4 commits into
fix/multiverse-launch-community-0905from
fix/multiverse-receiver-recovery-0905
Draft

fix(desktop): diagnose and recover Multiverse receiver initialization#7364
loganj wants to merge 4 commits into
fix/multiverse-launch-community-0905from
fix/multiverse-receiver-recovery-0905

Conversation

@loganj

@loganj loganj commented Sep 5, 2026

Copy link
Copy Markdown
Collaborator

Dependent repair atop #7363. Origin: Buzz channel f45d3304-dcf0-44e8-a46d-bcd63b235fbc, thread 16211fefcee85904f49802ce5e1709bf24b4895401a944f0585e70233c4e4d39.

Behavior

  • Safe receiver-stage diagnostics distinguish initialization failure from readiness timeout; raw IPC/transport exceptions are never displayed.
  • Ongoing subscription state survives EOSE. Transient initialization/CLOSED failures recover at most three times per owner, with 1s/2s/4s backoff and the relay rate-limit gate. Success does not reset that budget.
  • Every replacement subscribes live-only (limit: 0) and completes projection sync before command admission. Cancellation/generation fences retire late work. Explicit subscription retirement cannot reissue an orphan REQ after reconnect.
  • Terminal closure, known latched-terminal initialization, or exhausted recovery reports through the scope-owned Retry receiver notification. Deliberate receiver retry creates a new budget, not a new operation. No historical command replay, automatic operation retry, AUTH change or new daemon. Commands during recovery gaps remain unconfirmed.

Evidence and review

Current head: 8f2dd70f9633181c8603a37a837addecec19704e. Full local frontend suite: 6,291 passed; TypeScript and static checks passed. The preceding bounded-owner head a2380832 passed four mounted Stop/chrome checks; the final delta changes only terminal-init classification/tests/docs, not geometry. Independent source review found the prior receiver defects repaired and one P3 terminal-init classification gap, now fixed with a production owner regression. No GitHub technical approval or current-head CI completion is claimed here; see current checks/comments.

Native and launch limits

The completed parent 110374bd native run failed remote Stop before admission. Subsequent isolated diagnostic C captured subscribeLive → Failed to connect to relay; general connection recovered while that original receiver remained unavailable. The plugin discarded the lower-level rejection, so neither its precise cause nor identity with the original A/B failure is established. C was cleaned up; its separate relaunch splash stall is not repaired here. Successful native remote Stop/result, stored-result retry and no-resurrection acceptance remain outstanding. Local Stop auto-start policy is unchanged. Actual destination-host keyless provisioning remains unavailable; no successful new Start/Restart/Move is claimed. Nothing merged or deployed.

Signed-off-by: Logan Johnson <loganj@squareup.com>
Signed-off-by: Logan Johnson <loganj@squareup.com>
@loganj

loganj commented Sep 5, 2026

Copy link
Copy Markdown
Collaborator Author

Updated receiver repair at 7fdaa65b9710d806f94a4b91e334f7a0d4e9bac4 (fast-forward from 8c21ecb0).

Independent source review found the original new warning used a one-shot readiness callback: production cleared it on EOSE, so subsequent CLOSED was invisible. Fixed at the subscription owner with opt-in persistent state and explicit receiver retirement. Default subscribers retain shared retry/reconnect. New regression tests drive real RelayClient inbound EOSE/CLOSED/reset, queued lifecycle work fencing and deliberate fresh retry, rather than inventing a persistent fake readiness callback.

Validation: full 6,279 frontend tests, typecheck/check and production build passed on the exact patch; mounted desktop-stop smoke passed after local integration. Committed diff matches the tested patch byte-for-byte. Prior 8c21ecb0 completed full local just ci (6,273 frontend; 3,181 native + 10 native integration; selected relay and other package lanes), and unchanged native/backend evidence is reused. Fresh GitHub CI is running. No native remote Stop/recovery success claimed.

Still not approved: the separate runtime-authority collision finding in parent #7363 is being repaired. The original native initialization exception remains unproven; instrumented native onboarding continuation is active. Destination-local keyless provisioning remains unavailable. No merge/deployment.

@loganj

loganj commented Sep 5, 2026

Copy link
Copy Markdown
Collaborator Author

Native diagnostic update (2026-09-05): the isolated full-feature instrumented 110374b demo reached legitimate GUI onboarding and captured a real initialization error at subscribeLive: "Failed to connect to relay." The general relay connection subsequently authenticated/recovered, while that original receiver remained unavailable. This establishes the failure before history projection/admission, not the underlying non-Error plugin rejection (which was discarded), and not that the lost original A/B exception was identical. Current 7fdaa65 already offers deliberate Retry for initialization failures; native acceptance of that repair remains pending. C also stalled before receiver mount on a later relaunch; no healthy-restart claim. All private fixture processes/services were stopped; installed Desktop/production state and frozen A/B artifacts were untouched. No successful native remote Stop/retry/no-resurrection claim is made by the 6,279-test frontend pass.

@loganj

loganj commented Sep 5, 2026

Copy link
Copy Markdown
Collaborator Author

Follow-up review of exact 7fdaa65b9710d806f94a4b91e334f7a0d4e9bac4 confirms the prior post-EOSE readiness-callback P1 is fixed. This is not approval: a reconnect-send race can issue an explicit subscription's REQ after reset already retired its client owner. Fixing that by preventing the retired REQ's retry, not by accepting an orphan.

The availability finding is also actionable: retryable subscription CLOSED on an otherwise healthy socket has no connection-reconnect notification, and the native C diagnosis separately proved failed first connect followed by first success can leave the original one-shot initializer stranded. A bounded receiver-owned restart is being implemented. Each attempt retires/fences the old receiver, creates a fresh live-only subscription, and performs placement projection before admission; it must never replay a command or revive an old subscription. Terminal errors/budget exhaustion retain deliberate Retry. Rate-limit backoff and cancellation are explicit acceptance boundaries.

The review's disconnect() omission has no currently reachable user failure because both callers remount the community subtree; it will be covered proportionately at the contract boundary. No raw plugin exception text is being exposed.

CI on this exact head remains running with no failure at 03:23 UTC. No successful native remote Stop/recovery or destination-host keyless provisioning is claimed. The composed repair remains a draft, not approved/ready to merge.

Signed-off-by: Logan Johnson <loganj@squareup.com>
@loganj

loganj commented Sep 5, 2026

Copy link
Copy Markdown
Collaborator Author

Published receiver-owned recovery at a2380832e656db01b5cc17137cc94eca5ee51be5 (fast-forward from 7fdaa65b, whose CI finished terminal green before this update).

  • Fresh live-only receiver attempts with bounded 1s/2s/4s backoff, shared rate-limit pacing and cancellation fencing; no command replay. Terminal closure/exhaustion retains deliberate Retry.
  • Explicit subscription ownership is checked before reconnect-send retry, preventing an orphan REQ after retirement. Shared subscribers retain their ordinary retry path.
  • Full 6,290 frontend tests passed locally and on the isolated source worker; static checks/typecheck/build passed. Four mounted Stop/chrome smoke checks passed locally, including the final exhausted-recovery notification with unchanged chrome geometry. The test now waits through the deliberate recovery budget rather than requiring failure UI before retry has finished.
  • Native/backend code is unchanged from the previously tested base. No successful native remote Stop/recovery is claimed; the actual native initial-connect failure is addressed by production-bound regression coverage, not yet native acceptance.

Independent delta review is running; current-head CI has been refreshed. The separate authority-receipt repair is still under correction and not published. Real destination-host keyless provisioning remains unavailable. No merge/deployment/readiness claim.

Signed-off-by: Logan Johnson <loganj@squareup.com>
@loganj

loganj commented Sep 5, 2026

Copy link
Copy Markdown
Collaborator Author

Receiver review follow-through: published 8f2dd70f9633181c8603a37a837addecec19704e fixes the independent review’s sole P3 finding. Known latched-terminal initialization failures now report immediately; transient/unknown failures retain the bounded recovery budget. The regression runs the real owner + receiveLifecycle path and requires one subscription attempt, zero timers, no history or native admission, and the safe diagnostic. Full local frontend suite: 6,291 passed; typecheck/static check passed. The remainder of the reviewed owner repair is unchanged. Fresh exact-head CI is starting; no GitHub approval or native remote Stop/recovery acceptance is claimed. The independent source review found no P1/P2 defect in the receiver delta. Commands during receiver gaps still remain unconfirmed and require deliberate exact operation retry; no command replay or AUTH change.

@loganj

loganj commented Sep 5, 2026

Copy link
Copy Markdown
Collaborator Author

Composed runtime-authority follow-up is drafted in #7365 at 76fb1e66; exact repaired native Stop/result retry/recovery acceptance is running on that tip. Current receiver head remains 8f2dd70f. New keyless launch is separately blocked on identifying the trusted broker/key-custody host and its destination session issuer—not on this receiver or on a fictional credential from #6922/#6967.

@loganj

loganj commented Sep 5, 2026

Copy link
Copy Markdown
Collaborator Author

Current-head CI completed successfully at 8f2dd70f9633181c8603a37a837addecec19704e: 25 successful checks, 24 path-skipped, none pending/failed, including all four smoke shards, Desktop Core and both relay-backed integrations.

https://github.com/block/buzz/actions/runs/33943460385

This closes the fresh CI gate, not native acceptance or technical approval. The repaired native Stop/retry/recovery run is currently inaccessible because its isolated remote host lost its control connection; no pass/fail verdict is inferred. The original native failure remains recorded. Keyless session issuance remains unimplemented and requires the designated trusted host/interface. Nothing merged or deployed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant