Skip to content

Fix duplicate terminal-state notifications: epoch-scoped durable dedup - #222

Merged
johnnyelwailer merged 1 commit into
mainfrom
fix/duplicate-terminal-notifications-dedup
Sep 12, 2026
Merged

johnnyelwailer merged 1 commit into
mainfrom
fix/duplicate-terminal-notifications-dedup

Conversation

@johnnyelwailer

Copy link
Copy Markdown
Owner

What this changes

Fixes the spurious + duplicated terminal-state notifications ("[Child stopped abnormally]" and "[Thread stopped]"). Root cause: one real stop can publish several terminal events (provider ingestion of turn.aborted + the transient-retry settle back-off re-issue) and a restart re-resolves the same target, each firing a fresh notification.

One shared mechanism, two call sites — t3team-terminalNotifyDedup.ts:

  • An epoch-scoped "already notified" ledger. A terminal stop reports once; the marker (a per-key thread.activity.append) persists on the observed thread and is rehydrated into an in-memory map at boot. The marker is valid only until the observed thread resumes (running/starting) — a later terminal stop on a new turn still notifies.
  • Call site 1 — abnormal-stop (t3team-childAbnormalStopDedup.ts): a child that stops abnormally reports to its parent exactly once, re-arming on the child's resume.
  • Call site 2 — silence-watch (t3team-threadSilenceWatchReactor.ts + emit): a watch reporting its target's terminal state reports once per epoch; the marker persists on the watcher and rehydrates at boot, so a restart does not re-emit a watch that was already reported.

Also from the original lane:

  • Close the child-wait / actor-message startup race: the live domain-event stream is forked after the pending index rehydrates, so a terminal event is not lost to a handler that forks before rehydration.
  • Two narrow publisher guards: transient-retry skips the session.set re-issue when the thread is already terminal; the provider session-stop path skips setThreadSession("stopped") when already stopped.

What is NOT in this lane (follow-up)

The delivery-layer burst fold — "any automated burst folds into one compact message (count + one line/item + t3team_read_message pointer)" — is a separate concern owned by the digest framing (buildActorReactionDigestInput) + the restart-hold summary. Per the guidance the dedup lands first so the fold operates on a deduplicated set. The fold is not in this PR.

Verification

Note on the pi-jobs-ledger

Checked for overlap with the new background-job work (#220/#221): no duplication. packages/client-runtime/src/work-log/backgroundJobs.ts is a client-side, transcript-driven fold of background bash jobs (tool-result markers → running/finished state). This ledger is a server-side notification dedup over the orchestration event log. Different layer, different concern.

One shared 'already notified' ledger (t3team-terminalNotifyDedup.ts), two
call sites:
- abnormal-stop: a child that stops abnormally reports to its parent exactly
  once, re-arming only when the child resumes (running/starting).
- silence-watch: a watch reporting its target's terminal state reports once
  per epoch; the durable marker persists on the watcher and is rehydrated at
  boot.

Also: close the child-wait / actor-message startup race (fork after the
pending index rehydrates), and two narrow publisher guards (transient-retry
terminal-status guard; provider session-stop already-stopped guard).

Fixes spurious + duplicated '[Child stopped abnormally]' / '[Thread stopped]'
notifications. The delivery-layer burst fold is a follow-up (out of this lane).
@github-actions github-actions Bot added vouch:trusted PR author is trusted by repo permissions or the VOUCHED list. size:XL labels Sep 12, 2026
@johnnyelwailer
johnnyelwailer merged commit fe484ad into main Sep 12, 2026
9 of 16 checks passed
johnnyelwailer pushed a commit that referenced this pull request Sep 12, 2026
…n notice

Distinguishes a plan-mode child that stopped awaiting parent approval from a
genuinely finished child, splits the parent's settled-own-work + children
waiting into two labels (Monitoring = derived, Waiting = declared), and tells
the parent when a child silently completes.

- shared: t3team-threadAwaitingParent (awaitingParent predicate + actionable-
  plan fold), t3team-childWaitFacts (open child-wait predicate + kinds),
  ThreadRunStatus gains awaitingParent + waitingDeclared flags.
- server children tool: list/status emit awaitingParent; status (detail load)
  emits waitingDeclared from open child-wait activities; usage text updated.
- server start-child: kickoff_mode 'plan' result states the plan-obligation.
- server silent-completion notice (item 5/6): reuses the standalone
  abnormal-stop notifier path (no second notifier), extended to outcome
  'completed'. 'Did it report?' is derived from the PARENT's durable
  transcript (any actor message whose sender is the child) — a stuck child
  can't hide a finished one. The notice's once-per-terminal-epoch dedup is
  the shared terminal-notify ledger (t3team-childAbnormalStopDedup.ts /
  t3team-terminalNotifyDedup.ts) — one durable marker, both terminal
  outcomes; the parent-transcript check is only the 'was it silent?' gate.
  A plan-mode child awaiting approval is called out explicitly. One short
  decide line (settle via t3team_children op:'sweep', or follow up).
- web: ProjectThread.awaitingParent + waitingDeclared; bridge derives both;
  ForkSection labels Monitoring/Waiting (declared outranks derived); sub-run
  tree + sidebar badges; labels centralized in one place.

Rebased onto fe484ad (includes #222 dedup lane). Integration: widened
AbnormalStopGuards.notifyAbnormalStop to ChildTerminalOutcome so the
completed notice flows through the sibling's ledger; kept both the ledger
epoch-dedup and the parent-transcript silent-gate. Guarded, no second
execution runtime; focused tests + typecheck green.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XL vouch:trusted PR author is trusted by repo permissions or the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant