Skip to content

buzz-acp: failed/steered turns respawn cold sessions with ~2.4s capped backoff (token-burn storm); ephemeral event kinds bypass respond_to gate #5327

Description

@1dot-dev

Summary

Under sustained event load, buzz-acp (Buzz Desktop 0.5.7, macOS arm64) handles a failed or steered turn by killing the child agent process and cold-spawning a brand-new runtime session, with retry backoff hard-capped at ~2.4 s and a slot-refill timer that re-arms the circuit breaker every ~5 min. In a room with two managed agents this produced a self-sustaining storm: roughly one brand-new cold-start Claude session every 10-15 seconds, sustained for hours, which also 429-flooded our hosted community relay and burned through the account's Claude subscription monthly spend limit. A contributing second issue: ephemeral event kinds (20002 presence, 7 reactions, 5 deletions) are delivered to agents regardless of the BUZZ_ACP_RESPOND_TO author gate, so two agents in one channel wake each other indefinitely when the mention filter is disabled.

Environment

  • Buzz Desktop 0.5.7 (also reproduced on the immediately prior version; the worst burst ran entirely on 0.5.7)
  • macOS 15 (Darwin 24.6.0), Apple Silicon
  • Runtime: claude (@agentclientprotocol/claude-agent-acp 0.64.1), model opus[1m]
  • 9 managed agents, parallelism: 2, hosted community relay (<community>.communities.buzz.xyz)
  • Global agent env at the time: BUZZ_ACP_SUBSCRIBE=all, BUZZ_ACP_NO_MENTION_FILTER=true, respond_to=allowlist(3)

Bug 1: respawn-per-failed-turn with capped backoff and self-defeating circuit breaker

Observed cycle, repeated 406 times in one agent's log over ~3.4 active hours (a second agent logged 333 cycles in 2.3 h):

WARN buzz_acp: agent_returned — respawning (cancel-drain timeout) agent=0 outcome="cancel_drain_timeout" configured_model=opus[1m] pid=27612 grace=5s
INFO buzz_acp: respawn backoff agent=0 delay_ms=995
INFO buzz_acp: respawn complete agent=0
INFO buzz_acp: agent initialized agent=0 name="@agentclientprotocol/claude-agent-acp"

Key observations:

  1. Every respawn is a full cold start: 406 respawns correspond to exactly 406 distinct runtime session UUIDs in the same log. No session resume. With a large-context model each cold start re-pays full context load, so the client's failure handling is directly metered in provider tokens.
  2. All 405 observed respawn backoff delays fall between 823 ms and ~2,390 ms. No exponential growth. (The binary contains a backoff ladder reset string, so a ladder exists, but under this failure pattern it never escalated.)
  3. The circuit breaker opens but is re-armed by the refill timer, ~5 minutes later, into the same failing conditions:
04:13:16 ERROR buzz_acp: circuit open — not respawning agent=1
04:18:19  INFO buzz_acp: slot refill: spawning background respawn agent=1
  1. The kill trigger under load is the steer path: non-cancelling steer not accepted — falling back to cancel+merge ... error=Transport("no available capacity") (20,476 occurrences in one log), then the 5 s cancel-drain grace expires and the child is killed. The replayed context is immediately steered again on the fresh child, so the loop is stable.
  2. The same storm drove WARN buzz_acp::relay: POST /query returned retriable HTTP 429 Too Many Requests at up to 837/min from a single agent (45k+ occurrences), i.e. the client DDoS'd its own relay while retrying.
  3. Once the provider account hit its spend limit, every fresh cold session failed immediately (steered new turn failed ... You've hit your monthly spend limit) and buzz-acp kept respawning anyway, hundreds more times.

Expected: exponential backoff (or at minimum a much higher cap) on repeated respawns; a circuit breaker that stays open while the failure condition persists; ideally session resume instead of cold spawn where the runtime supports it (loadSession: true is advertised); and treating a provider spend-limit/auth-class error as fatal for the slot rather than retriable.

There appear to be no user-facing tunables for retry behaviour (the backoff cap is compiled in; no relevant env_vars documented).

Bug 2: ephemeral event kinds bypass the respond_to author gate

With BUZZ_ACP_SUBSCRIBE=all + BUZZ_ACP_NO_MENTION_FILTER=true and respond_to restricted to an allowlist of 3 human/coordinator pubkeys, agents were still woken by other AGENTS' events, because kind 20002 (ephemeral/presence), kind 7 (reactions) and kind 5 (deletions) are delivered regardless of author. Census across 1,718 storm-triggered sessions: 84% kind 20002 with empty content, 9% kind 5, 6% kind 7, under 1% kind 9 messages. Since the harness itself emits reactions/presence when an agent reads events, two agents sharing a channel wake each other indefinitely with no kind-9 message ever passing the gate. We had previously observed the same amplification with respond_to=owner-only (reaction-on-reaction loops between two agents in one channel).

Expected: the author gate should apply to all wake-capable event kinds, or ephemeral/housekeeping kinds should never wake an agent at all.

Workaround that ended the incident for us: reverting to BUZZ_ACP_SUBSCRIBE=mentions (mention-driven wakes filter the untagged ephemeral events), plus removing the second agent trigger conditions. Happy to provide fuller log excerpts on request.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions