[jaredLunde]

What

Client-side support for the MCP Events extension in crates/agent: a serve session can subscribe to events on its configured MCP servers and react when they fire — over webhook (the mode ChatGPT ships), push (events/stream) and poll (events/poll) — with every event surfaced as an mcp_event frame and, unless the subscription says notify, handed to the model as a rate-bounded follow-up.

Draft status

There is no SEP. This implements the Triggers & Events WG design sketch at 6682596d65eec778fe0b8b1f43b4e89d2fe2c546, cross-checked against OpenAI's MCP Events guide and the TS SDK proposal (modelcontextprotocol/typescript-sdk#2945). The commit is pinned in tools/mcp_events/mod.rs (SPEC_COMMIT, also returned by mcp_events_list) and in ARCHITECTURE.md. A server with no events entry is never sent an events/* request, and a session that never subscribes writes no events state. What does change for every stdio MCP server, events or not, is its transport (see below).

Configuration

// ~/.claude/settings.json (or a trusted project's)
{ "mcp_servers": [{
  "name": "tickets", "transport": "http", "url": "https://mcp.example.com/mcp",
  "headers": { "Authorization": "Bearer …" },
  "events": [{
    "name": "ticket.updated",
    "arguments": { "project": "alpha" },
    "delivery": "webhook",          // optional: poll | push | webhook (default: negotiate)
    "action": "follow_up",          // notify | follow_up (default) | steer
    "instructions": "Triage it; page me only for P1.",
    "max_age_ms": 3600000           // optional: the draft's maxAgeMs bound on resume replay
  }]
}]}

serve --listen 127.0.0.1:8080 --mcp-events-callback-url https://agent.example.com turns webhook delivery on. Callbacks land on the existing listener at /_beyond/mcp-events/<token>. Runtime, per session (accepted idle or mid-run): mcp_events_list, mcp_events_subscribe, mcp_events_unsubscribe. On a daemon, configured subscriptions are owned by one session, --mcp-events-session (default mcp-events), which is started at boot. --mcp-events-reapable opts that session back into idle reaping.

Deployment: servers require https callback URLs, and serve --listen speaks plain HTTP by design (it binds loopback or an internal interface and trusts the front door, like the rest of the daemon). So --mcp-events-callback-url is the public https:// base of whatever terminates TLS and routes /_beyond/mcp-events/* (and /.well-known/mcp-webhook-receiver.json) to the listener. Verification doesn't depend on end-to-end TLS, because the HMAC covers the raw body and timestamp. This is documented in ARCHITECTURE.md.

Design decisions

• Configured subscriptions are owned once and kept up. On stdio serve, the session owns them. On a daemon they belong to one session, --mcp-events-session (default mcp-events), which starts at boot. One server event means one subscription and one model run.

  • Each configured subscription is retried with backoff (capped at 60 s) for as long as the session runs. That covers a failed first subscribe and a later termination by the server.
  • An explicit unsubscribe stops the retrying, even while a retry is waiting.
  • While a subscription is not up, it counts as live, so a daemon whose servers are briefly down keeps its events session.
  • Other sessions stay reapable. They can still subscribe at runtime with mcp_events_subscribe.

• Durable delivery. An accepted event is appended to the session's pending log and fdatasync'd before anything is acknowledged.

  • Webhook: the 2xx waits for that write. A full queue, a failed write or a timeout answers 503 instead, and the server's retry lands once the write works.
  • Poll: the cursor advances only when the whole page was accepted and stored.
  • Push: reconnects from its cursor.

• What counts as delivered. A batch leaves the pending queue only when the model has received it in a transcript that persisted.

  • The run's own prompt counts once the run did not fail.
  • A batch steered in mid-run counts only if it is in the transcript. It is found by the batch id its text starts with, so two batches with identical content are told apart. An abort clears the steer lane, so a steered batch that never reached the model is returned and injected again.
  • Failed runs and transcripts that did not persist also return their batches.
  • After 3 attempts that never reached the model, an event is dropped and reported (kind: "dropped"), so a poison event cannot re-run the model forever.

• The guarantee, exactly. It applies to follow_up and steer subscriptions; a notify event only becomes a frame, and nothing is queued. An event counts as acknowledged once its webhook got a 2xx or its poll cursor moved past it. A push cursor is saved only after the event's record.

  Every acknowledged event is on disk (fsync'd) before that point, and reaches the model at least once, across graceful restarts and crashes alike. The single exception is reported: an event is dropped with kind: "dropped" after 3 injections that never reached the model.

  It reaches the model exactly once except in four cases, each of which re-injects the batch:

  • a run fails, or is aborted before the model saw a steered batch (re-injected by design);
  • a compaction during the run summarizes a steered batch's turn away, so the batch is no longer found in the transcript. The model did see it, so this is a duplicate;
  • the process dies between a run's transcript write and its done record;
  • the done record fails to be written in that same window.

  A graceful exit (SIGTERM, stdin EOF, the reaper) writes that record first, so it never duplicates. Two things the guarantee does not cover: events a server emits while no subscription exists, and events it emits during a graceful restart for a type with no cursor replay. Neither was ever acknowledged.

• State files: two files beside the transcript.

  • <session>.mcp-events.json is a small snapshot of cursors, dedup windows, the webhook callback and server keys.
  • <session>.mcp-events.log is an append-only pending log. A commit costs the size of the change, not of the queue, and the log is compacted when it is mostly dead records.
  • A compaction that fails puts back the records it displaced.
  • A torn tail from a crash mid-append is truncated and synced when the log is read, before anything is appended.
  • The queue is bounded by count (1000) and by stored bytes (16 MiB).
  • Nothing is written while there is nothing to persist.
  • On new_session/switch_session the state moves before the next command is read, merged with whatever the target already had.

• Everything enters through the session's own command channel. Injections are ordinary prompt commands tagged with an mcp_events batch number. They go through a weak sender, so EOF still ends the session. When the loop is busy they are deferred, never refused.

• Events never start unbounded runs.

  • Events are deduplicated by eventId.
  • At most one injection goes in per 2 s window, carrying at most 50 events.
  • follow_up events are held while a run is in flight.

• Webhook receiver:

  • Routing: a random token path. The token and the last confirmed secret persist across restarts, so a delivery the server retries across a crash still lands.
  • After a restart: the persisted tokens of configured subscriptions are held from startup. A retry that arrives before its subscription re-registers gets 503, never 410. A runtime subscription is not resubscribed after a restart, so its old callback answers 410.
  • Graceful shutdown: the subscription is unsubscribed. An event type that can't be replayed from a cursor therefore loses what it emitted during the downtime.
  • Secret rotation: a new secret on every refresh. The previous secret is accepted; nothing older is.
  • Checked before the body is read: token, method, Content-Length and a 1 MiB cap. The body read itself has a deadline (408).
  • Signatures and identity: only 8 webhook-signature entries are examined. X-MCP-Subscription-Id is required once the subscription id is known.
  • Service mode: the route does not exist.

• v1a fails closed. Before each subscribe and refresh, keys come from <origin>/.well-known/mcp-webhook-jwks.json.

  • The fetch has one deadline over headers and body.
  • Keys ever seen for an origin persist, and the set only grows.
  • If a first fetch fails while no key is known, it is retried 3 times and then the subscription is refused.
  • Only a 404/410 means "not published".
  • A subscription task that overstays its cancellation is aborted, which removes its route.

• Push router: each stream's queue is bounded. On overflow it drains what it has and reconnects from the last delivered cursor, so nothing is lost silently. The SSE reader's scan and drain are both linear.

• Re-discovery: a -32011/-32014 termination of a runtime subscription re-discovers and resubscribes, up to 5 times. That budget is restored after 10 healthy minutes. Any resubscribe after a termination tells the model that events may have been missed.

The stdio transport changes for every stdio MCP server

rmcp 3.x decodes every response into an untagged union in which CallToolResult matches any object with _meta. That turned 2026-07-28 custom results into empty tool results. So all stdio MCP servers, not only those with events, now go through tools/mcp_stdio.rs:

• Spawning: the server runs in its own process group with stderr inherited.

• Reading stdout: a non-UTF-8 line is skipped.

• Rescue: a byte-level pre-check means only responses that could be misdecoded are parsed and rescued (rescue/unwrap_rescued, behind one boundary).

• Shutdown: mcp_stdio::retire runs in three places: when the transport drops, when the connection drops (McpConnection's Drop, the idle reap), and for every server still running on the way out of the process (retire_all). Each time it:

  1. closes the server's stdin at once. This is synchronous: the child and its stdin are shared, not owned by a task.
  2. gives it a 3 s grace, measured by a non-blocking wait on the child itself (portable, and never answered by a recycled pid).
  3. kills it if it is still running.
  4. kills the rest of its process group by group only, with no kill-by-pid fallback.

  The waiting and killing run on a tracked OS thread. Every way out of run and serve after the servers are up waits for those sweeps, bounded by the grace plus 2 s:

  • each process::exit goes through exit_process: stdin EOF, SIGTERM, a cancelled or refused run, and every configuration error found after connecting;
  • main returning, or a panic unwinding out of it, passes the ExitSweep guard's Drop.

  Only argument-parsing failures skip the sweep, and they happen before anything is spawned.

  This costs exit time, by design. A process that ran a stdio server exits in about 166 ms instead of 44 ms when the server exits on EOF, because each server gets the chance to exit cleanly and is then swept. When a server ignores EOF and has to wait out its grace, exit takes up to about 3.15 s.

  This restores main's exit-time sweep, which earlier pushes of this branch had regressed, for both serve and run. A double-forked grandchild no longer outlives the agent. A hard-killed agent still sweeps nothing; that needs a supervisor.

• A pre-existing bug, fixed in passing: exec::kill_process_group's group kill now passes --. Without it, procps-ng kill still killed every member of a live group, but its exit status was inverted: 1 on success, 0 for a group already gone. The by-pid fallback therefore ran after every successful group kill, which is exactly the pid-reuse hazard. The group signal itself always worked. An earlier version of this description said the group kill signalled nothing; that was wrong.

Round 4 findings — all fixed

#	finding	proving test	fails before
N2	(regression vs main) a panic unwound past the sweep	unit mcp_stdio::a_panic_unwinding_past_the_exit_guard_still_sweeps: panics after a stdio server is up and finds no orphaned grandchild	yes, when the guard only retires without waiting. No end-to-end test: main has no panic hook to drive.
N1	config-error exits after MCP connect bypassed the sweep	mcp_stdio_transport::a_run_refused_for_bad_configuration_after_connecting_still_sweeps (--output-schema)	yes, with the raw exit(2) restored. All five such exits (--lifecycle-url, --approve, memory backend, --output-schema ×2) now use exit_process.
N4	steered batches matched by exact text; tokens held for runtime subscriptions	unit mcp_events::a_steered_batch_is_found_by_its_id_not_its_content; mcp_events_receiver::after_a_restart_a_runtime_subscriptions_old_callback_is_gone_not_held	the receiver test fails with the configured-key filter removed. Under exact-text matching, two identical batches are indistinguishable, so the unit test's second assertion fails by construction. The compaction duplicate is documented.
N3	exit latency	by design. Documented in ARCHITECTURE.md and above.	—

Round 3 findings — all fixed

"Fails before" means the test was run against df9ce79, or the fix was reverted in place, and the test failed.

#	finding	proving test(s)	fails before
R1	(regression vs main) agent run orphaned stdio grandchildren on exit	mcp_stdio_transport::run_exiting_sweeps_its_stdio_servers_groups_after_their_grace	yes, on df9ce79
R2	a torn log tail swallowed the next acknowledged event	unit state::a_torn_log_tail_does_not_swallow_the_next_event	yes, with the truncation reverted
R3	settle() was untested	removed: retire closes stdin synchronously, so nothing needs the runtime on the way out. Its replacement on that path, retire_all, is proven by mcp_stdio_transport::a_run_stopped_mid_turn_still_sweeps_its_stdio_servers. The exit wait itself is proven too: removing it fails all four exit tests.	yes (on df9ce79; and without retire_all)
R4	/proc-based liveness meant no grace on macOS	now a non-blocking wait on the owned child. Unit mcp_stdio::retire_gives_the_grace_then_kills_a_server_that_ignores_stdin.	no on Linux, where /proc worked. The defect is macOS-only and CI has no macOS runner.
R5	unsubscribing a down configured key got it resubscribed	mcp_events_routing::unsubscribing_a_configured_subscription_while_its_server_is_down_stops_the_retries	yes, on df9ce79 and with the check reverted
R6	an early retry after a restart got 410	mcp_events_receiver::a_retry_that_beats_the_resubscribe_after_a_restart_is_told_to_retry_not_to_stop	yes, on df9ce79
R7	a failed compaction dropped its displaced records	unit state::a_failed_compaction_loses_no_records	yes, with the restore reverted
R8	kill-by-pid fallback after the leader was reaped	unit mcp_stdio::the_sweep_after_a_reap_never_kills_by_pid: a non-leader standing in for a pid-reuser survives the sweep, while the fallback path kills it	see note

R8 note. The R8 test's stand-in is not a group leader, so its pid names no group. Without --, kill exits 0 for such a group, the fallback is skipped, and the old code passes there; on a live group the fallback did run. With -- the status is correct, and the test shows the fallback path would kill the stand-in while the group-only sweep leaves it alone.

Re-audit findings (round 2) — all fixed

"Fails before" means the test was run against the previous head, 295dea6, with this push's fixture and test helpers, and failed there.

#	finding	proving test(s)	fails before
N1	an aborted steer was counted as delivered	mcp_events_durability::a_steered_event_an_abort_drops_is_injected_again	yes
N2	(regression vs main) no group sweep or grace on serve exit	mcp_stdio_transport::serve_exiting_on_eof_sweeps_its_stdio_servers_groups_after_their_grace, ::serve_exiting_on_sigterm_…	yes, both
N3	configured subscriptions were tried once	mcp_events_routing::configured_subscriptions_that_fail_at_boot_are_retried_and_kept_alive, ::a_configured_subscription_the_server_ends_is_resubscribed	yes, both
N4	JWKS body unbounded in time; leaked task and route	mcp_events_v1a::a_jwks_body_that_stalls_is_timed_out_and_leaves_nothing_behind	yes
N5	queue bounded by count only; whole-queue rewrite per commit	mcp_events_receiver::the_pending_queue_is_bounded_in_bytes_and_answers_503_past_it; unit state::commit_cost_tracks_the_change_not_the_queue, ::the_pending_queue_is_bounded_in_bytes, ::the_log_is_compacted_once_mostly_dead	yes (e2e)
N6	durability failed open	mcp_events_receiver::a_delivery_that_cannot_be_stored_gets_503_and_lands_on_retry; unit state::a_failed_write_is_reported_and_retried	yes (e2e)
N7	delivered() despite a persist error or failed run	mcp_events_durability::an_event_whose_run_fails_is_injected_again_up_to_a_bound	yes
N8	empty sidecar without events	mcp_events_poll::a_session_without_events_writes_no_events_state; unit state::nothing_is_written_while_there_is_nothing_to_persist	yes (e2e)
N9	shutdown could wait on a blocked coalescer forever	unit mcp_events::tests::the_coalescer_ends_on_shutdown_even_while_its_send_is_blocked	yes
N10	duplicates-only hasMore page spun	mcp_events_poll::has_more_with_only_duplicates_does_not_spin	yes
N11	relocate only switched the path	mcp_events_durability::the_events_state_follows_the_session_to_a_new_transcript (now asserts a move); unit state::relocating_merges_into_the_target_and_removes_the_old_files	yes (e2e)
N12	re-discovery budget never reset; gap claim not implemented	mcp_events_push::rediscovery_tells_the_model_about_the_gap_and_its_budget_recovers	yes
N13	subscription id optional, body read unbounded in time, unlimited signature entries, quadratic SSE drain, token changed on restart	unit webhook::the_receiver_enforces_token_freshness_signature_and_echoes_challenges (missing id → 401), webhook::signature_entries_past_the_cap_are_ignored, wire::many_events_in_one_chunk_drain_in_linear_time (4.4 s vs 0.26 s); mcp_events_receiver::a_delivery_body_that_never_finishes_is_timed_out, ::the_callback_survives_a_crash_so_retried_deliveries_still_land	yes, all

About the "fails before" column:

• Unit tests marked "(e2e)" exercise the new state.rs API, which did not exist before, so they can't be run against 295dea6. The end-to-end test in the same row is the before/after proof.
• mcp_events_routing::a_session_whose_only_subscription_terminated_becomes_reapable was rewritten to use a runtime subscription, because configured ones are now kept up (N3). It passes on both heads and stays as the m21 guard.

Mutations from round 1 are still caught after this push:

mutant	caught by
m3	a_future_dated_or_absurd_timestamp_is_refused_without_harm
m4	rotation_grace_accepts_exactly_the_previous_secret_and_refuses_older_ones
m11	unit the_coalescer_holds_follow_ups_while_busy_and_releases_them_after
m18	poll_does_not_advance_its_cursor_past_events_it_could_not_accept
m21	a_session_whose_only_subscription_terminated_becomes_reapable

These were applied by script. The CI mutants job does not cover this crate: its "changed lines" shards pass in about 30 s without mutating crates/agent.

Round 1 (F1–F12, at 295dea6) remains fixed. Each of those findings is proven by its test listed in that commit's message.

Proof

Everything runs the real serve (and run) binary against src/bin/mcp_fixture_events_server.rs (stdio and streamable HTTP, with a control API) and a mock model. There are 60 end-to-end tests:

suite	tests
mcp_events_poll	10
mcp_events_webhook	9
mcp_events_push	6
mcp_events_routing	6
mcp_events_durability	5
mcp_events_receiver (new)	6
mcp_events_lifecycle	4
mcp_events_v1a	4
mcp_stdio_transport	7
mcp_events_busy	3

There are also 34 unit tests across tools::mcp_events::{mod,state,webhook,wire} and tools::mcp_stdio.

Checks run on this head:

• MCP tests: every tests/mcp_*.rs suite is green, 19 binaries including mcp_reap. The lib tests are green (1471).
• Other suites: serve_reaper, serve_http, serve_drain, run_signal_handling and run_cli_flags are green.
• Lint and format: cargo clippy -p beyond-ai-agent --all-targets --features code-mode -D warnings, a workspace clippy, cargo fmt --check and dprint check are all clean.
• CI: green on 79ff913, every job including agent-lib, agent-rest and agent-code-mode (4–5 min). It was also green on 295dea6.
  • On df9ce79 those three shards were cancelled at exactly 30 min. jared/mcp-tasks-proof, which does not contain this code, was cancelled in the same window. This is environmental and being investigated separately.
  • The same shards run clean locally under the CI nextest profile: lib 1515 tests in 39 s, rest 440 in 45 s, with no SLOW, LEAK or TIMEOUT.
  • Every wait this PR adds is bounded.

Independent check (not in CI)

crates/agent/eval/mcp_events_interop/check.py runs the real agent against mcp-webhook-events 0.2.0 on the official Python MCP SDK (mcp 2.3.0). Deliveries are signed by the official standardwebhooks library, and the package does send X-MCP-Subscription-Id.

transport to that server	result
streamable HTTP	12/12
stdio	12/12

Remaining caveats

• Draft wire: it will move. Field names follow the pinned sketch and OpenAI's guide.
• v1a key source: the standalone JWKS is implemented. SEP-2127 server cards don't exist yet.
• Hard crash: a crash between a run's transcript write and its done record re-injects that batch once. That is at least once, not exactly once.
• Agent hard-kill: if the agent itself is hard-killed, stdio servers' process groups are not swept. That needs a supervisor.
• R4: the portable liveness check is proven on Linux only. CI has no macOS runner.
• Emit-only event types: across a graceful restart they lose what they emitted during the downtime, because the subscription was unsubscribed and the server has no cursor to replay from.

🤖 Generated with Claude Code

https://claude.ai/code/session_01JimHGjsfk2Ktm5GxyZJKKk