feat(blockchain): add chain-event bus (head/block/justified/finalized) - #516
Conversation
🤖 Codex Code ReviewFindings
No other material correctness, security, or Rust-pattern issues stood out in the diff itself. I could not run Automated review by OpenAI Codex · gpt-5.4 · custom prompt |
🤖 Kimi Code ReviewOverall this is a clean, well-architected PR. The event bus design correctly prioritizes consensus liveness over delivery guarantees, and the integration points are surg Automated review by Kimi (Moonshot AI) · kimi-k2.5 · custom prompt |
Greptile SummaryThis PR adds chain-event publishing to the blockchain actor. The main changes are:
Confidence Score: 5/5No additional blocking issue was found in this follow-up.
|
| Filename | Overview |
|---|---|
| crates/blockchain/src/events.rs | Adds event types, topic mapping, the broadcast bus, snapshot diffing, and focused unit tests. |
| crates/blockchain/src/lib.rs | Publishes state changes around tick and block processing and exposes the event API. |
| bin/ethlambda/src/main.rs | Constructs the event bus and passes it to the blockchain actor. |
| crates/blockchain/Cargo.toml | Moves Serde into runtime dependencies for event serialization. |
Reviews (4): Last reviewed commit: "Merge branch 'main' into feat/chain-even..." | Re-trigger Greptile
🤖 Claude Code ReviewReview: PR 516 — chain-event busOverall this is clean, well-scoped work. Threading Three points worth a look, none blocking:
No fork-choice/justification/finalization logic changes here — the PR is additive observability around existing store semantics, and the diffs read faithfully against what's actually mutated (e.g., the Automated review by Claude (Anthropic) · sonnet · custom prompt |
🤖 Kimi Code ReviewOverall this is a clean, well-documented addition that correctly implements a best-effort event bus without affecting consensus safety. A few specific notes: Code correctness
Safety & performance
Automated review by Kimi (Moonshot AI) · kimi-k2.5 · custom prompt |
🤖 Codex Code ReviewFindings
Assumptions / gaps
Summary Aside from Point 1, the event plumbing looks disciplined: single-writer bus, non-blocking broadcast, and the Automated review by OpenAI Codex · gpt-5.4 · custom prompt |
🤖 Claude Code ReviewReview:
|
23825f1 to
3a3c804
Compare
🤖 Kimi Code ReviewOverall this is a well-architected PR. The snapshot-diff pattern cleanly separates observability from consensus logic, and the best-effort broadcast design correctly prioritizes liveness over event delivery guarantees. Correctness & Safety
Code Quality
Testing
Nits
The PR is safe to merge. The snapshot/diff approach is robust and keeps the store code path clean of event plumbing. Automated review by Kimi (Moonshot AI) · kimi-k2.5 · custom prompt |
🤖 Codex Code ReviewFindings:
No consensus-state-transition, attestation-validation, SSZ, or XMSS handling regressions stood out in the touched code beyond the event-stream consistency issues above. I couldn’t run Automated review by OpenAI Codex · gpt-5.4 · custom prompt |
🤖 Claude Code ReviewReview:
|
3a3c804 to
a966632
Compare
First PR of the chain-events series (re-cut of #460), shipping only the pub-sub mechanism so the real design decisions (event shape, publisher model, back-pressure policy) are reviewable on their own. The SSE transport and topic filtering follow in separate PRs; the bus stays unconsumed until the endpoint lands. Emission is actor-layer only, not threaded through the store. Before each state-changing store call (store::on_tick, store::on_block) the actor snapshots (head, latest_justified, latest_finalized), runs the store function completely unchanged, then diffs the snapshot against the post-call state and emits in order block -> justified_checkpoint -> head -> finalized_checkpoint. This keeps store.rs, both spec-test files, and test_driver.rs byte-identical to main: no Option<&EventBus> parameter spreads across on_tick/on_block/update_head or any of their test call sites, and the sole-publisher property is structural (only BlockChainServer ever holds a live EventBus) rather than something every call site has to preserve by convention. Two consequences of diffing at this level, both accepted: - `block` needs a was-it-new guard: on_block returns Ok early for an already-imported root, so the actor checks block-known-ness (the same has_state check the store itself uses) before importing, and only emits `block` for a genuinely new root. - Several head moves inside a single tick or block import coalesce into one `head` event carrying the net move; fine for subscribers, since the beacon-API analog is also last-value-wins per slot. The bus itself is a facade over a single bounded tokio broadcast channel: emit() never blocks or fails, since a slow subscriber lags and drops instead of back-pressuring consensus, and a receiver-count guard makes emission free when nobody is listening. ChainEvent serializes untagged so payloads stay flat: the topic name travels out-of-band via ChainEvent::topic(), fixing #460's double-tagged SSE payload before any transport exists. The `head` event is additionally gated by HEAD_EVENT_RECENCY_SLOTS, adopted from a review of Lighthouse's canonical-head recompute path: a new head is only emitted when its slot is within that many slots of the wall clock, so startup catch-up and backfill don't spam subscribers with historical head moves on the way to the tip. `block`, justified_checkpoint, and finalized_checkpoint stay ungated; consumers tracking sync progress should watch `block` instead.
a966632 to
60d9c2e
Compare
…nd_emit Reorder chain-event emission so the head event fires before justified_checkpoint, giving subscribers the head move ahead of the checkpoint updates that follow it. Drop the ordering assertion test, which only pinned the old sequence.
Colocate the pre-call snapshot type (and HEAD_EVENT_RECENCY_SLOTS) with the ChainEvent/EventBus definitions it depends on, keeping all chain-event machinery in events.rs. The actor references it via crate::events; the snapshot's diff/gate tests move alongside it. No behavior change.
The recency gate is already covered by chain_event_diff_gates_stale_head (gated) and chain_event_diff_emits_recent_head (within window); the explicit boundary case was redundant.
Eventless call paths (spec tests, test_driver.rs) never construct an EventBus at all, so the dormant-bus constructor had no production callers; its only user was its own test.
🤖 Kimi Code ReviewOverall Assessment: Good PR. The event bus is correctly designed as a best-effort, non-blocking observer that doesn't interfere with consensus logic. The snapshot/diff pattern cleanly separates event emission from store internals. Issues & Suggestions: 1.
2. Panic risk in snapshot capture (
3. Minor race in 4.
5. Redundant wall-clock calculation ( 6. Serde dependency move ( Positive Notes:
Automated review by Kimi (Moonshot AI) · kimi-k2.5 · custom prompt |
🤖 Codex Code Review
Open questions / assumptions:
Aside from that, I did not find a consensus-correctness or security regression in the new event ordering itself. I could not run Cargo tests in this environment because the Automated review by OpenAI Codex · gpt-5.4 · custom prompt |
🤖 Claude Code ReviewReview:
|
Shape ChainEvent payloads after the Ethereum beacon-API eventstream (v4.0.0): `block` is the block root, `state` the state root, and `slot` stands in for the beacon `epoch`. Drop the `#[serde(untagged)]` Serialize derive; the wire encoding (decimal-string ints, 0x-hex roots, out-of-band topic) returns with the SSE endpoint in a follow-up PR. This also removes the ambiguous untagged shape flagged in review. Since serde is no longer used in the crate, drop the serde/serde_json deps. `justified_checkpoint` has no beacon analog and is kept as an ethlambda extension mirroring `finalized_checkpoint`. Also fix a checkpoint event dropped on the proposer's own slots: the interval-0 catch-up in `get_proposal_head` advances head/finalized outside every snapshot window, so a finalization moved there was folded into the later block-import diff's baseline and never emitted. `propose_block` now snapshots around the build and emits those moves on both build success and failure, matching what a non-proposing node emits at its interval-0 tick.
#516 reshaped ChainEvent (root/parent_root -> block/state) and dropped its Serialize derive; move the wire encoding into an RPC-crate DTO.
#516 deferred the chain-event wire encoding to this SSE PR. Derive it on ChainEvent directly (untagged, flat body) rather than mirroring the enum in an RPC-crate DTO, so the handler stays a direct json_data(&ev) with no parallel type to keep in sync. Wire shape is unchanged.
Adds `GET /lean/v0/events`, a Server-Sent Events stream of the chain events introduced in #516: `head`, `block`, `justified_checkpoint`, `finalized_checkpoint`. Every subscriber currently receives all events; server-side `?topics=` filtering lands in the next PR of the series. Frames carry the topic on the SSE `event:` line and a flat JSON payload in `data:` (built from `ChainEvent::topic()`, no hardcoded name mapping). A client that reads too slowly skips the events it missed (debug-logged, best-effort; re-sync via the blocks endpoints) rather than back-pressuring the actor. Keep-alive comments hold idle connections open. Documented in `docs/rpc.md`. Stacked on #516. Has handler tests and passes clippy with `-D warnings`.
Continues the chain-event pub-sub series (#516 / #517 / #518, all merged). Adds the beacon `block_gossip` analog. ## Event added | Topic | Payload | Emitted when | |-------|---------|--------------| | `block_gossip` | `{ slot, block }` | A block is seen on the network, before import | ## Notes - Emitted from the blockchain actor's `NewBlock` handler, **not** a second P2P-side publisher, so the actor stays the sole event publisher and the write flow stays one-directional (a core requirement of the series' design). - Fires for both gossip'd and by-root-fetched blocks, since both re-enter through `NewBlock`. Import may pend the block while its parent chain is fetched, so this precedes import. - Ungated like `block` (not recency-gated), so subscribers can watch sync progress. Low-rate (one per block), so the payload is built unconditionally behind `emit`'s no-subscriber guard. ## Testing `cargo fmt`, `cargo clippy -p ethlambda-blockchain -p ethlambda-rpc` (clean), blockchain lib + rpc events tests pass. --- **Independent PR, based off `main`.** One of three sibling PRs continuing the series — alongside #533 (`chain_reorg`/`safe_target`) and #534 (`attestation`/`aggregate`), each independently based off `main`. They touch overlapping regions of `events.rs` / `docs/rpc.md`, so whichever two merge later will each need a small conflict rebase. Opened as draft.
Continues the chain-event pub-sub series (#516 / #517 / #518, all merged). Adds the two high-rate topics. ## Events added | Topic | Payload | Emitted when | |-------|---------|--------------| | `attestation` | `{ validator_id, data: AttestationData }` | A single validator vote passes gossip validation (signature omitted) | | `aggregate` | `{ participants: [u64], data: AttestationData }` | A committee-signature aggregate is produced locally or accepted from gossip (proof omitted) | Both fire only for messages the store **accepted** (data + signature validation), so subscribers see the same votes fork choice does. A node never receives its own aggregate back over gossip, so `aggregate` fires at two sites: the locally produced `AggregateProduced` path and the gossip-received path. ## Design points - The ~3 KB XMSS signature and the SNARK proof bytes are deliberately **omitted** — too heavy for a high-rate stream. - The shared broadcast channel capacity is bumped **256 → 8192**. All topics share one ring buffer, so a subscriber's tolerable stall is `capacity / total_event_rate` (dominated by the attestation rate), not per-topic. A per-topic split behind the `EventBus` facade remains the escape hatch if real devnet rates show the shared window biting — see the note in `docs/rpc.md`. - Emission uses the plain, no-subscriber-guarded `emit`. Consequence: the per-vote `AttestationData` is cloned even when nobody is subscribed (the guard only skips the send). If that actor-hot-path cost proves material, a payload-deferral helper can be added in a follow-up. ## Testing `cargo fmt`, `cargo clippy -p ethlambda-blockchain -p ethlambda-rpc` (clean), blockchain + rpc lib tests pass, including `events_streams_attestation_with_nested_data` (verifies the untagged `data:` nesting over the wire). --- **Independent PR, based off `main`.** One of three sibling PRs continuing the series — alongside #533 (`chain_reorg`/`safe_target`) and #535 (`block_gossip`), each independently based off `main`. They touch overlapping regions of `events.rs` / `docs/rpc.md`, so whichever two merge later will each need a small conflict rebase.
## Summary Adds `tooling/event-monitor/`, a standalone live arrival-time dashboard for lean-consensus (ethlambda) nodes. It dials the `GET /lean/v0/events` SSE stream of several nodes, timestamps each event on a single collector clock, and serves a browser dashboard visualizing: - a **rolling beeswarm** of each event's arrival offset *within the slot*, one lane per node, and - a **propagation-delta beeswarm**: for a given block / aggregate, how long after the *first* node each other node saw it. The rolling window is adjustable live from the header, and a fresh page load backfills recent history from the collector so it is never blank. ## Design - **Standalone Cargo workspace** under `tooling/event-monitor/` (empty `[workspace]` table); it is not a member of the parent `ethlambda` workspace and does not affect the main build. - **Zero dependency on any ethlambda crate** — it only speaks the documented SSE/HTTP wire shape. `CONTRACT.md` is the frozen interface between the Rust/axum backend (`src/`) and the vanilla-JS frontend (`web/`, no build step). - A single collector clock makes propagation deltas skew-free across nodes. - `?demo=1` runs the frontend fully offline with synthetic data. This PR contains the tooling plus the CI wiring needed to actually check it. The RPC / event-bus changes that make a node emit these events shipped separately in the chain-events series (below). ## Required ethlambda API functionality All dependencies are **satisfied on `main`** — the default dashboard view works against a current node with no extra PRs. | Capability | Endpoint / topic(s) | Status | |---|---|---| | SSE events stream | `GET /lean/v0/events` | ✅ #517 | | Topic filtering | `?topics=<csv>` | ✅ #518 | | Slot-geometry bootstrap | `GET /lean/v0/genesis`, `GET /lean/v0/config/spec` | ✅ | | Base topics | `block`, `head`, `justified_checkpoint`, `finalized_checkpoint` | ✅ #516 | | Attestation / aggregate topics | `attestation`, `aggregate` | ✅ #534 | | Block-gossip topic | `block_gossip` | ✅ #535 | The collector accepts `block`, `block_gossip`, `head`, `justified_checkpoint`, `finalized_checkpoint`, `attestation` and `aggregate`; any other topic is logged and skipped. `safe_target` / `chain_reorg` (#533, closed) were removed in 4eeafff — both panels' topic filters discarded them, and `chain_reorg` is a point-in-time event rather than a per-node arrival race, so it does not belong on a beeswarm. Both panels default to `block` / `attestation` / `aggregate`. ## Review fixes Three defects found in review, each with a regression test: - **Reconnect backoff never reset after a healthy session.** The reset was keyed on a clean end-of-stream, but a node restart surfaces as a stream *error*, so every restart ratcheted the delay one step permanently; after ~7 of them a healthy node reported `down` with 10s reconnects. Now keyed on session duration (survived ≥1 heartbeat). This also closes the inverse case: a peer that accepts the request and instantly closes the stream used to be retried at `INITIAL_BACKOFF` forever. - **One bogus slot blanked the dashboard.** Both the history ring and the rolling window key retention off the highest slot seen, and that watermark only moves up, so a single event from a node on a different genesis aged out every real event until a restart. Events more than `MAX_FUTURE_SLOTS` ahead of the collector's own slot are now dropped, warning once per node per connection. Only the future side is bounded; old slots are legitimate (`finalized_checkpoint` trails head) and cannot move the watermark. - **Stale status clobbered fresh status.** The frontend applies `status` immediately but buffers `chain` during backfill, so the `/api/history` snapshot could overwrite a newer live status. Live status now wins. Also from that review: - **Slot geometry is re-resolved every 60s** and republished when it changes, dropping the retained history and slot watermark, so a regenerated genesis no longer silently corrupts every subsequent `offset_ms`. Collectors re-read per frame and `/api/meta` per request; an already-open tab needs a reload to pick up new geometry. - **The arrival axis now spans two slots at two scales**: the first at full resolution (90.9% of the width at 5 intervals), the next compressed into a tinted band half a first-slot interval wide, saturating beyond that. A block spilling past its slot boundary is the failure mode worth seeing, and it used to be indistinguishable from one landing exactly on the boundary. - **`offset_ms` includes the collector↔node round trip.** One clock is what makes propagation deltas skew-free, but nodes reached over different links carry a systematic offset that reads as lag. Now documented in `README.md` and `CONTRACT.md §2`. - History events are `Arc`-shared, so `publish_chain` no longer deep-copies per event and `/api/history` no longer clones up to `HISTORY_MAX_EVENTS` events while holding the mutex. - `topics = []` / `nodes = []` are rejected at load rather than becoming an opaque retry loop against a 400. Known and deliberate, left as follow-ups: the collector cannot see upstream `: error - dropped N messages` comments (`eventsource-stream` swallows them), so a lagging subscription silently under-samples; both canvases still repaint unconditionally at 60fps; and `bootstrap` has no retry, so the monitor exits if started before its nodes are listening. ## CI `tooling/event-monitor` declares its own `[workspace]`, so `cargo fmt --all`, `cargo check --workspace`, `cargo clippy --workspace`, `make lint` and `make test` all stop at the root workspace members and never reached it — it landed with nothing verifying it. The `Lint` job now also runs `cargo fmt --check`, `cargo clippy --all-targets -- -D warnings` and `cargo test` under `working-directory: tooling/event-monitor`, and `rust-cache` lists it as a second workspace so its target dir is cached and its `Cargo.lock` feeds the cache key. ## Testing - `cargo test` in `tooling/event-monitor/`: **41 passed** (39 lib + 2 integration), 0 failed. - `cargo fmt --check` and `cargo clippy --all-targets -- -D warnings`: clean. - `?demo=1` offline synthetic mode renders both panels with no live nodes. - Verified end-to-end against a live local devnet (blocks land ~0.1–0.2 s into the slot, attestations ~0.8 s = interval 1). - Two-slot axis geometry checked numerically: the overflow band is exactly 0.5 × one first-slot interval, the mapping is monotonic, and the endpoints land on the margins. - Geometry refresh checked against a stub node: rewriting its `genesis_time` mid-run is picked up within one refresh interval (`WARN slot geometry changed`), `/api/meta` then serves the new epoch, and the history ring holds the restarted chain's low slots with in-slot offsets — which only works because the watermark is reset, otherwise they would prune as older than the previous high-water mark. ## Run ```bash cd tooling/event-monitor cp config.example.toml config.toml $EDITOR config.toml # list your nodes' RPC URLs + topics cargo run --release -- --config config.toml # open the `listen` address (default http://127.0.0.1:8080) ```
Adds a chain-event pub-sub mechanism to the blockchain actor: a
ChainEventenum (head,block,justified_checkpoint,finalized_checkpoint) published through anEventBusfacade over a bounded tokio broadcast channel. The actor is the sole publisher; subscribers attach their own receivers, and a slow consumer skips missed events instead of back-pressuring the actor. Near-zero cost with no subscribers (receiver_countguard). No HTTP surface in this PR: the RPC consumer arrives in the next PR of the series.Emission is actor-layer only, not threaded through the store. Before each state-changing store call (
store::on_tick,store::on_block)BlockChainServersnapshots(head, latest_justified, latest_finalized), runs the store function completely unchanged, then diffs the snapshot against the post-call state and emits in orderblock→head→justified_checkpoint→finalized_checkpoint.store.rs, both spec-test files, andtest_driver.rsstay byte-identical tomain: noOption<&EventBus>parameter spreads acrosson_tick/on_block/update_heador any of their test call sites, and the sole-publisher property is structural (onlyBlockChainServerever holds a liveEventBus) rather than something every call site has to preserve by convention.Two consequences of diffing at this level, both accepted:
blockneeds a was-it-new guard:on_blockreturnsOkearly for an already-imported root, so the actor checks block-known-ness (the samehas_statecheck the store itself uses) before importing, and only emitsblockfor a genuinely new root.headevent carrying the net move; fine for subscribers, since the beacon-API analog is also last-value-wins per slot.The
headevent is additionally recency-gated: it only fires when the new head is withinHEAD_EVENT_RECENCY_SLOTS(32) of the wall-clock slot, so catch-up/backfill never spams head events (adopted from Lighthouse's head-SSE recency filter); the other events stay ungated.Payload serialization is untagged, so the JSON body stays flat and topic names live only on the transport's
event:line.Supersedes #460, re-cut as a stacked series for easier review:
GET /lean/v0/eventsSSE endpoint (all events)?topics=server-side filteringchain_reorg+safe_targeteventsattestation/aggregateeventsHas unit tests and passes clippy with
-D warnings.