lightningd: add watchman module#9202
Merged
Merged
Conversation
watchman is the lightningd-side counterpart to the bwatch plugin landed in the previous group: it tracks how far we've processed the chain, queues outbound watch ops while bwatch is starting up, and dispatches watch_found/watch_revert/blockdepth notifications to subdaemon-specific handlers. This commit adds only the public surface (struct watchman, the three handler typedefs, and prototypes for watchman_new, watchman_ack, watchman_replay_pending) plus an empty watchman.c so the header is exercised by the build. Definitions land in subsequent commits.
Introduce the minimal storage scaffolding for the watchman module: - db_set_blobvar / db_get_blobvar helpers for persisting binary values (e.g. block hashes) in the SQL `vars` table. - load_tip(): recover last_processed_height and last_processed_hash from the wallet db. - apply_rescan(): honour --rescan by adjusting the loaded tip downward (negative = absolute height, positive = N blocks back). - watchman_new(): allocate the struct, initialise the pending-op array, and call load_tip + apply_rescan. Wire the watchman field into struct lightningd via a forward declaration; instantiation at startup lands in a later commit along with the rest of the wiring.
sangbida
changed the title
nGoline
reviewed
Jun 9, 2026
nGoline
left a comment
nGoline
left a comment
Collaborator
There was a problem hiding this comment.
Went deeply through the code to get a gist of how it works. Ran it locally and performed some tests before reviewing.
Comment on lines
+160
to
+161
| /* Optional callback invoked whenever a plugin reaches INIT_COMPLETE. */ | ||
| void (*on_plugin_ready)(struct lightningd *ld, struct plugin *plugin); |
Collaborator
There was a problem hiding this comment.
Thinking about the extensibility of this field. watchman writes to it here, but any future subsystem that also needs a plugin-ready callback would silently overwrite this one.
Around the codebase any one-to-many notification is a list. If this hook is intended to be generic, it should be a LIST_HEAD of callbacks, but if this is only ever for watchman, it should not live in struct plugins at all, watchman should check on_plugin_config_cb directly or hook into an existing notification mechanism. (do we have one?)
madelinevibes
added
the
Status::Ready for Review
Introduce the outbound RPC path from watchman to the bwatch plugin
plus the ack lifecycle that drops a pending op once bwatch confirms
it.
- struct pending_op carries an op_id of the form "{method}:{owner}"
(e.g. "addscriptpubkeywatch:wallet/p2wpkh/42"); method and owner
are recoverable without a separate field.
- Datastore helpers (make_key, db_save, db_remove) persist pending
ops at ["watchman", "pending", op_id] for crash recovery.
- send_to_bwatch finds the bwatch plugin via find_plugin_for_command
on the method name; if bwatch is not yet INIT_COMPLETE, the send is
silently dropped (the op stays queued and will be replayed when
bwatch comes up). Otherwise it builds a JSON-RPC request with the
owner suffix and the caller-supplied json_params body, registers
bwatch_ack_response as the callback, and sends it.
- watchman_ack searches pending_ops by op_id; on a hit it removes the
datastore entry and drops the in-memory op.
db_save and send_to_bwatch are marked __attribute__((unused)) here
because their callers (enqueue_op, watchman_replay_pending) land in
the next commit; the markers are removed there.
Both bwatch and watchman must be crash-resistant: a watch_send or an add_watch/del_watch op may be in flight when lightningd crashes, and neither side is allowed to lose it. We solve this by persisting every pending op to the datastore in enqueue_op and dropping it from the datastore in watchman_ack. On startup load_pending_ops rebuilds the in-memory queue from the datastore, and watchman_on_plugin_ready replays it once bwatch reaches INIT_COMPLETE. watchman_add cancels any prior add for the same owner; watchman_del cancels any pending add for the same owner before queueing the delete. This keeps the queue from accumulating stale or self-cancelling op pairs across restarts.
Register the two startup RPCs that bwatch calls on launch:
- getwatchmanheight: bwatch asks how far we've already processed the
chain so it knows what height to (re)scan from. Returns
{height, blockhash?} from wm->last_processed_{height,hash}.
- chaininfo: bwatch reports the chain name, header/block counts, and
IBD status. We fatal() on a network mismatch (wrong bitcoind),
toggle bitcoind->synced based on IBD/header-vs-block lag, fire
notify_new_block on the transition to synced, and remember the
blockcount on watchman.
Register watch_found, watch_revert, block_processed, revert_block_processed, getwatchmanheight, and chaininfo handlers. Add typed watchman_watch_* helpers and the dispatch-table skeleton for future subsystem handlers. Changelog-Added: lightningd: watchman module registers getwatchmanheight, chaininfo, watch_found, watch_revert, block_processed and revert_block_processed RPCs for bwatch plugin integration.
Create watchman after setup_topology so it can queue bwatch RPC requests and replay pending ops once the plugin is ready. No subsystems register watches yet — that lands in the wallet migration PR.
sangbida
force-pushed
the
sangbida/watchman
branch
from
0e89962 to
70c1924
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part 2 of the bwatch split series. No watches are registered yet —that comes in later PRs. @rustyrussell suggested splitting the original bwatch PR into smaller, independently bisectable pieces rather than turning off all tests and turning them back on.
watchman is the
lightningd-side counterpart to thebwatchplugin. It acts as the watch dispatcher: subsystems register scriptpubkey/txid watches with watchman, which forwards them to bwatch and fans out notifications (watch_found, watch_revert, block_processed, etc.) back to the relevant subsystems.