Skip to content

fix(graph-node): wire NAPI binding to persistent storage (hydrate on open, edge/hyperedge write-through, delete cascade) - #810

Open
Bolivar1234 wants to merge 2 commits into
ruvnet:mainfrom
Bolivar1234:fix/graph-node-persistence
Open

fix(graph-node): wire NAPI binding to persistent storage (hydrate on open, edge/hyperedge write-through, delete cascade)#810
Bolivar1234 wants to merge 2 commits into
ruvnet:mainfrom
Bolivar1234:fix/graph-node-persistence

Conversation

@Bolivar1234

Copy link
Copy Markdown

Fixes #809.

What

Wires the ruvector-graph-node NAPI binding to the persistence the ruvector-graph crate already provides, so storagePath graphs survive process restarts.

  • Hydration: new()/open() now replay storage into both the hypergraph (traversal index) and the property graph — nodes, binary edges, and multi-node hyperedges. __embedding/__confidence are round-tripped as FloatArray properties so embeddings survive too.
  • Write-through: createEdge, createHyperedge, and batchInsert edges now persist via storage.insert_edge/insert_hyperedge and mirror into the property graph (previously they only touched the in-memory hypergraph).
  • Deletes: deleteEdge also cleans the hypergraph traversal index; deleteNode cascades through persisted edges/hyperedges touching the node, so a later hydrate never replays dangling references.
  • Tolerant hydrate: dangling records (e.g. written by a pre-fix binary) are skipped and counted with a stderr report instead of failing open().

One file changed: crates/ruvector-graph-node/src/lib.rs (+227/−9). No crate/API changes; existing in-memory (no storagePath) behavior is untouched.

Verification

12-gate suite, each phase in its own process because redb is single-writer:

Gate Result
create nodes+edge / multi-node hyperedge / batchInsert / tx / Cypher label scan PASS
fresh-process reopen (exact stats) PASS
reopen kHop over binary, hyper and batch edges PASS
deleteEdge / deleteNode(cascade) persist across reopen PASS
in-memory mode unaffected PASS
single-writer lock still enforced on concurrent open PASS

Scale check: a 154,711-node / 353,496-edge code graph (12 repos) reopens with exact counts after restart (hydrate ~15s on a shared host). Before this patch the same reopen returned {totalNodes: 0, totalEdges: 0}.

Happy to move hydration into the crate instead of the binding if you prefer that layering, or to add the test suite as a __test__ file in the package.

The ruvector-graph crate has full redb-backed persistence
(GraphDB::with_storage / load_from_storage), but the NAPI binding never
used it, so @ruvector/graph-node silently lost all data on restart:

- GraphDatabase.open() returned an empty graph (never hydrated)
- createEdge / createHyperedge / batchInsert edges never reached storage
- deleteNode left dangling persisted edges that broke later hydration

Changes:
- new()/open() hydrate hypergraph + property graph from storage
  (nodes, binary edges, multi-node hyperedges; __embedding/__confidence
  round-tripped as FloatArray properties)
- createEdge / createHyperedge / batchInsert write through to storage
- deleteEdge also cleans the hypergraph traversal index
- deleteNode cascades through persisted edges/hyperedges touching the node
- hydrate tolerates dangling records (skips + reports to stderr) instead
  of failing open()

Verified with a 12-gate suite (create/reopen/delete/batch/tx/cypher/
kHop/in-memory/single-writer lock) and a 154,711-node / 353,496-edge
graph that now survives process restart with exact counts.
PropertyValue::from(Vec<f32>) resolves to the generic From<Vec<T>> impl and
produces Array(Float(f64)...), never FloatArray. Hydrate matched only
FloatArray, so every node/edge/hyperedge replayed with an empty embedding
and searchHyperedges returned f32::MAX sentinel scores for all results
after a restart (traversal was unaffected, which is why kHop gates passed).

- persist sites now write PropertyValue::FloatArray explicitly
- hydrate extracts via a helper tolerant of FloatArray AND Array/List,
  so databases written before this commit remain readable
- verified: 13-gate suite including a new 'vector search survives reopen'
  gate; real-workload check returns identical scores pre/post restart
@Bolivar1234

Copy link
Copy Markdown
Author

Pushed a follow-up commit (bbceebb) fixing a bug in my original patch: PropertyValue::from(Vec<f32>) resolves to the generic From<Vec<T>> impl and produces Array(Float(f64)...), never FloatArray — so hydrate (which matched only FloatArray) replayed every embedding as empty, and searchHyperedges returned f32::MAX sentinel scores for everything after a restart. Traversal was unaffected, which is why the kHop gates missed it.

Fix: persist sites write PropertyValue::FloatArray explicitly, and hydrate uses a tolerant extractor that also reads the boxed Array/List form, so databases written by the earlier revision stay readable. Verified with a new 'vector search survives reopen' gate (13/13 passing) plus a 16K-node real workload returning identical scores pre/post restart.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

@ruvector/graph-node: storagePath data silently lost on restart (open() never hydrates, edges never persisted)

1 participant