Skip to content

fix(#420): JVM new_edges sends a subnormal edge property down the per-edge path - #422

Open
shaunpatterson wants to merge 1 commit into
ArcadeData:mainfrom
shaunpatterson:fix/new-edges-subnormal-property
Open

shaunpatterson wants to merge 1 commit into
ArcadeData:mainfrom
shaunpatterson:fix/new-edges-subnormal-property

Conversation

@shaunpatterson

@shaunpatterson shaunpatterson commented Nov 21, 2026 •

Copy link
Copy Markdown

What does this PR do?

GraphBatch.new_edges wrote the rows of the untagged branch (no date-time, date, Decimal, bytes, list or dict value) with plain json.dumps. The engine's JSON parser reads the text of a subnormal double such as 5e-324 as a BigDecimal, so the edge property was stored as a DECIMAL and read back as decimal.Decimal. The other bulk paths refuse such rows through _dumps_checked and fall back to the per-value path, which stores the double.

The untagged branch now uses json_bulk_dumps, as the tagged branch uses json_bulk_dumps_tagged, and the two share one block: a ValueError starts the per-edge path over before anything is buffered.

Related issues

Closes #420

Performance changes

new_edges with one float property on 300,000 edges, three runs each, old and new interleaved: 1.49, 1.41, 1.32 s before and 1.37, 1.37, 1.46 s after. The differences are within the run-to-run spread; the check costs a substring search of the JSON text and runs the per-row walk only on a hit.

Docs

None: how-to/bulk-loads.adoc already says new_edges falls back for the whole call when a value cannot cross as text.

Checklist

  • A regression test fails on the old code and passes on the new (test_subnormal_doubles_stay_doubles[...-new_edges], parametrized over insert_many, create_vertices and new_edges; the other two entry points pass on the old code)
  • The test suite that the change can affect passes (every tests/ file that uses new_edges or graph_batch, 949 tests on the JVM wheel, the test_native_* files excluded as conftest.py does)
  • uvx pre-commit run --files <changed files> passes

…wn the per-edge path

The untagged branch of GraphBatch.new_edges wrote the rows with plain json.dumps, so a
subnormal double reached the engine as text and was stored as a DECIMAL. It now uses
json_bulk_dumps, as the tagged branch uses json_bulk_dumps_tagged, and a ValueError starts
the per-edge path over.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

JVM GraphBatch.new_edges: a subnormal float in an edge property is stored as a DECIMAL

2 participants