You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Both wheels: a per-row Python validation loop runs before json.dumps in insert_many and create_vertices #318
This comes from a read-only audit of the tag 26.11.1 (commit 2f28a328be); everything below marked read was seen in the code with the file and line, nothing was run or timed, and no speed claim is made. Related closed work that this builds on and does not repeat: #229 (native point reads), #231 (native insert_many ragged lists), #235 (JVM insert_many cyclic GC), #243 (JVM GraphBatch property shapes), #174 (commit sizes).
Read: insert_many and create_vertices walk every row in Python to validate it (type_conversion.py:931-983) and then walk it again in json.dumps. The native wheel also converts columns to rows and back to columns (_native/database.py:1346, _native/bulk.py:57-71), and NativeGraphBatch._create_vertices_json_bulk has no caller (_native/extras.py:232).
Proposal: validate during the single json.dumps pass (a default hook or a typed encoder), and remove the dead code path. Bindings only. Error timing changes subtly (an error appears during encoding, not before the first row is sent), so the existing tests need a look and a test for the first bad row of a large batch.
Measure first: CPU per row of insert_many on a plain scalar table, warmed, before and after. Done when the loop is gone and the measured gain, or the absence of one, is written here.
I measured the Python-side step before the JSON text goes to the engine, old against new, as this issue asks: the 26.11.1 tag (2f28a328be) against main (c90b3f9). It is pure Python on one laptop core (Intel Core Ultra X9 388H), no engine, three plain scalar tables (3 columns, 8 columns, 8 columns with a Hangul string), insert_many in chunks of 10,000 rows and create_vertices at 100,000 and 250,000 rows, with numpy and without it (numpy is only an extra). Median CPU time per row over 21, 7 or 5 repeats, in microseconds. A cell only counts as faster or slower when the slowest repeat of one build beats the fastest of the other.
cell
numpy: tag to main
no numpy: tag to main
insert_many, 3 columns, 10,000 rows
0.789 to 0.785 (1.01x, not separated)
0.796 to 0.780 (1.02x, not separated)
insert_many, 8 columns, 10,000 rows
1.911 to 2.135 (0.90x, slower)
1.903 to 2.181 (0.87x, slower)
insert_many, 8 columns, Hangul, 10,000 rows
2.082 to 2.264 (0.92x, not separated)
2.069 to 2.230 (0.93x, not separated)
create_vertices, 3 columns, 100,000 rows
0.774 to 0.698 (1.11x, faster)
0.777 to 0.856 (0.91x, slower)
create_vertices, 3 columns, 250,000 rows
0.806 to 0.767 (1.05x, not separated)
0.808 to 0.913 (0.88x, slower)
create_vertices, 8 columns, 100,000 rows
1.897 to 1.953 (0.97x, not separated)
1.823 to 2.350 (0.78x, slower)
create_vertices, 8 columns, 250,000 rows
1.891 to 2.082 (0.91x, slower)
1.805 to 2.464 (0.73x, slower)
create_vertices, 8 columns, Hangul, 100,000 rows
1.846 to 1.806 (1.02x, not separated)
1.885 to 2.222 (0.85x, slower)
create_vertices, 8 columns, Hangul, 250,000 rows
1.945 to 1.981 (0.98x, not separated)
1.962 to 2.443 (0.80x, slower)
So the gain is absent. With numpy the two builds are within about 11 percent of each other in both directions. Without numpy main is slower in 7 of the 9 cells, clearly, down to 0.73x of the tag's speed. The plain json.dumps of the same rows costs 0.39 to 1.07 microseconds per row, which is about half of the step and cannot go.
This is only the Python-side step, not end-to-end ingest: the engine, the crossing into it, parsing and the commit are not in it and are usually the larger part. The per-row loop is gone from the hot path, but the step did not get cheaper, so I opened #371 for the slower no-numpy case and am closing this one, since the measurement was what it asked for.
Activity
Tests: bandit annotations for the seeded corpus and the test-controll…
bulk dumps: raise the first bad row's error outside the except block …
create_vertices: decide the whole input before any chunk is sent when…
create_vertices: no text scan for a table of scalars (g1 regression o…
insert_many and create_vertices: check the rows in the one json.dumps…
tae898 commentedon Nov 12, 2026
I measured the Python-side step before the JSON text goes to the engine, old against new, as this issue asks: the 26.11.1 tag (
2f28a328be) against main (c90b3f9). It is pure Python on one laptop core (Intel Core Ultra X9 388H), no engine, three plain scalar tables (3 columns, 8 columns, 8 columns with a Hangul string),insert_manyin chunks of 10,000 rows andcreate_verticesat 100,000 and 250,000 rows, with numpy and without it (numpy is only an extra). Median CPU time per row over 21, 7 or 5 repeats, in microseconds. A cell only counts as faster or slower when the slowest repeat of one build beats the fastest of the other.So the gain is absent. With numpy the two builds are within about 11 percent of each other in both directions. Without numpy main is slower in 7 of the 9 cells, clearly, down to 0.73x of the tag's speed. The plain
json.dumpsof the same rows costs 0.39 to 1.07 microseconds per row, which is about half of the step and cannot go.This is only the Python-side step, not end-to-end ingest: the engine, the crossing into it, parsing and the commit are not in it and are usually the larger part. The per-row loop is gone from the hot path, but the step did not get cheaper, so I opened #371 for the slower no-numpy case and am closing this one, since the measurement was what it asked for.