Skip to content
This repository was archived by the owner on Nov 21, 2026. It is now read-only.

bindings: to_json_list() on a small result: default-sized JSON buffer and no extra call after a short batch - #151

Merged
tae898 merged 1 commit into
mainfrom
bindings-json-batch-small-results
Oct 4, 2026
Merged

tae898 merged 1 commit into
mainfrom
bindings-json-batch-small-results

Conversation

@tae898

@tae898 tae898 commented Oct 4, 2026

Copy link
Copy Markdown

Closes #144.

  • RowBatcher.nextJsonBatch starts from a default-sized StringBuilder instead of allocating 64 KB on every call (JFR: 99% of the allocation in a one-row read was AbstractStringBuilder.<init>(int)).
  • ResultSet.iter_json_batches stops after a batch shorter than batch_size (the batcher returns fewer rows only when the set is drained), so a one-row result makes one JVM crossing instead of two.
  • batch_size below 1 returned no rows at all; it now raises ValueError.

Measured on the laptop (relative only, balanced power profile): a bound openCypher point lookup plus to_json_list() went from 0.037 to 0.022 ms (0.59-0.62x) with identical rows, and a 9,892-row scan is unchanged (0.97-1.00x). The benchmark's embedded graph arm materializes every timed read this way, so the change reaches the published embedded numbers at the next measurement.

Tests: test_json_batches_end_on_a_short_batch_and_on_an_exact_multiple (a result that is an exact multiple of the batch size still ends, with no empty trailing batch) and test_json_batch_size_must_be_at_least_one.

🤖 Generated with Claude Code

https://claude.ai/code/session_01DeKEhHXJTy1H1GgbfBZ8mL

… and no extra call after a short batch (#144)

RowBatcher.nextJsonBatch allocated a 64 KB StringBuilder on every call and
iter_json_batches called it again to see "[]", so a one-row read paid two
JVM crossings and two 64 KB allocations: 0.037 ms through Python for a bound
openCypher point lookup whose engine part is 0.0034 ms. The builder now starts
at the default size and the loop stops after a short batch (nextJsonBatch
returns fewer than max rows only when the set is drained): 0.59-0.62x end to
end on the laptop, identical rows, no slowdown on a 9,892-row scan. A
batch_size below 1, which returned no rows at all, is refused.

Fixes #144

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DeKEhHXJTy1H1GgbfBZ8mL
@tae898
tae898 merged commit ced921b into main Oct 4, 2026
48 checks passed
@tae898
tae898 deleted the bindings-json-batch-small-results branch October 4, 2026 09:50
tae898 added a commit that referenced this pull request Oct 4, 2026
… a list of what the next measurement changes (#157)

* /next page: one-way friendships, the partial Q1, and the fixed start persons said as measured; the page lists what the next measurement changes

- Graph OLTP (l2): a sentence that each LDBC friendship is one edge from the
  smaller id to the larger, so 1-hop counts only friends with a larger id and
  2-hop and 3-hop reach along rising ids; the next measurement asks both
  directions (BUGS F169, DECISIONS #151 item 2). Graph analytics (l2olap,
  pending): the same lead, the triangle count finds none by construction, and
  most friends and degree distribution count friends with a larger id. Both
  keyed on LDBC rows without `knows_direction` (export_web._knows_one_way_note),
  and page_check now checks the disclosure the way it checks F146's.
- Document OLAP: Q1 is described as TPC-H Q1 with seven of its ten output
  columns (the average price, the average discount, and the charge with tax
  left out) and the cold column names it "Q1", while the rows lack `tpch_q1`;
  the column count is read from the rows' res_q1_sample (BUGS F170).
- Graph OLTP repetition sentence: every repetition and every engine reads the
  same start persons, picked with a fixed seed and with replacement, not "a
  fresh set" (BUGS F171).
- Page level: the next-measurement sentence becomes a lead plus one line per
  change (release move with the build date read from the rows, graph warm-up,
  data-only sync, both directions, full TPC-H Q1, re-run cells, and a pointer
  to the notes for the rest). Each line is shown only while its table
  sentence is on the page; built after _finish_table so it sees every table
  sentence.
- CAMPAIGN section 7 rows 56-59: undirected graph questions, full TPC-H Q1,
  digests for the stock decrement and the KNOWS edge writes plus a payments_n
  gate, and the answer profile with degenerate_check.py (patch referenced,
  not applied).

Payload regenerated from the tracked rows by refresh_web_page steps 2 and 3
by hand (BENCH_ENGINE_COMMIT=417314c18, BENCH_INSTRUMENT=2026-10,
BENCH_ONLY_LANES=e2,l1tpc,l2,l3s,lifecycle, untracked runs.jsonl,
e4decomp_417314c18, and sparse_mp_417314c18 copied from the main checkout;
the same procedure on the unchanged generator reproduced the tracked payload
byte for byte). No table, entry, cell, column, or top-level field changed;
only l2 and docs_olap conditions, the page conditions, and their provenance
records. Gates: equivalence, provenance, fairness, version consistency,
version pin, and structure ok with logs identical to the baseline;
page_check against the real site rc=0, 0 condition findings, l2 disclosed.
Retirement checked on altered rows: knows_direction, tpch_q1, read_warmup,
txWalFlush=1, and a release engine_version each remove their sentence and
page line.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DeKEhHXJTy1H1GgbfBZ8mL

* /next page: Q6 says it adds a row count, Q1 aggregates nearly every line item, and the start-person counts lead their scale labels

- Document OLAP query words: Q6 is "TPC-H Q6 ... with a row count added"
  (the lane returns revenue and `count(*) AS n`), not "TPC-H's own Q6".
- Q1 "groups and aggregates nearly every line item", in the partial-Q1 words
  and in the full-Q1 entry, since it keeps only line items shipped on or
  before its cutoff date.
- Graph OLTP repetition sentence: the per-scale counts are joined as "500 at
  SF1 (...) and 200 at SF10 (...)", so the scale labels' parentheses no
  longer nest inside another pair; the three write branches each read as one
  sentence.

Payload regenerated the same way as the previous commit (refresh_web_page
steps 2 and 3 by hand from the tracked rows). Against the previous commit
only docs_olap conditions[0] and l2 conditions[9] changed, with their two
provenance records; against origin/main still no table, entry, cell, column,
or top-level field changed. Gates: equivalence, provenance, fairness,
version consistency, version pin, and structure ok, logs identical to the
baseline; page_check against the real site rc=0, 0 condition findings, l2
disclosed for both the zero ages and the one-way friendships.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DeKEhHXJTy1H1GgbfBZ8mL

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

to_json_list() on a small result: a 64 KB buffer per batch and an extra Java call cost ~40% of a one-row read

1 participant