Skip to content
This repository was archived by the owner on Nov 21, 2026. It is now read-only.

benchmarks: Neo4j 2026.10.0, Memgraph 3.14.0, Qdrant v1.20.0 comparator pins and clients - #482

Merged
tae898 merged 3 commits into
mainfrom
bench/bump-neo4j-memgraph-qdrant
Oct 23, 2026
Merged

tae898 merged 3 commits into
mainfrom
bench/bump-neo4j-memgraph-qdrant

Conversation

@tae898

@tae898 tae898 commented Oct 23, 2026

Copy link
Copy Markdown

Bumps the Neo4j, Memgraph and Qdrant comparator pins and their clients to the latest stable releases for the 2026-11 comparator subset (SUBSET-READINESS item 3). Vendor configuration is unchanged: no release note requires a change.

was now (amd64 manifest digest)
Neo4j 2026.08.1-community 2026.10.0-community sha256:c3903fa96db31479c2a28e5d00e05260b039ff856f8b2cca7280c78299877731
Memgraph 3.13.1 3.14.0 sha256:c7478d1737edb77fa2b1fcd4e37e34a68f10272d0ec7d7f9aa92d0afc8e0be9f
Qdrant v1.19.1 v1.20.0 sha256:2d1b0bfd0a961275481ea09400fc98008216ae54e1b52ed7d24520d63050c644
neo4j / neo4j-rust-ext 6.3.1 / 6.3.1.0 6.4.1 / 6.4.1.0
qdrant-client 1.19.1 1.20.0

Changed: runner.BACKENDS, Dockerfile.composed, vendor_memory.NEO4J_IMAGE_DIGEST, build_images.sh [client], smoke_memgraph_server_limit.sh, COMPARATORS.md, QUANTIZATION.md, PROTOCOL.md, code comments that name the pin, and stage_drafts_2611.sha256 (12 drafts bake the digests). If the other engine-bump PRs merge first, rewrite that list with python3 test_stage_generator_2611.py --rewrite.

Neo4j memory recommendation (DECISIONS #202): neo4j_memory_recommendation.json regenerated on the new image with vendor_memory.py --regenerate. The tool output is identical at all eight caps, so NEO4J_MEMORY and test_vendor_memory.py need no change.

Release notes that matter. Neo4j 2026.09.0 (skipped by our pin) made re-scored binary and scalar quantization search about 5x faster and changed the memory guidance for vector indexes (Lucene files live in the OS file cache); 2026.10.0 has no vector or index change. A probe on the pinned image confirms an index with no vector.quantization.type is still BINARY (expansion 3.0), NONE 1.0, SCALAR 1.5, and the arms set and read back the type and 1.0. Memgraph 3.14.0 lists comparison and NaN semantic changes and fixes vector index crash recovery; the lane's Cypher is unchanged and every result digest still matched; the vector index still takes no HNSW parameters (expansion_add 128, expansion_search 64 read back). Qdrant v1.20.0 changes HNSW level assignment (6.25% of points on level 1 at m=16, was 25%), adds turbo8/turbo16 and inline_storage (unused); the sparse default is still Float32. Details in DECISIONS #209.

Smoke (laptop, micro/e2 scale, one rep, private dbbench:client-nmq and dbbench:composed-nmq images built from the new pins): l2 graph oltp and olap on Neo4j and Memgraph, all result digests equal between the two engines; e2 on Neo4j, Memgraph and the composed Qdrant+Neo4j stack, retrieval and filtered-candidate digests equal across all three; l3d dense, six arms (qdrant fp32/int8, neo4j fp32/int8, memgraph fp32/int8) ran, with the Neo4j type and expansion read back. Client image reads back neo4j 6.4.1, neo4j-rust-ext 6.4.1.0, qdrant-client 1.20.0.

Tests: test_vendor_memory, test_engine_pin, test_gen_engines_doc, test_sparse_mp_versions, test_memory_probe, test_fairness_headers, test_stage_generator_2611, test_stage_generator_after, test_milvus_compose, test_jpype_pin: 198 passed, 1 skipped; version_pin_check.py: 0 disagreements.

Not done here: the l3s sparse Qdrant smoke (the runner's host lock was held by another agent's run), docker pull and the dbbench:client / dbbench:composed rebuild on mini (install step).

🤖 Generated with Claude Code

tae898 and others added 3 commits October 23, 2026 16:38
…rator pins, with their clients

Latest stable releases, pinned by amd64 manifest digest. Neo4j 2026.08.1 -> 2026.10.0
(c3903fa9...), Memgraph 3.13.1 -> 3.14.0 (c7478d17...), Qdrant v1.19.1 -> v1.20.0
(2d1b0bfd...). Clients: neo4j 6.3.1 -> 6.4.1, neo4j-rust-ext 6.3.1.0 -> 6.4.1.0,
qdrant-client 1.19.1 -> 1.20.0. Vendor config unchanged (no release note requires a change).

neo4j_memory_recommendation.json regenerated on the new image (DECISIONS #202): the
recommendation is identical at all eight caps, so NEO4J_MEMORY is unchanged. The 12 stage
drafts that bake the digests have a new sha list.

Smoke on the laptop (graph, dense, cross-model): result digests equal across Neo4j, Memgraph and
the composed stack; Neo4j quantization defaults re-probed (unset BINARY 3.0, NONE 1.0, SCALAR 1.5).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…s_2611.sha256

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…raph-qdrant

# Conflicts:
#	benchmarks/experiments/COMPARATORS.md
#	benchmarks/experiments/QUANTIZATION.md
#	benchmarks/experiments/stage_drafts_2611.sha256
@tae898
tae898 merged commit 831f4b5 into main Oct 23, 2026
4 checks passed
@tae898
tae898 deleted the bench/bump-neo4j-memgraph-qdrant branch October 23, 2026 10:41
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant