Skip to content

fix(#11576): a node pattern on the ordered plan seeks the bound leading prefix of a composite index - #11614

Merged
robfrank merged 2 commits into
ArcadeData:mainfrom
shaunpatterson:fix/11576-composite-prefix-seek
Nov 14, 2026
Merged

robfrank merged 2 commits into
ArcadeData:mainfrom
shaunpatterson:fix/11576-composite-prefix-seek

Conversation

@shaunpatterson

@shaunpatterson shaunpatterson commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

What does this PR do?

On the non-optimized (traditional) plan, MatchNodeStep.tryFindAndUseIndex (inline map) and tryFindAndUseIndexFromWhere (WHERE equalities) kept an index only when the pattern bound every one of its properties, because Database.lookupByKey needs the whole key. A pattern binding only the leading property of a composite index, (n:T {a: $x}) on T(a, b), scanned the whole type. The comments around both methods already said a leftmost prefix was supported.

That path runs a MATCH after a WITH, an OPTIONAL MATCH and UNWIND $rows AS row MATCH .... In the issue's measurements over 50,000 vertices, UNWIND of 100 values with the WHERE spelling read 5,000,100 vertices (2 s), where the optimizer path seeks the index.

This PR:

  • moves the two copies of the index choice into bestKeyIndex: the index with the longest bound leftmost prefix, an index covering all its properties preferred at equal length. A prefix shorter than the key is only taken on an index that walks its keys in order (supportsOrderedIterations()); a hash index still needs the whole key;
  • seeks a whole key as before (lookupByKey), and a prefix with TypeIndex.range(true, prefix, true, prefix, true), the same read NodeIndexSeek does for a prefix on the optimizer path (prefixSeek). The prefix range is read whole before it is returned, like lookupByKey's result, closes its cursor, falls back to the scan when a concurrent DDL dropped or rebuilt the index, and filters an inherited index's records by label (OpenCypher ignores inherited parent-type indexes while SQL uses them; child duplicate indexes cannot coexist #7021);
  • the step's row filter (matchesProperties and the WHERE filter) checks the unbound properties as before;
  • the profile names the index and the prefix: MATCH NODE (n:T) [index: T[a, b], prefix: a].

Motivation

Reported in #11576. Results were correct; the cost grew with the type.

Related issues

Part of #11576: MATCH after WITH, OPTIONAL MATCH and UNWIND ... MATCH (cases B, C, E and F of the issue). MERGE {a: $x} (case D) has its own lookup in MergeStep.tryFindByIndex, which reads keys through a consistent whole-key lookup so that a concurrent commit cannot make MERGE create a node that exists (#9811); I left it unchanged, so the issue stays open for that case.

Additional Notes

New test CypherCompositeIndexPrefixSeekIssue11576Test: the four statement shapes return the right rows and their profile shows the prefix seek; a WHERE on the unbound property still filters, and a missing value returns nothing; a whole key still uses the exact lookup ([index: T[a, b]]). The prefix test fails without the fix.

Tests run: the whole com.arcadedb.query.opencypher test package (-Dtest='com/arcadedb/query/opencypher/**/*Test' -DexcludedGroups=slow): 8380 tests, 0 failures, 11 skipped.

Checklist

  • I have run the build using mvn clean package command
  • My unit tests cover both failure and success scenarios

@coderabbitai

coderabbitai Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 3 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available.

Learn how review limits work.

Review configuration:

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: a6cba213-dfb1-432e-a4f3-8e903e5aac76
Commits

Reviewing files that changed from the base of the PR and between 272588f and cbce468.

Files selected for processing: 2

  • engine/src/main/java/com/arcadedb/query/opencypher/executor/steps/MatchNodeStep.java
  • engine/src/test/java/com/arcadedb/query/opencypher/CypherCompositeIndexPrefixSeekIssue11576Test.java
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@mergify

mergify Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

This pull request is merged.

@codacy-production

codacy-production Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

🟢 Metrics 7 complexity

Metric Results
Complexity 7

View in Codacy

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

@robfrank robfrank self-assigned this Nov 13, 2026
@robfrank
robfrank force-pushed the fix/11576-composite-prefix-seek branch from 57d2b59 to e652569 Compare November 13, 2026 20:59
@robfrank

Copy link
Copy Markdown
Collaborator

@claude review this PR against the repository's contribution standards: tests for every changed
entry point, no security issues, no unrelated changes. End your review with exactly one line:
VERDICT: LGTM | VERDICT: NEEDS_CHANGES | VERDICT: UNSAFE
Judge only this PR's diff. Ignore CI failures the diff did not cause.

@claude

claude Bot commented Nov 14, 2026 •

Copy link
Copy Markdown

Claude finished @robfrank's task in 17s —— View job


Review of PR #11614

I read the diff and did not build it or run the tests.

Scope: The PR changes MatchNodeStep.java and adds one test class. Both match the stated fix, and I found nothing unrelated.

Correctness

Security: I found no issues. There is no new input surface and no new dependency. The seek uses the existing index API with values the step already bound.

Tests

  • CypherCompositeIndexPrefixSeekIssue11576Test covers the four statement shapes, the WHERE on the unbound property, a missing value, and the whole-key exact lookup. It asserts the profile text [index: ..., prefix: a].

  • Two things I would still like to see:

    • A hash composite index with only a prefix bound should fall back to the scan.
    • An inherited-index case that exercises the label filter in prefixSeek.

    Neither blocks the merge.

Nits

  • The mvn clean package checkbox in the PR description is unchecked. The description does list a run of the opencypher test package.

VERDICT: LGTM

@robfrank

Copy link
Copy Markdown
Collaborator

@claude review this PR against the repository's contribution standards: tests for every changed
entry point, no security issues, no unrelated changes. End your review with exactly one line:
VERDICT: LGTM | VERDICT: NEEDS_CHANGES | VERDICT: UNSAFE
Judge only this PR's diff. Ignore CI failures the diff did not cause.

@claude

claude Bot commented Nov 14, 2026 •

Copy link
Copy Markdown

Claude finished @robfrank's task in 20s —— View job


Review of PR #11614

I read the diff of both commits (HEAD~2..HEAD, because the shallow checkout has no merge base with origin/main). I did not build it or run the tests.

Scope: The PR touches MatchNodeStep.java and adds CypherCompositeIndexPrefixSeekIssue11576Test. Both fit the stated fix. I found nothing unrelated.

Correctness

Security: I found no new input surface and no new dependency. The seek uses the existing index API with values the step already bound. It adds no unbounded memory, because the eager read is capped.

Tests

  • The earlier request for more tests is now met:
    • aHashIndexDeclinesAPartialKey
    • anInheritedIndexDoesNotReturnSiblingRecords
    • aWholeKeyBeatsAPrefixOfAnotherIndex
    • aPrefixHoldingManyEntriesIsStreamed
  • The four statement shapes, the unbound-property filter and the whole-key exact lookup are covered too.
  • The tests follow the repo style. I saw no hard-coded ports and no wall-clock assertions.

Nit: The mvn clean package checkbox in the description is unchecked. The description does report the opencypher package run.

VERDICT: LGTM

@robfrank robfrank added the pr_approved review-pr: CI LGTM + clean triage + green sandbox tests on the reviewed head label Nov 14, 2026
@robfrank

Copy link
Copy Markdown
Collaborator

Automated review: fixed and approved

The review pushed fixes; CI review LGTM on them, triage clean, sandbox build and tests green. Merge is a maintainer decision.

Step Result
Triage clean
CI review LGTM
Sandbox build and tests on cbce46862888e16c8d787d128db1100eee54f58a green
Fix commits pushed cbce46862888e16c8d787d128db1100eee54f58a to the PR branch

Improvements (applied):

  • quality - Do not let a prefix range seek displace a whole-key exact lookup on another index
  • quality - Guard supportsOrderedIterations in bestKeyIndex against a concurrent index drop
  • performance - Bound the prefix seeks eager materialization as seekKeys does
  • tests - Test the hash-index decline and the inherited-index label filter of the prefix seek

@robfrank
robfrank merged commit bcb81cc into ArcadeData:main Nov 14, 2026
26 of 30 checks passed
lvca added a commit that referenced this pull request Nov 15, 2026
…e looked-up vertex; no prefix read over a BY ITEM column

- The walk seeded from one vertex kept one frontier entry per partial path, which
  parallel edges multiply without bound; it now keeps one entry per distinct vertex
  reached with the number of paths reaching it, and counts the last hop as paths times
  passing relationships. An undeclared label on the last position now counts 0 rather
  than every neighbour.
- A key anchor hands the walk the record the consistent lookup loaded, so a commit that
  moves the key after the lookup cannot fail the predicate on a re-read. A key on a
  document type is not a seed, and a document or edge holding a key counts 0.
- TypeIndex.prefixReachesEveryRecord: an unbound column indexed BY ITEM/KEY/VALUE gives a
  record one entry per element and none for an empty collection, so a prefix read misses
  records and repeats others. MERGE, the ordered MATCH path (#11614) and the optimizer
  (an index such composite is treated as answering its whole key only) no longer read a
  prefix of one. A null in an unbound column is no hole (#8978).
- MERGE keeps the property names read when it chose the index, and refuses a prefix
  holding a null.
lvca added a commit that referenced this pull request Nov 15, 2026
… key pins; MERGE seeks the bound prefix of a composite index (#11828)

* fix(#11594, #11576): a chain count is seeded from the vertex a UNIQUE key pins; MERGE seeks the bound prefix of a composite index

#11594: main was red. #11613 made the chain count push-down start from the
keyed end (still enumerating that end's whole label), and #11694 (#11680),
merged later, declined the same shape to the row pipeline, so
CypherCountChainStartsAtSelectiveEndIssue11594Test failed 3 tests. An
uncorrelated chain whose first or last node is pinned by an equality on
every column of a UNIQUE index (inline or in WHERE, literals and
parameters) now keeps COUNT CHAIN PATHS, seeded from that one vertex:
looked up through the index when the count runs (UniqueKeyAnchor, read
against one committed state), the chain reversed when the key is on the
last node. The count costs the seek and that vertex's own paths, with no
label walk and no row per path. EXPLAIN shows
"seeded anchor: unique key Record[external_id] = [r7]". A key that no
record holds counts 0; a lookup a concurrent DDL refuses falls back to
the label walk, whose predicate applies the same equality. Chains with an
inequality, IN lists and star joins keep #11680's decline.

#11576: #11614 fixed MATCH, but MERGE matches through its own lookup,
which still required the whole key, so MERGE (n:T {a: $x}) over T(a, b)
scanned the type per row. It now reads the longest bound prefix of an
ordered index through ConsistentKeyLookup.prefixForQuery /
TypeIndex.prefixForQuery: whole, DDL-guarded and, up to MAX_ENTRIES,
against one committed state, like the whole-key lookup, since a missed
entry would make MERGE create a node that exists. The MERGE profile line
now names the index it read.

* review(#11594): test a converted and inherited seeded key, document the prefix read's memory cost

A Long parameter for the INTEGER column of a composite UNIQUE key is seeded and counts as the row pipeline does; a vertex the key finds that fails another predicate of its position, or that a sibling subtype holds under an index declared on the supertype, counts 0. TypeIndex.prefixForQuery holds one RID per entry of the prefix: documented as meant for a reader that needs every match, as MERGE does.

* review(#11594, #11576): seeded walk counts paths per vertex, keeps the looked-up vertex; no prefix read over a BY ITEM column

- The walk seeded from one vertex kept one frontier entry per partial path, which
  parallel edges multiply without bound; it now keeps one entry per distinct vertex
  reached with the number of paths reaching it, and counts the last hop as paths times
  passing relationships. An undeclared label on the last position now counts 0 rather
  than every neighbour.
- A key anchor hands the walk the record the consistent lookup loaded, so a commit that
  moves the key after the lookup cannot fail the predicate on a re-read. A key on a
  document type is not a seed, and a document or edge holding a key counts 0.
- TypeIndex.prefixReachesEveryRecord: an unbound column indexed BY ITEM/KEY/VALUE gives a
  record one entry per element and none for an empty collection, so a prefix read misses
  records and repeats others. MERGE, the ordered MATCH path (#11614) and the optimizer
  (an index such composite is treated as answering its whole key only) no longer read a
  prefix of one. A null in an unbound column is no hole (#8978).
- MERGE keeps the property names read when it chose the index, and refuses a prefix
  holding a null.

* review(#11594): group a vertex's paths with a while loop rather than advancing the outer index in an inner for (Codacy)

* review(#11594): import IndexException for the Javadoc link, document that the anchor key is not copied
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr_approved review-pr: CI LGTM + clean triage + green sandbox tests on the reviewed head

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants