Repository navigation
fix(#11576): a node pattern on the ordered plan seeks the bound leading prefix of a composite index - #11614
Conversation
|
Warning Review limit reachedYou've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Next included review available in 3 minutes. View limit details
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. |
|
This pull request is merged. |
Up to standards ✅🟢 Issues
|
| Metric | Results |
|---|---|
| Complexity | 7 |
NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.
…ound leading prefix of a composite index
57d2b59 to
e652569
Compare
|
@claude review this PR against the repository's contribution standards: tests for every changed |
|
Claude finished @robfrank's task in 17s —— View job Review of PR #11614I read the diff and did not build it or run the tests. Scope: The PR changes Correctness
Security: I found no issues. There is no new input surface and no new dependency. The seek uses the existing index API with values the step already bound. Tests
Nits
VERDICT: LGTM |
… whole-key lookups (p
|
@claude review this PR against the repository's contribution standards: tests for every changed |
|
Claude finished @robfrank's task in 20s —— View job Review of PR #11614I read the diff of both commits ( Scope: The PR touches Correctness
Security: I found no new input surface and no new dependency. The seek uses the existing index API with values the step already bound. It adds no unbounded memory, because the eager read is capped. Tests
Nit: The VERDICT: LGTM |
|
Automated review: fixed and approved The review pushed fixes; CI review LGTM on them, triage clean, sandbox build and tests green. Merge is a maintainer decision.
Improvements (applied):
|
…e looked-up vertex; no prefix read over a BY ITEM column - The walk seeded from one vertex kept one frontier entry per partial path, which parallel edges multiply without bound; it now keeps one entry per distinct vertex reached with the number of paths reaching it, and counts the last hop as paths times passing relationships. An undeclared label on the last position now counts 0 rather than every neighbour. - A key anchor hands the walk the record the consistent lookup loaded, so a commit that moves the key after the lookup cannot fail the predicate on a re-read. A key on a document type is not a seed, and a document or edge holding a key counts 0. - TypeIndex.prefixReachesEveryRecord: an unbound column indexed BY ITEM/KEY/VALUE gives a record one entry per element and none for an empty collection, so a prefix read misses records and repeats others. MERGE, the ordered MATCH path (#11614) and the optimizer (an index such composite is treated as answering its whole key only) no longer read a prefix of one. A null in an unbound column is no hole (#8978). - MERGE keeps the property names read when it chose the index, and refuses a prefix holding a null.
… key pins; MERGE seeks the bound prefix of a composite index (#11828) * fix(#11594, #11576): a chain count is seeded from the vertex a UNIQUE key pins; MERGE seeks the bound prefix of a composite index #11594: main was red. #11613 made the chain count push-down start from the keyed end (still enumerating that end's whole label), and #11694 (#11680), merged later, declined the same shape to the row pipeline, so CypherCountChainStartsAtSelectiveEndIssue11594Test failed 3 tests. An uncorrelated chain whose first or last node is pinned by an equality on every column of a UNIQUE index (inline or in WHERE, literals and parameters) now keeps COUNT CHAIN PATHS, seeded from that one vertex: looked up through the index when the count runs (UniqueKeyAnchor, read against one committed state), the chain reversed when the key is on the last node. The count costs the seek and that vertex's own paths, with no label walk and no row per path. EXPLAIN shows "seeded anchor: unique key Record[external_id] = [r7]". A key that no record holds counts 0; a lookup a concurrent DDL refuses falls back to the label walk, whose predicate applies the same equality. Chains with an inequality, IN lists and star joins keep #11680's decline. #11576: #11614 fixed MATCH, but MERGE matches through its own lookup, which still required the whole key, so MERGE (n:T {a: $x}) over T(a, b) scanned the type per row. It now reads the longest bound prefix of an ordered index through ConsistentKeyLookup.prefixForQuery / TypeIndex.prefixForQuery: whole, DDL-guarded and, up to MAX_ENTRIES, against one committed state, like the whole-key lookup, since a missed entry would make MERGE create a node that exists. The MERGE profile line now names the index it read. * review(#11594): test a converted and inherited seeded key, document the prefix read's memory cost A Long parameter for the INTEGER column of a composite UNIQUE key is seeded and counts as the row pipeline does; a vertex the key finds that fails another predicate of its position, or that a sibling subtype holds under an index declared on the supertype, counts 0. TypeIndex.prefixForQuery holds one RID per entry of the prefix: documented as meant for a reader that needs every match, as MERGE does. * review(#11594, #11576): seeded walk counts paths per vertex, keeps the looked-up vertex; no prefix read over a BY ITEM column - The walk seeded from one vertex kept one frontier entry per partial path, which parallel edges multiply without bound; it now keeps one entry per distinct vertex reached with the number of paths reaching it, and counts the last hop as paths times passing relationships. An undeclared label on the last position now counts 0 rather than every neighbour. - A key anchor hands the walk the record the consistent lookup loaded, so a commit that moves the key after the lookup cannot fail the predicate on a re-read. A key on a document type is not a seed, and a document or edge holding a key counts 0. - TypeIndex.prefixReachesEveryRecord: an unbound column indexed BY ITEM/KEY/VALUE gives a record one entry per element and none for an empty collection, so a prefix read misses records and repeats others. MERGE, the ordered MATCH path (#11614) and the optimizer (an index such composite is treated as answering its whole key only) no longer read a prefix of one. A null in an unbound column is no hole (#8978). - MERGE keeps the property names read when it chose the index, and refuses a prefix holding a null. * review(#11594): group a vertex's paths with a while loop rather than advancing the outer index in an inner for (Codacy) * review(#11594): import IndexException for the Javadoc link, document that the anchor key is not copied
What does this PR do?
On the non-optimized (traditional) plan,
MatchNodeStep.tryFindAndUseIndex(inline map) andtryFindAndUseIndexFromWhere(WHEREequalities) kept an index only when the pattern bound every one of its properties, becauseDatabase.lookupByKeyneeds the whole key. A pattern binding only the leading property of a composite index,(n:T {a: $x})onT(a, b), scanned the whole type. The comments around both methods already said a leftmost prefix was supported.That path runs a
MATCHafter aWITH, anOPTIONAL MATCHandUNWIND $rows AS row MATCH .... In the issue's measurements over 50,000 vertices,UNWINDof 100 values with theWHEREspelling read 5,000,100 vertices (2 s), where the optimizer path seeks the index.This PR:
bestKeyIndex: the index with the longest bound leftmost prefix, an index covering all its properties preferred at equal length. A prefix shorter than the key is only taken on an index that walks its keys in order (supportsOrderedIterations()); a hash index still needs the whole key;lookupByKey), and a prefix withTypeIndex.range(true, prefix, true, prefix, true), the same readNodeIndexSeekdoes for a prefix on the optimizer path (prefixSeek). The prefix range is read whole before it is returned, likelookupByKey's result, closes its cursor, falls back to the scan when a concurrent DDL dropped or rebuilt the index, and filters an inherited index's records by label (OpenCypher ignores inherited parent-type indexes while SQL uses them; child duplicate indexes cannot coexist #7021);matchesPropertiesand theWHEREfilter) checks the unbound properties as before;MATCH NODE (n:T) [index: T[a, b], prefix: a].Motivation
Reported in #11576. Results were correct; the cost grew with the type.
Related issues
Part of #11576:
MATCHafterWITH,OPTIONAL MATCHandUNWIND ... MATCH(cases B, C, E and F of the issue).MERGE {a: $x}(case D) has its own lookup inMergeStep.tryFindByIndex, which reads keys through a consistent whole-key lookup so that a concurrent commit cannot makeMERGEcreate a node that exists (#9811); I left it unchanged, so the issue stays open for that case.Additional Notes
New test
CypherCompositeIndexPrefixSeekIssue11576Test: the four statement shapes return the right rows and their profile shows the prefix seek; aWHEREon the unbound property still filters, and a missing value returns nothing; a whole key still uses the exact lookup ([index: T[a, b]]). The prefix test fails without the fix.Tests run: the whole
com.arcadedb.query.opencyphertest package (-Dtest='com/arcadedb/query/opencypher/**/*Test' -DexcludedGroups=slow): 8380 tests, 0 failures, 11 skipped.Checklist
mvn clean packagecommand