Skip to content

fix(#8666): serve LIKE 'prefix%', STARTS WITH and Cypher min()/max() from the index - #8669

Merged
lvca merged 4 commits into
mainfrom
fix/8666-prefix-like-min-max
Sep 29, 2026
Merged

lvca merged 4 commits into
mainfrom
fix/8666-prefix-like-min-max

Conversation

@lvca

@lvca lvca commented Sep 29, 2026

Copy link
Copy Markdown
Member

Fixes #8666

What

  • SQL LIKE 'abc%': the planner adds s >= 'abc' AND s < 'abd' next to the LIKE when an ordered, case sensitive index covers a STRING property. The LIKE stays as a filter, so the answer never depends on the collation. The prefix is what precedes the first wildcard (\% and \? read as literals), so 'ab?d%' and 'abc%def' are bounded too. A parameter pattern shapes the plan, so that plan is not cached for other parameters.
  • Cypher STARTS WITH: contributes the same two bounds to NodeIndexRangeScan (literal or parameter, the successor is computed when the parameter is resolved); the predicate stays as a filter.
  • Cypher min(n.p) / max(n.p): MATCH (n:L) RETURN min(n.p) reads one end of the index (IndexMinMaxStep, shown as MIN FROM INDEX). Only when the index is defined on the label itself, single property, LSM, not NULL_STRATEGY INDEX, case sensitive, on a key type whose index order is Cypher's.
  • QueryHelper.prefixSuccessor steps over U+D7FF, surrogates and U+FFFF so the bound holds under both UTF-16 and UTF-8 key order.

Tests

Issue8666SqlLikePrefixIndexTest (8), Issue8666CypherPrefixAndMinMaxIndexTest (15). Ran the whole query.opencypher and query.sql unit suites (888 classes) green.

…from the index

SQL: the planner adds the range [prefix, successor) next to a LIKE on an
ordered, case sensitive index over a STRING property, keeping the LIKE as a
filter. A parameter pattern shapes the plan, so that plan is not cached.

Cypher: STARTS WITH becomes a NodeIndexRangeScan range (the predicate stays a
filter); a bare min()/max() over one indexed property reads one end of the
index (IndexMinMaxStep), like SQL's MIN/MAX FROM INDEX.
@lvca lvca added this to the 26.10.1 milestone Sep 29, 2026
@lvca lvca self-assigned this Sep 29, 2026
@coderabbitai

coderabbitai Bot commented Sep 29, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

Next included review available in 5 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: 148d35bb-78ad-4b28-aaf0-15aed76f0c4f

📥 Commits

Reviewing files that changed from the base of the PR and between 66a82a8 and 7adb86a.

📒 Files selected for processing (13)
  • engine/src/main/java/com/arcadedb/query/opencypher/executor/CypherExecutionPlan.java
  • engine/src/main/java/com/arcadedb/query/opencypher/executor/operators/NodeIndexRangeScan.java
  • engine/src/main/java/com/arcadedb/query/opencypher/executor/steps/IndexMinMaxStep.java
  • engine/src/main/java/com/arcadedb/query/opencypher/optimizer/AnchorSelector.java
  • engine/src/main/java/com/arcadedb/query/opencypher/optimizer/CypherOptimizer.java
  • engine/src/main/java/com/arcadedb/query/opencypher/optimizer/RangePredicate.java
  • engine/src/main/java/com/arcadedb/query/opencypher/optimizer/statistics/IndexStatistics.java
  • engine/src/main/java/com/arcadedb/query/opencypher/optimizer/statistics/StatisticsProvider.java
  • engine/src/main/java/com/arcadedb/query/sql/executor/QueryHelper.java
  • engine/src/main/java/com/arcadedb/query/sql/executor/SelectExecutionPlanner.java
  • engine/src/test/java/com/arcadedb/query/opencypher/Issue8666CypherPrefixAndMinMaxIndexTest.java
  • engine/src/test/java/com/arcadedb/query/sql/executor/Issue8666SqlLikePrefixIndexTest.java
  • engine/src/test/java/com/arcadedb/query/sql/executor/QueryHelperPrefixTest.java

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@mergify

mergify Bot commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

This pull request does not currently match the merge queue conditions, so it cannot be queued from here. The box comes back if it matches again.

@codacy-production

codacy-production Bot commented Sep 29, 2026 •

Copy link
Copy Markdown

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

🟢 Metrics 47 complexity

Metric Results
Complexity 47

View in Codacy

🟢 Coverage 96.72% diff coverage · -6.00% coverage variation

Metric Results
Coverage variation ✅ -6.00% coverage variation
Diff coverage ✅ 96.72% diff coverage

View coverage diff in Codacy

Coverage variation details
Coverable lines Covered lines Coverage
Common ancestor commit (66a82a8) 198662 167204 84.17%
Head commit (7adb86a) 231408 (+32746) 180872 (+13668) 78.16% (-6.00%)

Coverage variation is the difference between the coverage for the head and common ancestor commits of the pull request branch: <coverage of head commit> - <coverage of common ancestor commit>

Diff coverage details
Coverable lines Covered lines Diff coverage
Pull request (#8669) 183 177 96.72%

Diff coverage is the percentage of lines that are covered by tests out of the coverable lines that the pull request added or modified: <covered lines added or modified>/<coverable lines added or modified> * 100%

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

@claude

claude Bot commented Sep 29, 2026

Copy link
Copy Markdown

Review of #8669 (LIKE prefix / STARTS WITH / Cypher min-max from index)

Overall this is a careful change. The range is only ever a superset with the original predicate kept as a filter, prefixSuccessor handles the UTF-16 vs UTF-8 ordering edge cases, IndexMinMaxStep re-resolves the index at execution time, and the plan-cache handling for parameter patterns is right. Tests are extensive. Details below, most important first.

1. Possible missed rows: Cypher STARTS WITH on a case-insensitive index (worth verifying)

The SQL side skips case-insensitive indexes (!ciCollation), and tryCreateIndexMinMaxOptimization does too (isCaseInsensitive(0)), but the Cypher STARTS WITH path (AnchorSelector -> RangePredicate.forPrefix -> NodeIndexRangeScan) has no such guard. On a CI index the keys are stored lower-cased, and convertKeysToDeclaredTypes lower-cases the keys it is given. If the range bounds go through it, the successor (computed before lower-casing) is not order-preserving: STARTS WITH 'AZ' gives ['az', lower('A[') = 'a['), and a stored key azb is >= 'a[' ('z' 0x7A > '[' 0x5B), so a row that satisfies the predicate is dropped by the index before the filter can see it. Prefixes ending in Z are the ASCII case; non-ASCII characters with odd case mappings could hit it too.

The PR description says the filter makes the answer independent of collation, but a filter can only remove rows, not restore missed ones. Suggest either skipping the prefix bounds when the index is case-insensitive, or computing the successor after lower-casing. A regression test with a case-insensitive index and STARTS WITH 'AZ' would settle it. Issue8666SqlLikePrefixIndexTest.aCaseInsensitiveIndexIsNotBounded covers this for SQL only; there is no Cypher equivalent. (I read the code path but did not run it, so please confirm.)

2. Transaction visibility of IndexMinMaxStep

readEnd reads index.iterator(...) directly. Please add a test for min/max inside an open transaction after inserting a new extreme value and after deleting the current one, and confirm it matches what the label scan would return. If the LSM cursor doesn't merge uncommitted changes the way a scan does, this can return a stale result. If SQL's MIN FROM INDEX already behaves the same way, at least document it.

3. Smaller points

  • NodeIndexRangeScan.resolveIndexAndBounds: for a non-String parameter (e.g. $p = 5) the successor bound is skipped but the >= lower bound is still pushed to the index. boundMatchesIndexKey should catch it, but startsWithAnEmptyOrNullPrefix only tests null. A non-string param test (integer, list) would lock that in.
  • SQL planDependsOnInputParameters disables plan caching for every LIKE :p query with a non-empty literal prefix on an indexed STRING property, even when the resulting plan doesn't end up using the index. That is safe, but prepared-statement heavy workloads lose caching for these queries. Could be narrowed to when the descriptor built from the added range is actually chosen; otherwise fine, maybe with a Javadoc note.
  • CypherOptimizer.INDEX_ORDERED_KEY_TYPES is now public static on a mutable EnumSet. Consider Collections.unmodifiableSet or an isIndexOrderedKeyType(Type) helper so external code can't mutate it.
  • Style: new imports in CypherExecutionPlan (IndexMinMaxStep, RangeIndex, LSMTreeIndexAbstract, Property) and SelectExecutionPlanner (ValueExpression, LikeOperator) are not in the alphabetical position of their neighbours. The long condition in QueryHelper.prefixSuccessor could become a small named helper.
  • IndexMinMaxStep.readEnd throws "re-plan the query" if the index is dropped. That matches NodeIndexRangeScan; just check that DDL invalidates the Cypher plan cache so users normally never see it.

Test coverage

Good breadth (empty/null/escaped prefixes, U+D7FF/U+FFFF boundaries, null-valued vertices). Gaps: case-insensitive index in Cypher (item 1), transactional min/max (item 2), non-string parameters.

Review only, tests not run locally. Nothing blocks besides resolving or refuting item 1.

… more tests

A case-insensitive index holds case-folded keys, so the folded successor bound could drop rows the predicate accepts (STARTS WITH 'AZ' missed 'azb'). Skipped at anchor selection and at execution. Adds tests for a CI index, non-string parameters and a delete in an open transaction; INDEX_ORDERED_KEY_TYPES is now unmodifiable.
@claude

claude Bot commented Sep 29, 2026

Copy link
Copy Markdown

Review of #8669 (prefix range for LIKE / STARTS WITH, index-backed min() / max())

I read the diff and cross-checked it against the code it touches. I did not run the tests: the sandbox blocked the Maven command.

Verdict: no blockers. The design is sound and the tests are thorough. Everything below is minor.

What I checked and found correct

  • Range is only a superset. LIKE and STARTS WITH stay as filters, so a wrong bound can only cost speed, never rows. Case-insensitive indexes are excluded in all three places: the SQL planner, AnchorSelector, and NodeIndexRangeScan.
  • SQL LIKE is case sensitive. convertForRegExp compiles a plain Pattern with no CASE_INSENSITIVE flag, so the >= prefix AND < successor range never excludes a row the LIKE would match. likeLiteralPrefix reads \%, \? and a lone \ the same way convertForRegExp does.
  • UTF-8 vs UTF-16 key order. BinaryComparator orders strings by unsigned UTF-8 bytes, while String.compareTo uses UTF-16 units. prefixSuccessor steps over U+D7FF, surrogates and U+FFFF so the bound holds under both orders.
  • min / max on strings agrees with the index. CypherFunctionHelper.cypherCompare uses BinaryComparator.compareStrings, the same order as the index. The scan and the index therefore agree on non-BMP characters.
  • Type mismatch on STARTS WITH. A string bound on a numeric index is neutralised by the existing boundMatchesIndexKey guard in NodeIndexRangeScan.
  • Skip/limit and nulls. SKIP and LIMIT on the min / max push-down go through applySkipAndLimit. NULL_STRATEGY INDEX is excluded, and a deleted record is skipped via RecordNotFoundException.
  • Cached plans. planDependsOnInputParameters correctly stops a parameter-shaped SQL plan from being cached.

Suggestions

  1. Import order.
    • CypherExecutionPlan: IndexMinMaxStep sits between TypeCountStep and UnionStep, and RangeIndex, LSMTreeIndexAbstract and Property are out of alphabetical order.
    • SelectExecutionPlanner: ValueExpression and LikeOperator are out of alphabetical order.
    • Please put them where the rest of the imports in each file are sorted.
  2. Parameterised LIKE :p is never plan-cached. planDependsOnInputParameters is set while the planner evaluates candidate indexes, before it knows the range is used in the final plan. Any WHERE strField LIKE :p with a non-empty literal prefix on an indexed STRING property is therefore re-planned on every execution, even if the planner picks another access path. Setting the flag only once the descriptor is chosen would narrow this. If you accept the cost, a one-line comment saying so would help. The trade is a fair one, since the range pays off on large types.
  3. Numeric min / max compare through Double. cypherCompare uses Double.compare(doubleValue()), so LONG values above 2^53 can tie in the scan path, while the index path returns the exact extreme. The index answer is the more correct one, but the two paths can now differ for such values. This is an edge case, and I would not block on it.
  4. IndexMinMaxStep ignores DISTINCT in min(DISTINCT n.p). That is harmless for min / max, but a short note in the Javadoc would save the next reader a question.
  5. Tests.
    • QueryHelper.prefixSuccessor and QueryHelper.likeLiteralPrefix are pure static functions with many corner cases (empty, lone surrogates, U+FFFF, U+10FFFF, U+D7FF, trailing \). A small direct unit test would be cheaper and more precise than the query-level coverage. I saw none in the diff.
    • minAndMaxOfAStringKey covers U+FFFF but no supplementary (non-BMP) character. That is the case where UTF-8 and UTF-16 order differ, so one such key would pin the claim in the Javadoc.
    • Issue8666CypherPrefixAndMinMaxIndexTest loads 3000 vertices and does a lot of work. Check that the runtime stays in the normal unit-test range. If it is more than a few seconds, consider @Tag("slow").
  6. Style nit. CypherOptimizer.INDEX_ORDERED_KEY_TYPES is now public static final and wrapped in Collections.unmodifiableSet(...). That is fine, but the wrapped line is long. Please wrap it to match the style of the file.

Nice work on the failure-mode reasoning. Both the case-folded index and the NULL_STRATEGY INDEX traps are handled and tested.

@claude

claude Bot commented Sep 29, 2026

Copy link
Copy Markdown

Review

Solid, carefully reasoned change. The design (range bounds as a superset with the original predicate kept as a filter, collation-independent) is the right one, and the edge cases are well covered: UTF-16 vs UTF-8 successor, case-insensitive indexes, NULL_STRATEGY INDEX, subtype hierarchies, open-transaction inserts and deletes, and the non-cacheable parameter plan. I checked likeLiteralPrefix against convertForRegExp (escapes for \%, \? and a lone backslash) and they agree. LIKE is case sensitive here, so skipping ciCollation indexes is correct.

Things to address

  1. Type.FLOAT silently dropped from INDEX_ORDERED_KEY_TYPES (CypherOptimizer.java:126). The old set contained FLOAT; the new one doesn't, and neither the PR description nor the tests mention it. That set is also used by the existing ORDER BY-from-index path (line 364), so ORDER BY n.floatProp will now stop using the index. If FLOAT's index order really differs from Cypher's (I'd guess because of how float keys are serialized), please say so in the PR/javadoc and add a test pinning the fallback. If it was unintentional, restore it. Either way, it is a behaviour change outside the stated scope of Index-answerable queries planned as full scans: SQL LIKE 'prefix%', Cypher STARTS WITH, and Cypher min()/max() (80-340 ms against 0.3-4 ms for the equivalent rewrite at 1M rows) #8666.

  2. Formatting: the new EnumSet.of(...) line in CypherOptimizer.java and the if (!(codePoint >= 0xD800 ... line in QueryHelper.prefixSuccessor exceed 160 chars, as does the last javadoc line of IndexMinMaxStep. Also the new imports in CypherExecutionPlan.java (RangeIndex, LSMTreeIndexAbstract, Property) are inserted out of order relative to their neighbours.

Minor / optional

  • IndexMinMaxStep.readEnd loops over index entries and skips only null values and RecordNotFoundException. That is fine given NULL_STRATEGY is excluded at plan time, but a short comment on why a null can still show up (an entry whose vertex was updated in the open transaction) would help the next reader.
  • SelectExecutionPlanner.planDependsOnInputParameters is instance state that is never reset. That is fine as long as a planner instance is used for one createExecutionPlan, but if planners are ever reused it would permanently disable caching. Worth confirming.
  • addLikePrefixRange returns early if any sub-block isIndexAware. Good, but a one-line test that WHERE s = 'x' AND s LIKE 'x%' keeps the equality lookup would lock that in.
  • Explain output next($p) for the successor bound is a little cryptic, though acceptable.

Tests

Good coverage (25 new tests plus QueryHelperPrefixTest), following the repo assertion style, and no timing assertions. The one gap is item 1: nothing covers FLOAT either way.

No security concerns. Performance-wise the change is a clear win (index range or one-end read instead of a type scan), with no extra allocations on the hot paths beyond the one-off bound computation.

Not run locally: I reviewed the diff only, not the test suite.

@lvca
lvca merged commit 5991331 into main Sep 29, 2026
14 checks passed
@lvca
lvca deleted the fix/8666-prefix-like-min-max branch September 29, 2026 17:48
@claude

claude Bot commented Sep 29, 2026

Copy link
Copy Markdown

Review of #8669

Overall this is a careful, well-reasoned change. Keeping the original LIKE / STARTS WITH as a residual filter makes the range a pure superset, so a wrong collation guess cannot change results. The guards on the min/max shortcut (label-level index, LSM, no NULL_STRATEGY INDEX, case sensitive, ordered key types) are also thorough. Test coverage is strong: transactions, deletes, case-insensitive indexes, escapes, parameter plan caching, surrogate edge cases, subtype hierarchies.

I did not compile or run the tests here. This review is from reading the diff.

Things to address

  1. FLOAT was silently dropped from INDEX_ORDERED_KEY_TYPES (CypherOptimizer.java:126). The old set contained Type.FLOAT, the new one does not. The set was already used at CypherOptimizer.java:364 for the existing ORDER BY-from-index optimization, so this quietly disables that optimization for FLOAT properties. It is not mentioned in the PR description and no test covers it. If FLOAT was removed on purpose (for example NaN or -0.0 ordering), please say so in the description and add a test that pins it. If not, restore it. The new declaration line is also well over the file's usual width, so a re-wrap would help.

  2. Javadoc and code disagree on the "defined on the label itself" guard (CypherExecutionPlan.java, tryCreateIndexMinMaxOptimization). The Javadoc says the index must be defined on the label, "not inherited: a parent's index also holds the siblings' vertices". The code only calls type.getIndexByProperties(propertyName) and never checks which type owns the index. It is probably safe because indexesByProperties is per type and minAndMaxOverASubTypeHierarchy passes for DChild. Please confirm that a subtype which only inherits the parent's index falls back to the scan rather than reading the parent's index, and either add a one-line comment saying why the lookup is enough or an explicit check.

  3. STRING min/max and key order. QueryHelper.prefixSuccessor is carefully written to be correct under both UTF-16 and UTF-8 key order. IndexMinMaxStep returns the first or last index entry directly, though, so for STRING keys with supplementary characters or code points above U+D7FF the index's end could differ from what Cypher's min / max returns if the LSM key order is not UTF-16. Please check that the index comparator matches Cypher's string ordering, or add a test with such keys (for example "�" vs "😀").

Minor

  • IndexMinMaxStep class Javadoc has a very long last line (the DISTINCT sentence) that should be wrapped like the rest.
  • New imports in CypherExecutionPlan (RangeIndex, LSMTreeIndexAbstract, Property) are inserted out of order next to TypeIndex; move them next to the other com.arcadedb.index and com.arcadedb.schema imports.
  • In addLikePrefixRange, planDependsOnInputParameters is set as soon as a parameter pattern is found, even if that index descriptor is not the one finally chosen. That only costs an unneeded plan-cache miss, so it is acceptable, but a comment would make the trade-off explicit.
  • If any of the 3,000-vertex tests prove slow, consider @Tag("slow") per CLAUDE.md.

Security / performance

No security concerns: the LIKE prefix bounds are built with plain string operations and no new regex compilation. Performance is the point of the change: index-bounded min/max and prefix range scans instead of full scans, with negligible planning overhead.

@codecov

codecov Bot commented Sep 29, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 82.51366% with 32 lines in your changes missing coverage. Please review.
✅ Project coverage is 75.90%. Comparing base (66a82a8) to head (7adb86a).
⚠️ Report is 31 commits behind head on main.

Files with missing lines Patch % Lines
...query/opencypher/executor/CypherExecutionPlan.java 72.72% 1 Missing and 11 partials ⚠️
...ery/opencypher/executor/steps/IndexMinMaxStep.java 79.06% 4 Missing and 5 partials ⚠️
...edb/query/opencypher/optimizer/AnchorSelector.java 70.58% 0 Missing and 5 partials ⚠️
...ncypher/executor/operators/NodeIndexRangeScan.java 70.00% 1 Missing and 2 partials ⚠️
...edb/query/sql/executor/SelectExecutionPlanner.java 93.93% 0 Missing and 2 partials ⚠️
...ypher/optimizer/statistics/StatisticsProvider.java 0.00% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               main    #8669      +/-   ##
============================================
+ Coverage     75.87%   75.90%   +0.02%     
- Complexity     3485     3487       +2     
============================================
  Files          2058     2059       +1     
  Lines        198662   198838     +176     
  Branches      41803    41864      +61     
============================================
+ Hits         150734   150920     +186     
+ Misses        31499    31467      -32     
- Partials      16429    16451      +22     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant