Skip to content

feat(engine) #5680: CHECK DATABASE RECORD <rid> checks and repairs named records only - #5710

Merged
lvca merged 10 commits into
mainfrom
fix/5680-vertex-delete-ghost-edges
Aug 2, 2026
Merged

lvca merged 10 commits into
mainfrom
fix/5680-vertex-delete-ghost-edges

Conversation

@lvca

@lvca lvca commented Aug 1, 2026 •

Copy link
Copy Markdown
Member

Follow-up to #5680, now that #5707 has merged the strict vertex delete.

This PR was rescoped. It originally carried its own fix for #5680; #5707 landed an equivalent (and in places wider) fix while this was in review, so everything duplicated there has been dropped and this branch was rebuilt on top of main. What remains is the part main does not have.

Why RECORD

#5707 made an ordinary deleteVertex strict: an edge list it cannot walk is a retryable conflict rather than a short collection, and a genuinely broken chain therefore makes its vertex undeletable until CHECK DATABASE ... FIX rebuilds the adjacency from the surviving edge records.

That makes CHECK DATABASE ... FIX the mandated escape hatch — but it could not be aimed. The narrowest scope was TYPE or BUCKET, so recovering one vertex meant two full passes over its entire vertex type.

CHECK DATABASE RECORD #12:3 FIX
CHECK DATABASE RECORD #12:3, #12:9 FIX

The per-record work and the fix are identical to the type-wide run, including the edge-list rebuild. Only the enumeration changes: a lookup per listed RID instead of two passes over every bucket of the type.

Design decisions worth flagging

Rejected, not silently resolved, when combined with TYPE/BUCKET. The grammar accepts the combination; execution refuses it with an IllegalArgumentException (which the HTTP layer already maps to 400). RECORD is already the narrowest scope, so letting one win silently would run a check nobody asked for, and an intersection would be a third semantics nobody asked for either. Enforcing it in the grammar would be stricter but would lose the tailored message.

Database-wide passes skipped. Buckets, external properties and indexes cannot be narrowed to a record, and paying for them would defeat the point of naming one. The orphaned-segment reclaim is skipped for a stronger reason: it builds its reachable set by walking every vertex, so under any partial scope it would misclassify unwalked vertices' segments as orphans.

Two costs it does NOT bound, stated rather than glossed:

  • rebuilding an adjacency means finding every surviving edge that points at the vertex, and no index maps endpoints back to edges — so the scoped run saves the vertex passes and still scans the edge types;
  • a listed record that turns out to be corrupted still has every index on its bucket dropped and rebuilt. That matches type-wide semantics and only fires on genuine corruption (an edge-list rebuild alone deletes no record and triggers none of it), but RECORD bounds the check, not necessarily the fix.

A test that was passing vacuously

Issue5670EdgeDeleteDanglingBackRefTest.assertIntegrityClean asserted totalErrors == 0, and check database reports no totalErrors field — the null-tolerant reader turned it into 0 == 0 on every run, whatever the check found. It now asserts on autoFix, invalidLinks, totalWarnings and totalCorruptedRecords, and longProperty fails loudly on a field that does not exist rather than silently answering 0. Included here because it guards the same area.

Tests

CheckDatabaseRecordScopeTest:

Test Shape
checkDatabaseRecordFixMakesTheVertexDeletableAgain end-to-end repair: break the chain, scoped FIX rebuilds it, the ordinary delete then completes and takes every edge with it
checkDatabaseRecordVisitsOnlyTheNamedRecord proves RECORD is a scope — naming a healthy vertex leaves another vertex's broken chain untouched, so a "scoped" run that quietly fell back to a type scan would fail
checkDatabaseRecordRejectsBeingCombinedWithTypeOrBucket both TYPE and BUCKET, plus each clause still working alone
checkDatabaseRecordReportsARidBelongingToNoType a RID in no type's bucket is reported, not ignored

Plus parser coverage in CheckDatabaseStatementTestParserTest.

Verification

com.arcadedb.graph.**, com.arcadedb.engine.** and com.arcadedb.query.sql.parser.**: 1335 tests, 0 failures — including Issue5680VertexDeleteEdgeCollectionTest from #5707, so the rebase did not disturb the merged fix. DatabaseChecker's getPageSizeForNewFile() change from #5713 is preserved (it landed in the same method this PR touches).

🤖 Generated with Claude Code

https://claude.ai/code/session_01Puky78GhudfgTHPtYSuFPm

@mergify

mergify Bot commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

Tick the box to add this pull request to the merge queue (same as @mergifyio queue).

  • Queue this pull request

@codacy-production

codacy-production Bot commented Aug 1, 2026 •

Copy link
Copy Markdown

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

🟢 Metrics 32 complexity

Metric Results
Complexity 32

View in Codacy

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

@claude

claude Bot commented Aug 1, 2026

Copy link
Copy Markdown

Review: #5680 vertex delete no longer outlives its edges

Thorough, well-reasoned PR. I traced the strict/best-effort split through GraphEngine.deleteVertex, the exception mapping in getEdgeHeadChunkForWrite/EdgeIterator.previousContainer, the RECORD-scope plumbing in DatabaseChecker/GraphDatabaseChecker, and the grammar/AST changes. The core fix is correct and the test suite (including the mutation-checked reverts and the @Tag("slow") concurrency test) is genuinely convincing. A few observations below - none blocking.

Strengths

  • The retryability chain is airtight where it matters: getEdgeHeadChunkForWrite maps a RecordNotFoundException on the head read (including the lazy-load of an ImmutableVertex) to ConcurrentModificationException, and EdgeIterator.previousContainer() does the same on the chunk hop, so an unreadable head or mid-chain chunk both surface as a NeedRetryException rather than a short collection. Transaction atomicity guarantees a mid-delete throw rolls back the already-deleted edge records too.
  • getMostUpdatedVertex re-resolution in both collectEdgesToDelete and deleteEdgeList correctly closes the Cypher: a relationship pattern with a pinned target expands the source's whole edge list #5660 stale-handle window - a nice catch given the per-edge deletes reshape the very list being reloaded.
  • The force neighbour-conflict absorption is the right call: without it, a broken list on an unnamed vertex could hold the repair of the named one hostage.
  • Excellent catch on the vacuous Issue5670...assertIntegrityClean test (totalErrors is never reported, so 0 == 0 always passed). Asserting on real fields + failing loudly on a missing field is the correct fix.
  • The scoped checkVertices/checkEdges mirror the type-wide merge (updateStats + warnings/corruptedRecords addAll + mergeMissingReferences) exactly, and updateStats only sums Long values so there is no double-counting against the Set accumulators.

Minor observations (non-blocking)

  1. TYPE + RECORD together parse but TYPE is silently ignored. The parser test asserts check database type Customer record #12:3 fix is valid syntax, and check() lets recordScope take precedence, ignoring the types/buckets filters entirely. Documented in the PR body, but a user combining them might reasonably expect an intersection. Consider rejecting the combination at parse/exec time, or at least logging that TYPE/BUCKET was ignored.

  2. Unbounded warnings on the "invalid RID" branch. In checkScopedRecords, the type == null path adds directly to the warnings set and bumps totalWarnings without honoring maxWarnings (unlike the addWarning helper used everywhere else). A caller passing a huge list of bogus RIDs could blow past the cap. Low severity, but worth using addWarning for consistency.

  3. RECORD ... FIX on a corrupted record still triggers a full-bucket index rebuild. When a scoped record ends up in corruptedRecords, the shared tail (affectedBuckets -> rebuildIndexes) drops and rebuilds every index on that bucket - a full bucket scan hidden behind a "scoped" command. It matches type-wide semantics and only fires on genuine corruption, but a one-line note that a corrupted scoped record still costs an index rebuild would set expectations.

  4. enrichWithRepairHint drops the hint for a non-CME, non-corrupt-buffer failure. A strict walk that fails with some other RuntimeException (not RecordNotFoundException/CME, not a corrupt-buffer type) propagates untouched, so the operator never sees the CHECK DATABASE RECORD ... FIX guidance even though the vertex is undeletable. Intentional per the Javadoc (callers branch on type), and such a failure is unlikely, but it is the one path where "undeletable vertex" reaches the user without the repair instruction.

Verification

I reviewed against the engine's page-conflict model (engine/CLAUDE.md) and confirmed the strict-path exceptions are all NeedRetryException subtypes, so database.transaction(...) and the server's single-request auto-retry absorb them as claimed. I could not run mvn in this environment to independently reproduce the test run, so I am relying on the reported "401 tests, 0 failures" for com.arcadedb.graph.** plus the checker suites.

Overall: high quality, ship-ready modulo the minor notes above.

@claude

claude Bot commented Aug 1, 2026

Copy link
Copy Markdown

Review: issue #5680 - vertex delete no longer outlives its edges

Reviewed the full diff plus surrounding code in GraphEngine, DatabaseChecker, GraphDatabaseChecker, EdgeIterator/StripedEdgeList and the SQL wiring. This is a careful, well-reasoned fix and the write-up is exemplary: the mutation-checked tests, asserted preconditions, and the symmetry with the #5670 edge-side work all inspire confidence. Structurally everything lines up (edgeIteratorForWrite, allChains(true), getPreviousRID, the scoped checkVertices/checkEdges overloads, stats merge without double-counting). A few points, one worth a closer look.

1. isCorruptedBuffer tolerance can still leave ghost edges on a mid-chain corrupt buffer (medium)

collectEdgesToDelete (GraphEngine.java:764-772) rethrows with the repair hint only when !isCorruptedBuffer(e). The Javadoc intent is that the tolerated case is a vertex whose own record buffer is corrupt, so the edge pointers cannot be read at all - i.e. nothing was collected.

But isCorruptedBuffer keys purely on the exception type (SerializationException, IndexOutOfBoundsException, BufferUnderflowException, IllegalArgumentException, ...), not on where the failure happened. EdgeIterator.previousContainer() only converts RecordNotFoundException into a strict conflict; a chunk whose buffer fails to deserialize mid-walk throws one of the corrupt-buffer family instead. That escapes to this catch, is classified as a corrupted buffer, and is swallowed - after some edges were already added to edgesToDelete. The delete then proceeds with a partial collection and deletes the vertex, leaving every entry behind the corrupt chunk as a ghost edge, with no CHECK DATABASE RECORD ... FIX hint surfaced. That is precisely the outcome the PR sets out to prevent, just reached through a corrupt (rather than missing) mid-chain chunk.

Concretely: a mid-chain EdgeSegment whose page bytes are damaged (not merely relinked away) -> edgeIteratorForWrite yields the head-side edges, then throws SerializationException/IndexOutOfBoundsException -> isCorruptedBuffer true -> swallowed -> vertex deleted, tail edges ghosted.

Suggestion: gate the corrupt-buffer tolerance on edges == null (nothing walked yet), so the #4420/#4432 vertex-body-unreadable case stays tolerant while a failure that occurs after the walk started is treated as a genuinely broken chain and routed to enrichWithRepairHint. That preserves the invariant the whole PR rests on: once collection has started, an incomplete collection must never complete the delete. (Not a perfect split - a corrupt head chunk also leaves edges == null - but it closes the mid-chain hole, the one that silently strips edges after collecting some.)

2. enrichWithRepairHint drops the causal chain (low)

new ConcurrentModificationException(hint) (around GraphEngine.java:830) does not chain cause, so the original stack trace is lost except for what was inlined into hint. Since that type extends NeedRetryException, consider preserving the cause for diagnosability on the run where the retry budget is finally exhausted and the exception does surface.

3. Minor / consistency notes

  • checkScopedDocuments (DatabaseChecker.java:419-426) adds warnings without a maxWarnings cap. This matches the existing type-wide checkDocuments, so it is not a regression - just noting it, given commit 2 deliberately capped the sibling record-belongs-to-no-type branch. If the cap matters for RECORD scope, worth making the two document arms consistent.
  • groupRecordsByType correctly handles an unknown bucket via getTypeByBucketId returning null (verified: plain map lookup, no throw), reported by the type == null branch. Good.
  • The RECORD-vs-TYPE/BUCKET rejection at execution time (rather than in the grammar) is a reasonable trade-off, clearly documented and tested.

Strengths worth calling out

  • The Issue5670 assertIntegrityClean fix is a genuine catch: totalErrors never existed, so the null-tolerant reader made it 0 == 0 forever. Asserting on real fields and failing loudly on a missing one is the right correction.
  • Issue5680VertexDeleteGhostEdgeTest is thorough: deterministic head-chunk and mid-chain shapes, the forced escape hatch, the neighbour-conflict case (comment 2), the end-to-end repair, the scope-isolation test, and the slow-tagged concurrency test. Preconditions are asserted so the tests cannot pass on an unrelated throw.
  • Scope skips (orphaned-segment reclaim disabled under RECORD scope) are correctly reasoned to avoid misclassifying unwalked vertices segments as orphans.

Overall: solid fix, good tests, aligned with the codebase page-conflict/retry model. Point 1 is the one I would like to see addressed or explicitly dismissed before merge.

Reviewed by Claude Code.

@lvca

lvca commented Aug 1, 2026

Copy link
Copy Markdown
Member Author

Round 2 addressed in 4c93ab04b.

1. isCorruptedBuffer could extend the tolerance down the chain — fixed, with a caveat on the evidence.

You are right that keying on the exception type alone does not express the intent. The tolerance is now scoped by construction rather than by classification: isOwnRecordUnreadable reads the vertex's own edge-list pointer first, before anything touches the chain, and only a damaged buffer there is tolerated. Past that probe a corrupt buffer is a broken chain and gets no escape (catch is now plain if (!force) throw). Same probe added to deleteEdgeList. Cost is one extra read of an already-materialised buffer per direction, against a delete that is about to walk the whole list.

I preferred this to your suggested edges == null gate for exactly the reason you flagged about it: a corrupt head chunk also leaves edges == null, and that case should be strict — the vertex record is fine, the chain is broken, and CHECK DATABASE RECORD ... FIX rebuilds it. Probing the vertex record directly splits the two cleanly instead of approximately.

Caveat worth stating plainly: I could not build a fixture where the old classification actually stranded edges. I injected a mid-chain chunk whose used counter points past its buffer — which does throw BufferUnderflowException, i.e. squarely in the tolerated family — and the delete still failed, because the partial collection's per-edge deleteEdge calls and deleteAll() walk the same damaged chain and those paths are strict since #5678. Mutation-checking the new test against a build with both catches reverted to the old classification: it still passes. So vertexDeleteRefusesWhenAMidChainChunkBufferIsDamagedRatherThanMissing pins the contract, it does not prove the old code was exploitable. I kept the scope fix anyway — it makes the invariant explicit instead of incidental on other paths happening to be strict — but I did not want to claim a demonstrated bug I could not demonstrate.

2. Causal chain dropped — fixed. ConcurrentModificationException now exposes the (message, cause) constructor NeedRetryException already had, and enrichWithRepairHint uses it. As you say, the run where that exception actually surfaces is the retry-exhausted one, which is precisely the run whose stack trace has to be diagnosable.

3. checkScopedDocuments cap — fixed. Made consistent with the sibling branch: the retained messages are bounded by maxWarnings, the totals still count every occurrence. Not a regression, but the inconsistency was mine to remove.

Verification: com.arcadedb.graph.** plus the checker/parser suites, 404 tests, 0 failures.

@claude

claude Bot commented Aug 1, 2026

Copy link
Copy Markdown

Code review: #5680 vertex delete no longer outlives its edges

Reviewed the full diff (all 3 commits). This is a high-quality, well-reasoned change. The core fix is correct: routing deleteVertex through the write-side head lookup and a strict chunk hop so an unreadable chunk becomes a retryable ConcurrentModificationException instead of a short collection that silently strands edges. The extraction of collectEdgesToDelete / deleteEdgeList / isOwnRecordUnreadable reads much more clearly than the previous duplicated OUT/IN blocks, and the symmetry with the #5678 edge-delete path (both endpoints strict, force as the single documented escape hatch) is a genuine improvement.

The test suite is a real strength: 8 targeted tests, the mutation-check note (3 fail with the fix reverted, failing in the right way), preconditions that assert the chain really spans several chunks and the victim sits behind the hole, and message matching so a test cannot pass on an unrelated throw. Fixing the vacuously-passing totalErrors assertion in Issue5670EdgeDeleteDanglingBackRefTest (and making longProperty fail loud on a missing field) is a nice catch that hardens an adjacent test.

A few minor points, none blocking:

1. catch (final Throwable e) in the RECORD-scoped checks (GraphDatabaseChecker.checkVertices/checkEdges scoped branches). The type-wide path classifies load failures through a scanType error callback that only fires on load exceptions; the new scoped loop catches Throwable, which also swallows Error (OOM, StackOverflowError) and reports it as "vertex/edge cannot be loaded". catch (final Exception e) would match the type-wide semantics and avoid masking JVM-level failures as record corruption.

2. Breadth of isCorruptedBuffer on the ordinary (!force) path. The family includes IllegalArgumentException and IndexOutOfBoundsException, which are broad. Now that patch 3 correctly scopes the tolerance to isOwnRecordUnreadable (reading the vertex's own already-materialised buffer), the blast radius is small - but on an ordinary delete a spurious IAE/IOOBE there is treated as "tolerate and delete without disconnecting", i.e. it can strand edges without a retry. Given the read is off the vertex's own record buffer and this mirrors the family LocalDatabase.deleteRecord already tolerates, this is acceptable; worth a line confirming no non-corruption path through getInEdgesHeadChunk()/getOutEdgesHeadChunk() can raise those.

3. Forced-delete log message can misattribute. In the edge-delete loop, the NeedRetryException catch logs "cannot disconnect edge ... from the vertex at the other end". Because ConcurrentModificationException extends NeedRetryException, a conflict raised while edge.delete() reshapes this vertex's own list would also land here and be logged as if it came from the neighbour. Functionally fine (force absorbs it), only the message is potentially misleading. Cosmetic.

4. Hot-path cost on healthy deletes. Every vertex delete now does up to four getMostUpdatedVertex calls plus four isOwnRecordUnreadable head-pointer reads (collect + deleteEdgeList, OUT + IN). These are reads of already-materialised buffers so the cost should be negligible, and it is justified by correctness - just flagging it since the delete path is hot.

5. RECORD + TYPE/BUCKET accepted by the grammar, rejected at execution. Rejecting at runtime with a clear IllegalArgumentException (mapped to HTTP 400) is a reasonable call and is tested both at the parser and execution layers. Enforcing it in the grammar would be stricter but would lose the tailored error message; the current choice is defensible.

Overall: correct, well-tested, and the documentation/reasoning in the Javadoc and release notes is unusually thorough. Nice work.

Reviewed with Claude Code.

@lvca

lvca commented Aug 1, 2026

Copy link
Copy Markdown
Member Author

Round 3 addressed in c0456de08.

1. catch (Throwable) in the RECORD-scoped loops — fixed. Narrowed to Exception. Worth spelling out why this mattered more than it looks: in fix mode a record recorded as "cannot be loaded" gets deleted, so masking an OutOfMemoryError as record corruption would have destroyed a healthy record. Good catch.

I deliberately left the shared per-record bodies (checkConnectivity / checkEndpoints) on their existing Throwable catch — that is the type-wide path's long-standing behaviour, and changing it would alter the type-wide check, which is outside this fix's remit. Only the enumeration I added is narrowed.

2. Breadth of isCorruptedBuffer — verified rather than assumed, and documented. You asked for confirmation that no non-corruption path can raise IAE/IOOBE through the head-pointer read. Traced it:

  • ImmutableVertex.parseEdgePointers range-checks the 25-byte prefix and throws SerializationException explicitly — that is the PostCommandHandler: Cannot execute command #4432 fix, added precisely to replace a raw BufferUnderflowException;
  • the lazy load raises RecordNotFoundException, handled separately as transient;
  • MutableVertex returns a field.

So the broad members are not reachable through this call. I kept the set aligned with the family LocalDatabase.deleteRecord tolerates rather than trimming it to today's reachable subset — over-tolerating at the probe fails toward "a corrupt vertex stays deletable", and trimming it would let a future serializer change silently recreate the #4420/#4432 complaint. Reasoning recorded in the Javadoc.

3. Log line could misattribute — fixed. You are right that ConcurrentModificationException extends NeedRetryException means a conflict on this vertex's own list lands in the same catch. Reworded to "one of its endpoints" rather than guessing.

4. Hot-path cost — reduced. Acting on the flag rather than just noting it: deleteEdgeList's isOwnRecordUnreadable probe is redundant — it only runs for a direction whose collection already passed the probe on the same record, so the edge pointers are known readable. Removed, halving the added reads. The remaining two (one per direction, off an already-materialised buffer) sit in front of a full edge-list walk.

5. Grammar vs runtime rejection — agreed, keeping the runtime rejection for the tailored message.

Verification: com.arcadedb.graph.** plus the checker/parser suites, 404 tests, 0 failures.

@claude

claude Bot commented Aug 1, 2026

Copy link
Copy Markdown

Code review: PR 5710 - strict vertex delete + CHECK DATABASE RECORD

I read the full diff plus the surrounding GraphEngine, DatabaseChecker, GraphDatabaseChecker, the grammar/AST wiring and the new test. This is a careful, well-reasoned change - the fix mirrors the edge-side strictness from issues 5670/5678 symmetrically on the vertex side, the tolerances are deliberately scoped, and the Javadoc explains why at each decision point. The tests are strong: preconditions are asserted so they cannot pass vacuously, the mutation-check callout is credible, and the damaged-vs-missing chunk distinction is pinned. Nice work.

A few observations, mostly minor.

Correctness / behaviour

  1. Repair hint is lost from the exception message on the damaged-buffer path (minor UX). 'enrichWithRepairHint' re-creates only ConcurrentModificationException with the hint in its message. For a genuinely broken chain caused by a damaged mid-chain chunk buffer (the vertexDeleteRefusesWhenAMidChainChunkBufferIsDamagedRatherThanMissing case), the walk throws a BufferUnderflowException/IndexOutOfBoundsException, which is a non-CME RuntimeException and therefore propagates untouched - the hint is only logged, not surfaced to the SQL client. So the one operator who hits a permanently-broken chain via corruption (rather than a missing record) gets a raw BufferUnderflowException at the API with no 'CHECK DATABASE RECORD ... FIX' guidance in the message itself. The reasoning for not re-typing (callers branch on exception type) is sound, but consider whether the message could still carry the hint, or at least documenting that this failure class only names the repair in the log. The test asserts isInstanceOf(Exception.class), so it would not catch a regression in the hint here.

  2. force == true leaves a neighbour-side ghost edge (intended, worth a doc line). When a forced delete absorbs a NeedRetryException from a neighbour's unreadable list, the edge record and the neighbour's back-reference both survive while this vertex is removed - i.e. the neighbour now points at a deleted vertex. This is the documented 'dangling reference left for CHECK DATABASE' outcome and is correct for the escape hatch, but the class Javadoc frames the tolerance mainly as 'a reference to it is left dangling'; noting that the neighbour keeps a forward ref to a now-missing vertex would make the residual state explicit for whoever runs the repair afterward.

Tests

  1. concurrentVertexDeleteAndAppendNeverLeaveAGhostEdge mutates a global. It sets GlobalConfiguration.TX_RETRY_DELAY to 1 and restores it in finally (good). Since that is process-global static state, if Surefire ever runs test classes concurrently in the same JVM it could perturb unrelated tests during the window. It is @tag(slow) and restored, so risk is low - just flagging in case CI parallelism changes.

Things I checked that are fine

  • No double-counting in the RECORD scope: updateStats only merges Long values, and checkScopedRecords explicitly merges warnings/corruptedRecords/missingReferences afterward - identical to the type-wide merge pattern. The null-type and document branches increment totalWarnings/totalCorruptedRecords directly, which is consistent since they do not go through the graph checker.
  • Grammar: RECORD is already a lexer token and in the identifier keyword set, so 'CHECK DATABASE TYPE record ...' still parses; the new optional clause composes with FIX/COMPRESS. Parser test covers it.
  • Strict-walk exception surface: getEdgeHeadChunkForWrite, the strict chunk hop, and StripedEdgeList.allChains(true) all funnel transient failures to ConcurrentModificationException (a NeedRetryException), so the retry loop absorbs them; only genuine non-retryable failures reach the un-hinted branch of enrichWithRepairHint.
  • No partial state on strict failure: all per-edge deletes happen after both collectEdgesToDelete passes, so a throw during collection rolls back with nothing removed.
  • Head re-resolution: deleteEdgeList re-reads through getMostUpdatedVertex after the per-edge deletes reshaped the list (issue 5660), avoiding a spurious stale-head conflict.
  • isOwnRecordUnreadable scoping: reading the head pointer before touching the chain confines the 4420/4432 corrupt-buffer tolerance to the vertex's own record, so a damaged mid-chain chunk (same exception family) is not swallowed - the subtle bug the design guards against, and the test pins it.
  • No new engine thread pools introduced, so the engine-concurrency guidance is not implicated.

Overall this looks solid and safe to merge; items 1-3 are polish, not blockers.

@lvca

lvca commented Aug 1, 2026

Copy link
Copy Markdown
Member Author

Round 4 addressed in c6e5de867. All three taken, and point 1 turned out to matter more than "minor UX".

1. Repair hint lost from the message on the damaged-buffer path — fixed.

You were right, and you were also right that the test would not have caught it: isInstanceOf(Exception.class) passed on the raw BufferUnderflowException. Non-CME failures are now wrapped in DatabaseOperationException carrying the hint and the cause, so the operator facing a permanently undeletable vertex gets CHECK DATABASE RECORD ... FIX in the message, not only in a log line they may never read.

I checked the objection I had originally raised against wrapping (callers branch on the type) rather than trusting it: the corrupt-buffer family LocalDatabase.deleteRecord keys on is caught around the indexer read, and the deleteVertex call site catches ConcurrentModificationException alone. So wrapping breaks no branching. The retryable conflict is still re-created as its own type — re-typing that one would stop the retry loop absorbing it and break the confirmed-broken-chain fallback.

Same treatment added to the per-edge disconnect loop, naming the edge as well, since the broken list may be the one on the vertex at the other end.

The test now pins the exception type, the CHECK DATABASE RECORD text and the root cause, so this cannot silently regress again.

2. Forced delete leaves a neighbour-side dangling reference — documented. Added to both the deleteVertex Javadoc and the log site: the edge record survives, so the neighbour is left holding a back-reference to an edge whose other endpoint is gone, which CHECK DATABASE reports as an invalid link and fix mode drops. Worth having explicit for whoever runs the repair.

3. Global TX_RETRY_DELAY mutation — removed rather than just flagged. The test no longer touches it. The retry budget was already far beyond what the contention needs, so the default delay costs only wall-clock, and there is now no process-global state for the test to perturb if CI parallelism ever changes.

Verification: com.arcadedb.graph.** plus the checker/parser suites, 404 tests, 0 failures.

…med records only

CHECK DATABASE could be narrowed to a TYPE or a BUCKET, but not to a record. That is the sharp edge
of the strict vertex delete merged in #5707: a genuinely broken edge chain makes its vertex
undeletable until CHECK DATABASE ... FIX rebuilds the adjacency from the surviving edge records, and
the smallest way to ask for that repair was two full passes over the whole vertex type.

  CHECK DATABASE RECORD #12:3 FIX
  CHECK DATABASE RECORD #12:3, #12:9 FIX

The per-record work and the fix are identical to the type-wide run, including the edge-list rebuild;
only the enumeration changes. Combines with FIX and COMPRESS. Rejected when combined with TYPE or
BUCKET - RECORD is already the narrowest scope, so letting one silently win would run a check nobody
asked for and an intersection would be a third semantics nobody asked for. The database-wide passes
(buckets, external properties, indexes) and the orphaned-segment reclaim are skipped, since none can
be narrowed to a record.

Two costs it deliberately does not bound, documented rather than glossed: rebuilding an adjacency
needs every surviving edge that points at the vertex and no index maps endpoints back to edges, so
the scoped run saves the vertex passes but still scans the edge types; and a listed record that
turns out to be corrupted still has every index on its bucket rebuilt.

Also fixes a vacuously-passing assertion in Issue5670EdgeDeleteDanglingBackRefTest: it asserted
totalErrors == 0, and check database reports no such field, so the null-tolerant reader turned it
into 0 == 0 on every run whatever the check found. It now asserts on autoFix, invalidLinks,
totalWarnings and totalCorruptedRecords, and fails loudly on a field that does not exist.
@lvca
lvca force-pushed the fix/5680-vertex-delete-ghost-edges branch from c6e5de8 to 1e55f6c Compare August 1, 2026 21:02
@lvca lvca changed the title fix(engine) #5680: a vertex delete no longer outlives its edges when a chunk is unreadable feat(engine) #5680: CHECK DATABASE RECORD <rid> checks and repairs named records only Aug 1, 2026
The setup job failed on a trailing-whitespace line an earlier edit left after a comment in the
RECORD-scoped edge branch. Removed, along with the stray blank line beside it.
@claude

claude Bot commented Aug 1, 2026

Copy link
Copy Markdown

Review: CHECK DATABASE RECORD <rid> (#5710)

Overall this is a clean, well-scoped follow-up. The shared-Consumer refactor in GraphDatabaseChecker keeps the per-record logic genuinely identical between the type-wide and scoped paths (no forked semantics to drift), the combination-with-TYPE/BUCKET rejection is the right call, the warning caps are propagated correctly, and the docs/comments are unusually candid about the two costs the scope does not bound. The rewrite of assertIntegrityClean to stop reading a totalErrors field that check database never emits is a good catch on its own. A few things worth a look:

1. Progress step index double-counts document-scoped RIDs (minor)

In checkScopedRecords every group does ++currentStep (DatabaseChecker.java:376), but the document arm then calls checkScopedDocuments -> stepBegin (:421), which does ++currentStep again (:540). Meanwhile totalSteps is budgeted at scoped.size() + ... (:149), i.e. one step per group. So when the RID list contains a plain-document RID, currentStep overruns totalSteps and progress is reported as e.g. "step 3 of 2". The edge/vertex arms don't hit this because they pass the fixed currentStep into the nested GraphDatabaseChecker rather than calling stepBegin. Harmless (progress reporting only, and stepTick clamps stepDone), but the step index isn't clamped. Simplest fix: don't ++currentStep at :376 for the document branch, or have checkScopedDocuments not call stepBegin.

2. A merely non-existent listed RID triggers a full-bucket index rebuild under FIX (worth a guard)

The checkVertices/checkEdges scoped arms do database.lookupByRID(rid, true); a RID whose bucket exists but whose slot is empty/deleted throws RecordNotFoundException, which is caught and recorded via addCorrupted(...). That RID's bucket then lands in affectedBuckets, and under FIX the shared tail drops and rebuilds every index on that bucket (a full bucket scan) - exactly the cost RECORD scope exists to avoid. The documented caveat is about corrupted records, but a simple typo'd or already-deleted RID isn't corruption. Given the intended workflow is "operator hand-types a RID after a failed delete", this is an easy foot-gun. Consider distinguishing not-found from corrupted (e.g. skip addCorrupted when the record simply doesn't exist), or at least documenting that naming a stale RID with FIX rebuilds that bucket's indexes.

3. Stray blank line with trailing whitespace

GraphDatabaseChecker.java:1233 - the edge arm's catch has an empty line (with trailing spaces) right after the comment. The vertex arm doesn't. Trivial, just tidy it.

4. Test coverage gaps

CheckDatabaseRecordScopeTest is solid for the vertex path, but a few reachable branches are untested:

  • Edge-typed RID in the RECORD scope (LocalEdgeType arm of checkScopedRecords).
  • Document-typed RID (checkScopedDocuments) - also the branch that trips finding build support #1.
  • Multiple RIDs spanning different types in one command, which is the whole point of groupRecordsByType / LinkedHashSet ordering.
  • RECORD ... FIX on a non-existent-but-valid-bucket RID would pin down the behavior in finding Bump gremlin.version from 3.4.10 to 3.5.1 #2.

Also a naming nit: the parser test comment references Issue5680VertexDeleteGhostEdgeTest.checkDatabaseRecordRejectsBeingCombinedWithTypeOrBucket, but that method now lives in CheckDatabaseRecordScopeTest - update the reference so it doesn't dangle.

None of these are blockers; #1 and #2 are the only ones with runtime effect and both are narrow. Nice work on the documentation discipline throughout.

(Note: reviewed statically; I did not run the build/tests myself - the PR reports 1335 green in the affected packages.)

@claude

claude Bot commented Aug 1, 2026

Copy link
Copy Markdown

Review: CHECK DATABASE RECORD <rid> (#5710)

Overall this is a clean, well-scoped follow-up. The refactor of checkVertices/checkEdges into a shared per-record Consumer with two enumerators (type scan vs. lookupByRID) is the right shape - it keeps the per-record check and the fix identical to the type-wide path, which is exactly what you want for a scoped repair. The Exception-not-Throwable reasoning (so an Error never makes FIX delete a healthy record) is a nice touch, and the PR description/Javadoc are unusually clear about what the scope does and does not bound. Rejecting RECORD + TYPE/BUCKET at execution with a tailored message rather than silently picking a winner is the right call.

A few things worth addressing:

1. Progress step double-count for document RIDs (minor, cosmetic)

In DatabaseChecker.checkScopedRecords() each group does ++currentStep, but the document arm then calls checkScopedDocuments(), which calls stepBegin() - and stepBegin() also does ++currentStep. So a scope containing document RIDs advances currentStep by 2 per group, while totalSteps (scoped.size() + fix + compress) budgets 1 per group. A CHECK DATABASE RECORD over document records will therefore report step indexes past totalSteps. The vertex/edge arms are correct (they pass currentStep straight to GraphDatabaseChecker.setProgress without a second stepBegin). Only progress reporting is affected, not correctness. Simplest fix: don't ++currentStep in the loop for the document branch (let stepBegin own it), or drop the stepBegin increment there.

2. Stale class reference in test comment (trivial)

CheckDatabaseStatementTestParserTest.recordScope() points at Issue5680VertexDeleteGhostEdgeTest.checkDatabaseRecordRejectsBeingCombinedWithTypeOrBucket, but that class doesn't exist - the test lives in CheckDatabaseRecordScopeTest. Worth fixing so the cross-reference stays navigable.

3. Null-element robustness in setRecords (low)

groupRecordsByType() calls rid.getBucketId() on each element, and setRecords() guards against a null/empty set but not null elements. Rid.toRecordId(...) can return null (expression evaluating to null), so the stream in CheckDatabaseStatement.execute() could feed a null into the set. Not reachable through the RECORD #n:n literal grammar (legacy RIDs never return null), so this is defensive only - but since setRecords(Set<RID>) is also a public API, filtering nulls in the stream/setRecords would make it robust against a caller or a future non-literal RID form.

Test coverage

The four scope tests are well-chosen - the "visits only the named record" test in particular guards against a silent fall-back to a type scan, which is the failure mode that matters most here. Two gaps:

  • No test exercises the document-type record scope (checkScopedDocuments) - which is also where finding build support #1 lives - nor an edge RID scope directly (only vertex hubs and the no-type RID are covered).
  • The vacuous-assertion fix in Issue5670EdgeDeleteDanglingBackRefTest is a genuinely good catch (totalErrors was never emitted, so 0 == 0 passed unconditionally); asserting on autoFix/invalidLinks/totalWarnings/totalCorruptedRecords plus the loud longProperty null check is the right hardening.

Nits

  • checkScopedRecords passes Math.max(0, maxWarnings - currentCorrupted) as the maxCorrupted argument (bounding the corrupted cap by the single maxWarnings field) - consistent with the existing checkEdges/checkVertices callers, so this is carrying forward the existing convention, not a new issue.

None of these are blockers. #1 and #2 are quick fixes; #3 is optional hardening. Nice work.

Reviewed against CLAUDE.md conventions; I read the diff but did not re-run the 1335-test suite cited in the description.

…ogress step stops double-counting

Two independent review passes on the RECORD scope, four actionable points between them.

- A listed RID that simply is not there was recorded through addCorrupted, which puts its BUCKET
  into affectedBuckets - so CHECK DATABASE RECORD ... FIX dropped and rebuilt every index on that
  bucket, a full bucket scan, over what is usually a mistyped or already-deleted RID. That is
  exactly the cost the scope exists to avoid, and the scope is meant to be hand-typed after a failed
  delete. RecordNotFoundException is now reported and never flagged corrupted, in all three arms
  (vertex, edge, document). Mutation-checked: restoring addCorrupted fails the new test with
  totalCorruptedRecords=1, autoFix=1.
- checkScopedRecords bumped currentStep for every group, but the document arm reaches stepBegin,
  which bumps it again - so a scope containing a plain document reported "step 3 of 2" against a
  totalSteps budgeted at one step per group. Each arm now owns its bump; the graph arms need it
  because they pass a fixed value into the nested GraphDatabaseChecker instead.
- setRecords and the statement's stream now drop null RIDs. Rid.toRecordId can answer null for a
  non-literal form, and groupRecordsByType would meet it as an NPE on getBucketId(). Not reachable
  through the RECORD #n:n literal grammar, but setRecords(Set<RID>) is public API.
- The parser test's cross-reference named a test class that no longer exists after the rescope.

Tests: four added for the branches that were unexercised - an edge-typed RID, a document-typed RID
(the branch the progress bug lived in), several RIDs spanning different types in one command, and a
missing-but-valid RID under FIX, which pins the first item above.
@lvca

lvca commented Aug 1, 2026

Copy link
Copy Markdown
Member Author

Both review passes addressed in bb08db04a. Four actionable points between them:

1. A missing RID was treated as corruption — fixed, and this one had teeth.

addCorrupted puts the record's bucket into affectedBuckets, and under FIX the shared tail drops and rebuilds every index on that bucket — a full bucket scan, over what is usually a mistyped or already-deleted RID. Since the whole point of RECORD is to be hand-typed after a failed delete, that foot-gun bought precisely the cost the scope exists to avoid. RecordNotFoundException is now reported and never flagged corrupted, in all three arms (vertex, edge, document).

Mutation-checked rather than assumed: restoring addCorrupted on that path fails the new test with totalCorruptedRecords=1, autoFix=1.

2. Progress step double-count — fixed. Every group bumped currentStep, but the document arm then reaches stepBegin, which bumps it again, against a totalSteps budgeted at one step per group — hence "step 3 of 2". Each arm now owns its bump; the graph arms need theirs because they pass a fixed value into the nested GraphDatabaseChecker rather than calling stepBegin.

3. Null RIDs — hardened. setRecords and the statement's stream now drop nulls. Not reachable through the RECORD #n:n literal grammar (legacy RIDs never answer null), but setRecords(Set<RID>) is public API and groupRecordsByType would meet a null as an NPE on getBucketId().

4. Stale cross-reference — fixed. The parser test named a class that no longer exists after the rescope.

Test gaps — closed. Four added, for the branches nothing exercised: an edge-typed RID, a document-typed RID (the branch the progress bug lived in), several RIDs spanning different types in one command (the point of groupRecordsByType), and a missing-but-valid RID under FIX, which pins item 1. Eight tests total.

On the verification numbers

Being precise, since my earlier "1335 green" is worth qualifying: a sweep reported 3 failures (TimeSeriesTagDictionaryTest, LightweightEdgeTypeTest, LightweightEdgeBenchmark), and they were my own tooling artifact — I had two Maven runs writing to the same target/ concurrently, so Error on creating new database instance came from leftover database directories. Re-run in isolation on a clean target/databases: 26/26 pass, and none of the three touch code this PR changes. Flagging it rather than quietly reporting the clean number.

@codacy-production

codacy-production Bot commented Aug 1, 2026 •

Copy link
Copy Markdown

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

🟢 Metrics 44 complexity

Metric Results
Complexity 44

View in Codacy

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

@claude

claude Bot commented Aug 1, 2026

Copy link
Copy Markdown

Review: CHECK DATABASE RECORD <rid> (#5710)

Reviewed the full diff against main. This is high-quality, unusually well-documented work: the scoped path faithfully mirrors the type-wide aggregation (updateStats + warnings/corruptedRecords/mergeMissingReferences), the shared index-rebuild tail is reused rather than reimplemented, and the progress-step accounting is careful and directly tested. A few observations, mostly minor.

Strengths

  • Corrupted-vs-missing distinction has real teeth. Treating a missing/typo'd RID as a warning rather than corruption avoids putting its bucket into affectedBuckets, which would otherwise trigger a full drop-and-rebuild of every index on that bucket - exactly the cost the scope exists to avoid. Nice, and explicitly regression-tested (checkDatabaseRecordDoesNotTreatAMissingRidAsCorruption).
  • catch (Exception) not Throwable in the scoped arms, so an Error (OOM/StackOverflow) cannot be misrecorded as record corruption and get a healthy record deleted under FIX. Good instinct.
  • Progress bookkeeping (the "step 3 of 2" comment / per-arm currentStep bump vs stepBegin) is subtle and covered by checkDatabaseRecordAcceptsADocumentRid.
  • The vacuous-test fix (Issue5670EdgeDeleteDanglingBackRefTest): asserting on totalErrors, a field check database never emits, made the null-tolerant reader collapse to 0 == 0. Switching to autoFix/invalidLinks/totalWarnings/totalCorruptedRecords and making longProperty fail loudly on a missing field is a genuine improvement, not just churn.
  • getTypeByBucketId returns null for an unknown bucket, so the #9999:0 null-key grouping is safe - verified, and tested.

Minor points

  1. Silent fallback to a full-database check when every RID resolves to null. In CheckDatabaseStatement.executeSimple the RIDs are mapped through toRecordId(...) and filter(Objects::nonNull), and DatabaseChecker.setRecords drops nulls again; if the result is empty, recordScope is false and the command runs a full CHECK DATABASE. Via the literal-RID grammar this cannot happen (a literal #x:y always resolves), so it is an edge case reachable only through the public setRecords API or an unusual RID form. Still, a CHECK DATABASE RECORD ... that quietly becomes a whole-database scan is a surprising outcome - worth a log line (or a rejection) when the input list was non-empty but everything filtered out.

  2. Over-cap warnings dropped silently in checkScopedDocuments. When warnings.size() >= maxWarnings the message is discarded, whereas GraphDatabaseChecker.addWarning logs the dropped message at WARNING. Small inconsistency within the same feature; the type == null arm in checkScopedRecords has the same behavior. Low priority.

  3. Duplicated catch blocks. The RecordNotFoundException / generic-Exception handling in the scoped vertex arm and scoped edge arm of GraphDatabaseChecker is verbatim-identical. Consistent with the surrounding style (the two type-wide methods already parallel each other closely), so not a blocker - just flagging it as a candidate for a shared helper if these paths grow.

Tests

Coverage is excellent and the intent behind each test is stated: end-to-end repair, RECORD genuinely being a scope (checkDatabaseRecordVisitsOnlyTheNamedRecord guards against a silent fall-through to a type scan), TYPE/BUCKET rejection, RID-in-no-type, edge/document arms, and multi-type grouping - plus parser coverage. The one small gap is a test for point 1 above (non-empty RECORD list that resolves to nothing), but that path is not reachable from SQL, so it is optional.

Overall: LGTM. The points above are all minor/optional.

…, not widened to the whole database

Review round on the rescoped PR: LGTM with three minor points, two taken.

- CHECK DATABASE RECORD whose RIDs all resolve to null left the checker with an empty scope, which
  reads as "no scope" - so a command that explicitly named records quietly ran a FULL database
  check. The review judged this unreachable from SQL; it is not. Only the `#n:n` literal always
  resolves, while the non-literal {"@Rid": <expression>} form goes through an expression that can
  answer null, so `CHECK DATABASE RECORD {"@Rid": null}` reached it. Refused now, with a test that
  fails when the guard is stubbed out.
- A warning dropped for exceeding maxWarnings was discarded silently in the two scoped arms, where
  GraphDatabaseChecker.addWarning logs it. Extracted addScopedWarning so both behave the same:
  always counted, retained under the cap, logged when dropped.

Not taken: extracting the duplicated RecordNotFoundException/Exception catch blocks in the scoped
vertex and edge arms. They would need six accumulators threaded through a shared helper, which reads
worse than the duplication, and the surrounding type-wide methods already parallel each other the
same way - the review noted as much in raising it.
@lvca

lvca commented Aug 1, 2026

Copy link
Copy Markdown
Member Author

Addressed in eb616d951. Two of the three taken; the third dismissed with reasoning.

1. Silent fallback to a full-database check — fixed, and it was more reachable than we both thought.

You judged this "reachable only through the public setRecords API or an unusual RID form" because a literal #x:y always resolves. That is true of the literal, but the grammar's rid rule also accepts the non-literal {"@rid": <expression>} form, and Rid.toRecordId returns null when that expression evaluates to null. So this reaches it straight from SQL:

CHECK DATABASE RECORD {"@rid": null}

and quietly became a whole-database scan. Now refused with a message naming how many records were given and that none resolved. Mutation-checked: stubbing the guard to if (false) fails the new test.

Worth flagging since it moves the finding from "defensive only" to "a user can hit this", and on a large database the difference between the requested scope and a full check is not small.

2. Over-cap warnings dropped silently — fixed. Extracted addScopedWarning, used by both the type == null arm and checkScopedDocuments: always counted, retained under maxWarnings, and logged when it has to be dropped, matching GraphDatabaseChecker.addWarning.

3. Duplicated catch blocks — not taken. A shared helper would need the six accumulators (warnings, totalWarnings, maxWarnings, corruptedRecords, totalCorrupted, maxCorrupted) threaded through it, which reads worse than the six duplicated lines, and the two type-wide methods those arms live in already parallel each other the same way — as you noted in raising it. Happy to revisit if these paths grow a third arm.

Verification

com.arcadedb.graph.**, com.arcadedb.engine.**, com.arcadedb.query.sql.parser.** plus CheckDatabaseExtendedTest: 1352 tests, 0 failures, BUILD SUCCESS.

This run also settles the artifact I flagged last time: run as a single Maven invocation with a clean target/databases, the three failures I had reported (TimeSeriesTagDictionaryTest, LightweightEdgeTypeTest, LightweightEdgeBenchmark) do not occur. They were caused by two of my own Maven runs sharing a target/ directory, not by anything in this branch.

@claude

claude Bot commented Aug 1, 2026

Copy link
Copy Markdown

Review: CHECK DATABASE RECORD <rid> (#5710)

Reviewed the full diff and traced the touched paths in DatabaseChecker, GraphDatabaseChecker, CheckDatabaseStatement, the grammar, and the tests. Overall this is a clean, well-scoped feature: the extension reuses the existing per-record workers, the enumeration is the only thing that changes, and the design trade-offs (skipped DB-wide passes, unbounded edge scan, index rebuild on genuine corruption) are documented both in code and in the release notes. The tests are unusually strong - several are mutation-checked and the "is it really a scope" test guards against a silent type-wide fallback. Nice work.

A few observations, none blocking.

Correctness / robustness

1. The empty-scope widening guard lives only in the SQL layer, not in the public API. CheckDatabaseStatement.executeSimple refuses a RECORD list that resolves to nothing (good), but DatabaseChecker.setRecords(Set<RID>) silently drops nulls and, if the result is empty, check() reads it as "no scope" and runs a full database check. An embedded caller doing new DatabaseChecker(db).setRecords(setOfAllNulls).check() hits exactly the widening the SQL guard was added to prevent. The distinction ("named records but none resolved" vs "no records named") genuinely can't be reconstructed inside setRecords, so I'm not asking to move the guard - but a one-line Javadoc on setRecords/check() warning that an empty resolved set means full scope would be worth it, since setRecords is the documented public surface (the new test even asserts this is "the public API surface where it IS reachable", yet the protection isn't there).

2. Inherited misleading message in the document arm. checkScopedDocuments flags a corrupted document with "... cannot be loaded, removing it" and adds it to corruptedRecords, but as with the type-wide checkDocuments, nothing actually deletes the record - corruptedRecords only drives index rebuild. This is not introduced here, and the new arm is actually more correct than the type-wide one (which says "vertex ... removing it" for a document), so no action needed - just flagging that the "removing it" wording is aspirational in both places.

Test coverage

  • The parser test covers RECORD ... FIX COMPRESS, but no execution test exercises the combined FIX+COMPRESS path through the scoped enumeration, nor the maxWarnings/addScopedWarning cap. Both are low-risk (shared code), but a bogus-RID-flood test would pin the cap behavior cheaply.
  • checkDatabaseRecordAcceptsAnEdgeRid deletes the IN endpoint via bucket.deleteRecord and asserts invalidLinks > 0 - good coverage of the edge arm.

Style / minor

  • checkScopedRecords: the per-arm ++currentStep split (null arm, graph arms, and stepBegin in the doc arm) is subtle. The comment explains it well, and the follow-up commit fixed the "step 3 of 2" double-count with a test - just noting this is the kind of thing that is easy to regress; the checkDatabaseRecordAcceptsADocumentRid test guarding the budget is the right call.
  • The RecordNotFoundException-before-Exception catch ordering and the Exception-not-Throwable choice (so an Error never gets a healthy record deleted in FIX mode) are both correct and well-commented.

Verification claims in the PR description (1335 tests across com.arcadedb.graph/engine/query.sql.parser) line up with the surface area touched. LGTM once the setRecords widening note is considered.

… protects the public API too

Review round: LGTM once the setRecords widening note was considered. Went one step further than the
note it asked for.

The guard against "a RECORD scope that resolves to nothing silently becomes a full database check"
lived only in the SQL layer, so an embedded caller reaching DatabaseChecker.setRecords directly
still hit the widening. The review reasoned that setRecords cannot tell "named records, none
usable" from "named no records" - true only because the SQL layer filtered the nulls out BEFORE
calling it. It no longer does: the nulls are passed through, setRecords owns both the filtering and
the refusal, and one guard now covers both entry points. A genuinely empty (or null) set still means
"no scope" and is accepted. Both are asserted.

Also: the document arm said "cannot be loaded, removing it", but nothing removes the record there -
corruptedRecords only drives the index rebuild. The review called the wording inherited and asked
for no action; it is new code here, so it now says what is true, with a note on why it diverges from
the type-wide checkDocuments (which also calls a document a "vertex").

Adds the warning-cap test the review asked for: 40 bogus RIDs against maxWarnings=10 retains 10
messages and still counts 40.

Not added: an execution test for RECORD ... FIX COMPRESS. The compress pass is shared, unscoped code
already covered elsewhere, and routing it through the scoped enumeration exercises no new branch.
@lvca

lvca commented Aug 1, 2026

Copy link
Copy Markdown
Member Author

Addressed in a8e5e6eab.

1. setRecords widening — guarded, not just documented.

You suggested a Javadoc note on the grounds that the distinction "named records but none resolved" vs "named no records" "genuinely can't be reconstructed inside setRecords". That was true only because the SQL layer was filtering the nulls out before calling it. It no longer does — the nulls are passed through, and setRecords owns both the filtering and the refusal. One guard now covers both the SQL path and the embedded caller you described (new DatabaseChecker(db).setRecords(setOfAllNulls).check()), and the inconsistency you spotted in my own test comment is gone.

A genuinely empty set — and null — still mean "no scope" and are accepted, which the test also pins, so the guard cannot creep into refusing legitimate whole-database checks.

2. Document arm's "removing it" — fixed. You judged this inherited and needing no action. It is inherited in checkDocuments, but the scoped arm is new code in this PR, and nothing there removes the record (corruptedRecords only drives the index rebuild). It now says what is true, with a comment recording why it diverges from the type-wide version — which, as you noted, also calls a document a "vertex".

3. maxWarnings cap test — added. 40 bogus RIDs against maxWarnings=10: 10 messages retained, 40 counted. That pins addScopedWarning, which was previously only exercised incidentally.

Not added: an execution test for RECORD ... FIX COMPRESS. The compress pass is shared, unscoped code already covered elsewhere; routing it through the scoped enumeration exercises no new branch, so the test would cost a compress pass and assert nothing this PR changed. Happy to add it if you disagree.

Verification

com.arcadedb.graph.**, com.arcadedb.engine.**, com.arcadedb.query.sql.parser.** plus CheckDatabaseExtendedTest: 1353 tests, 0 failures, BUILD SUCCESS.

@claude

claude Bot commented Aug 1, 2026

Copy link
Copy Markdown

Review: CHECK DATABASE RECORD (PR #5710)

I read the full diff plus the surrounding DatabaseChecker / GraphDatabaseChecker flow. This is a clean, well-scoped follow-up and the implementation holds up under scrutiny. Summary: approve, with a few minor notes below - none blocking.

What works well

  • Correctness of the shared worker split. Extracting the per-record checkConnectivity / checkEndpoints Consumer<Record> and dispatching both the type-wide scan and the scoped enumeration through it means the RECORD path can never drift from type-wide semantics. Good reuse.
  • Progress accounting is consistent. totalSteps = scoped.size() + fix + compress, each group bumps currentStep exactly once (graph/no-type arms via ++currentStep, document arm via stepBegin), and the per-record progressTick() fires exactly once per RID on both the success and each catch path. The "step 3 of 2" note in checkScopedRecords shows this was actually exercised.
  • The missing-RID vs corrupted-RID distinction has real teeth and is the right call: a stale/mistyped RID only produces a warning, so FIX does not drop+rebuild every index on that bucket - exactly the cost the scope exists to avoid. Covered by checkDatabaseRecordDoesNotTreatAMissingRidAsCorruption.
  • Exception vs Throwable in the scoped catch (an Error must not be recorded as record corruption and have FIX delete a healthy record) is a subtle, correct choice, and the comment explains it.
  • setRecords refusing an all-null resolved scope rather than silently widening to a full-database scan is the safe default, and putting the guard at the public entry point (not just the SQL layer) is correct - it is the last place that can distinguish "named records, none usable" from "named no records".
  • No double-counting of totalWarnings: graph arms go through updateStats (which sums the Long stats), the document/no-type arms increment via addScopedWarning and are never passed through updateStats. Verified consistent.
  • Test suite is genuinely adversarial - checkDatabaseRecordVisitsOnlyTheNamedRecord in particular guards against a "scoped" run that quietly falls back to a type scan, the failure mode that would otherwise pass every other assertion. The Issue5670 fix (asserting on fields check database actually reports, so a null-tolerant reader can no longer turn the check into 0==0) is a real improvement and worth carrying here.

Minor notes (non-blocking)

  1. COMPRESS under a RECORD scope still compresses the whole database. It is user-opt-in and the docs mention it combines with COMPRESS, but it is the one clause that breaks the "naming one record bounds the cost" promise. Worth an explicit line in the release note so nobody pairs RECORD ... COMPRESS expecting a cheap run.
  2. Grammar accepts TYPE/BUCKET + RECORD, execution rejects it. The trade-off (runtime IllegalArgumentException to keep the tailored message vs. a stricter grammar) is reasonable and documented. One consequence: a programmatic DatabaseChecker caller that sets both types and records only learns of the conflict at check() time, not at the setter. Acceptable, just slightly surprising for the embedded API.
  3. Test-coverage gap (optional). There is no test for a RECORD-scoped FIX where a listed vertex is genuinely corrupted (not merely chain-broken), asserting the documented "every index on its bucket is dropped and rebuilt" behaviour. The missing-RID case is covered; a corrupted-record case would turn the "RECORD bounds the check, not necessarily the fix" paragraph into an executable assertion. The scoped arm shares the type-wide worker, so risk is low, but it would nail down the one cost the design deliberately does not bound.

Verified, not an issue

  • getTypeByBucketId returns null (not throws) for an unknown bucket, so the no-type / bogus-RID path is safe.
  • The IllegalArgumentException from check() propagates cleanly through executeSimple try/finally, so the live-progress registration is still unregistered.
  • (Result) null in toRecordId((Result) null, context) correctly disambiguates the Result/Identifiable overloads.
  • RECORD was already a lexer keyword, so no new identifier is reserved.

Nice work - the comments carry the why at every non-obvious decision, which made this reviewable.

Automated review by Claude. I could not run mvn in this environment, so the compile/test claims in the PR description were taken as reported; the above is a static read of the diff and surrounding code.

…record fix cost with a test

Review approved with three non-blocking notes; two taken.

- COMPRESS is unaffected by the RECORD scope and still compresses the whole database. It is opt-in,
  but it is the one clause that breaks the "naming a record bounds the cost" promise, so it now sits
  in the release notes beside the other two costs the scope does not bound.
- Added the test the review asked for: a listed record that is GENUINELY corrupted - not merely
  missing, not merely chain-broken - is flagged, and under FIX every index on its bucket is dropped
  and rebuilt. That turns the "RECORD bounds the check, not necessarily the fix" paragraph into an
  executable assertion, so a future change cannot quietly make a scoped repair either cheaper or
  more destructive than documented. Mutation-checked: dropping addCorrupted from the scoped arm
  fails it, and the injected corruption is the intended shape (buffer truncated to 8 bytes against
  the 25-byte vertex prefix), distinct from the missing-RID path.

Not taken: moving the RECORD-plus-TYPE/BUCKET rejection from check() into the setters. setTypes and
setRecords can be called in either order, so a setter-level guard would fire or not depending on
call order; check() is the only place that sees the whole configuration. The review called the
current placement acceptable.
@lvca

lvca commented Aug 1, 2026

Copy link
Copy Markdown
Member Author

Addressed in 9db8cac03. Two of the three notes taken.

1. COMPRESS under a RECORD scope — documented. You are right that it is the one clause breaking the "naming a record bounds the cost" promise. It now sits in the release notes beside the other two unbounded costs, worded so nobody pairs RECORD ... COMPRESS expecting a cheap run.

2. Corrupted-record FIX coverage — added. This was the best of the three suggestions: it converts the "RECORD bounds the check, not necessarily the fix" paragraph from prose into an executable assertion, so the one cost the design deliberately does not bound can't drift silently in either direction.

checkDatabaseRecordFixRebuildsTheBucketIndexesOfAGenuinelyCorruptedRecord shrinks a vertex's record-size varint below the fixed 25-byte prefix, so the record still occupies its slot but no longer reads back as a vertex — genuinely corrupted, distinct from both the missing-RID and the chain-broken cases the other tests cover. It asserts the record is flagged and that rebuiltIndexes is non-empty.

Mutation-checked: dropping addCorrupted from the scoped arm fails it with totalCorruptedRecords: 0, rebuiltIndexes: [], and the warning confirms the intended corruption shape — record buffer is truncated or corrupted (size=8, expected at least 25).

3. Setter-time rejection of TYPE/BUCKET + RECORD — not taken. setTypes and setRecords can be called in either order, so a setter-level guard would fire or not depending on call order, which is worse than a consistent failure. check() is the only point that sees the complete configuration. You called the current placement acceptable and I agree; flagging the reasoning rather than leaving it implicit.

Verification

com.arcadedb.graph.**, com.arcadedb.engine.**, com.arcadedb.query.sql.parser.** plus CheckDatabaseExtendedTest: 1354 tests, 0 failures, BUILD SUCCESS.

@claude

claude Bot commented Aug 1, 2026

Copy link
Copy Markdown

Review: CHECK DATABASE RECORD (#5680 follow-up)

Reviewed the full diff plus the surrounding DatabaseChecker / GraphDatabaseChecker flow. This is a genuinely high-quality change: the scoped enumeration is layered cleanly onto the existing per-record workers, the step/progress accounting is handled carefully, and the test suite is thorough (missing-vs-corrupt distinction, cross-type grouping, warning caps, the "is it really a scope?" test, and the anti-vacuous rewrite of assertIntegrityClean). The documentation of the costs the scope does not bound is exemplary. A few observations, all minor.

Correctness / design

  • Exception-not-Throwable guard is only half-effective (matches type-wide, so not a regression). In GraphDatabaseChecker.checkVertices/checkEdges, the outer scoped loop deliberately catches Exception (not Throwable) so an Error thrown by lookupByRID will not get a healthy record flagged corrupted and deleted under FIX. Good intent - but the shared checkConnectivity/checkEndpoints consumer still wraps its body in catch (final Throwable e) and flags corrupted. So an Error raised during the connectivity/endpoint check (inside checkOutgoingEdges etc.), rather than during the lookup, is still recorded as corruption. This mirrors the type-wide path exactly, so it is not a new hazard, but the elaborate "an Error is a fact about the JVM, not about this record" comment slightly oversells the protection it actually provides.

  • Partial null RIDs are silently dropped; only all-null is refused. setRecords refuses a scope that resolves to entirely nulls (great - prevents silent widening to a full-DB scan), but a mix like RECORD one-valid-rid plus one null-expression keeps the valid one and silently drops the null. That is the documented behavior and probably the right call, but a caller who fat-fingered one of several RIDs gets no signal that one was discarded. Worth confirming this is intended vs. worth surfacing a warning in the result.

  • RECORD ... COMPRESS still compresses the whole database. Documented clearly in the release notes as the one clause that breaks the "naming a record bounds the cost" promise. Since the PR already rejects RECORD + TYPE/BUCKET at execution with a tailored message, it is slightly inconsistent to reject those combinations but silently honor a whole-DB compress under a record scope. Not blocking - the docs are explicit - just flagging the asymmetry.

Minor

  • The null-type group in checkScopedRecords bumps currentStep but never emits a setProgress/progress callback for that step, unlike the other arms. Purely cosmetic (a poller sees no name for that step), not a correctness issue.
  • RECORD combined with TYPE/BUCKET is rejected at execution rather than in the grammar. The PR explains the trade-off (grammar-level rejection would lose the tailored message), and it is covered by checkDatabaseRecordRejectsBeingCombinedWithTypeOrBucket. Reasonable.

Style / conventions

Consistent with the codebase: final on locals/params, imported types rather than FQNs, JSON handling untouched, tests use the assertThat(...) style from CLAUDE.md. One convention check: checkDatabaseRecordFixMakesTheVertexDeletableAgain and the multi-type test create 200 edges one-transaction-each. If these run noticeably long on CI, consider @tag("slow") per the project convention. Otherwise nothing to change.

Nice work - the "costs this does NOT bound" honesty and the vacuous-test fix are the standout parts.

Note: I reviewed the diff and surrounding code but did not re-run the build/test suite locally; the PR own verification run is the source for the 1335-tests-passing claim.

…ed, and a comment stops overselling

Review round: approve, all notes minor. Three taken.

- The "Exception, not Throwable" comment claimed more protection than the code gives. It guards the
  LOOKUP, but the shared per-record consumer still catches Throwable around the connectivity check
  itself - matching the type-wide path, which is why it stays. The comment now says exactly what is
  and is not covered rather than implying an Error can never be recorded as corruption.
- setRecords refused an all-null scope but dropped a PARTIAL one silently, so a caller who mistyped
  one of several RIDs got a clean report for a check that skipped it. Narrowing is still allowed -
  unlike widening it cannot do more than asked - but it now surfaces a warning naming how many were
  dropped, with a test.
- The no-type group advanced currentStep without ever naming the step, so a progress poller watched
  it go by blank. Routed through stepBegin like the document arm, which also owns the bump.

Not taken:

- Rejecting RECORD together with COMPRESS for symmetry with the TYPE/BUCKET rejection. The two are
  not the same case: RECORD plus TYPE/BUCKET has no coherent meaning, whereas "check this record,
  then compress the database" is perfectly meaningful - it is just not cheap. That is a
  documentation problem, and the release note now states it.
- @tag("slow") on the 200-edge tests. Measured rather than assumed: the whole class runs in 0.53s,
  and CLAUDE.md's criteria are multi-second elapsed or large batches. 200 tiny transactions under a
  second is neither.
@lvca

lvca commented Aug 1, 2026

Copy link
Copy Markdown
Member Author

Addressed in 95358fb74. Three taken, two declined with reasoning.

1. The Exception-not-Throwable comment oversold the protection — fair, and fixed. You are right: the guard covers the lookup, but the shared per-record consumer still catches Throwable around the connectivity check itself, so an Error raised in there is still flagged. That matches the type-wide path, which is why it stays — but my comment implied a guarantee the code does not make. It now states exactly what is and is not covered. Good catch on a claim rather than on code.

2. Partial null RIDs dropped silently — now surfaced. You asked whether this was intended. It is: narrowing cannot do more than the caller asked for, unlike widening, so refusing would be too harsh. But silent was wrong — a caller who mistyped one of several RIDs got a clean report for a check that skipped it. check() now emits a warning naming how many were dropped, with a test (checkDatabaseRecordReportsRidsItHadToDrop).

3. No-type group advanced the step counter without naming the step — fixed. Routed through stepBegin like the document arm, which also owns the bump, so a progress poller no longer sees a blank step go by.

4. RECORD ... COMPRESS asymmetry — declined, and I think the asymmetry is correct. The two cases differ in kind: RECORD + TYPE/BUCKET has no coherent meaning (three competing answers to "what is the scope?"), which is why it is refused. "Check this record, then compress the database" is perfectly meaningful — it is just not cheap. Refusing a legitimate combination because it is expensive would be a worse trade than documenting the cost, which the release note now does explicitly.

5. @Tag("slow") on the 200-edge tests — declined, measured rather than assumed. The whole class runs in 0.53s. CLAUDE.md's criteria are multi-second elapsed time or large batches; 200 tiny transactions in half a second is neither, and tagging would move them out of the default lane for no benefit.

Verification

com.arcadedb.graph.**, com.arcadedb.engine.**, com.arcadedb.query.sql.parser.** plus CheckDatabaseExtendedTest: 1355 tests, 0 failures, BUILD SUCCESS.

@claude

claude Bot commented Aug 1, 2026

Copy link
Copy Markdown

Review: CHECK DATABASE RECORD (PR 5710)

Reviewed the full diff against main. This is a well-scoped, carefully reasoned change: the new RECORD scope reuses the existing per-record workers (GraphDatabaseChecker.checkVertices/checkEdges gain a scopedRecords overload that only swaps the enumeration), the shared index-rebuild/compress tail is untouched, and the design trade-offs are documented both in code and in the release notes. Test coverage is genuinely thorough - end-to-end repair, scope isolation, missing-vs-corrupted distinction, multi-type grouping, warning caps, and mutation-checked assertions. Nice work.

A few observations, mostly minor:

Code quality / correctness

  • setRecords throws before the TYPE/BUCKET combination guard runs. executeSimple calls setTypes then setRecords, and the RECORD + TYPE/BUCKET rejection lives in check(). So "CHECK DATABASE TYPE Hub RECORD {@Rid: null}" surfaces "none of them resolves to a RID" rather than the more helpful "cannot be combined with TYPE or BUCKET". Narrow edge case (a null-resolving RID combined with a type clause), but the error points at the wrong problem. Not blocking.

  • droppedRecords is not reset on the early-return path of setRecords. If setRecords runs once with a non-empty set (setting droppedRecords greater than 0) and is then called again with null/empty, the early return leaves the stale value. Harmless today because the dropped-records warning is only emitted inside the recordScope block (false in that case), but resetting this.droppedRecords to 0 on that branch keeps the invariant regardless of call order, since setRecords is public API.

Consistency note (pre-existing, not introduced here)

  • The new checkScopedDocuments increments both totalWarnings and totalCorruptedRecords, but the type-wide checkDocuments (around line 274) increments neither - it only adds to the warnings/corruptedRecords sets. So document corruption reported by a scoped run vs a type-wide run will differ in those totals. The scoped path is arguably the more correct one; worth deciding whether to align the type-wide path in a follow-up.

Minor

  • RECORD ... COMPRESS still compresses the whole database. Documented and opt-in, but since it is the one clause that breaks the "naming a record bounds the cost" promise, a runtime warning when RECORD + COMPRESS are combined could save an operator an expensive surprise. Optional.

  • The duplicated RecordNotFoundException/Exception catch blocks across the scoped vertex and edge arms are an acknowledged deliberate non-extraction; agree the duplication reads better than threading six accumulators through a helper here.

Verification

Reviewed the logic (step-counter accounting, warning-cap propagation via maxWarnings minus currentWarnings, updateStats double-count avoidance, getTypeByBucketId returning null for unknown buckets) but did not run the build locally. The PR reports 1335 tests passing across the affected packages, which matches what the change touches. Recommend confirming CI is green before merge.

Overall: solid, defensible, and unusually well-documented. The points above are refinements, not blockers.

(Automated review by Claude.)

…RESS is unscoped

Review round: refinements, not blockers. Four taken.

- CHECK DATABASE TYPE Hub RECORD {"@Rid": null} reported "none of them resolves to a RID", because
  the RIDs are resolved before check() reaches the combination guard - pointing the operator at the
  inner failure while the actual mistake is the clause combination. The statement now diagnoses the
  combination before resolving anything; check() keeps its own guard for programmatic callers.
- setRecords left droppedRecords stale on its early-return path. Harmless today, since the warning
  is only emitted under a record scope, but the field is public-API-visible state and should not
  depend on call order.
- COMPRESS now warns that it is not limited by the RECORD scope. The review suggested this as a
  middle ground between refusing the combination and staying silent, and it is a better answer than
  the documentation-only one: the command still does what it was asked, but an operator who named a
  record and expected a bounded run is told otherwise. Tested both ways, so the warning tracks the
  clause rather than the scope.
- Recorded, not fixed: the type-wide checkDocuments adds to the warnings/corrupted sets without
  incrementing the corresponding totals, so a corrupt document reports different totals depending on
  which path found it. The scoped arm here is the correct one; aligning the type-wide path is a
  behaviour change to a command this issue does not otherwise touch.

Agreed with the review that the duplicated catch blocks read better than threading six accumulators
through a shared helper; no change.
@lvca

lvca commented Aug 1, 2026

Copy link
Copy Markdown
Member Author

Addressed in 81c2a9b99. All four taken.

1. Wrong error for TYPE ... RECORD {@rid: null} — fixed. You are right that the resolution failure was masking the real mistake. The statement now diagnoses the clause combination before resolving any RID, so the operator is told about the outer problem; check() keeps its own guard for programmatic callers. Tested.

2. droppedRecords stale on the early-return path — fixed. Harmless today for exactly the reason you gave, but it is public-API-visible state and should not depend on call order. Reset on that branch.

3. RECORD ... COMPRESS runtime warning — taken, and it is a better answer than mine. I had argued against refusing the combination (it is meaningful, just not cheap) and settled for a release-note line. Your middle ground is strictly better: the command still does what was asked, and an operator who named a record and expected a bounded run is told plainly that it will not be. Tested both with and without COMPRESS, so the warning tracks the clause rather than the scope.

4. Scoped vs type-wide document totals — recorded, not silently diverged. Agreed the scoped arm is the more correct one. I have not changed the type-wide path: that would alter what CHECK DATABASE TYPE ... reports for every user, and this issue does not otherwise touch that command. The divergence is now stated in the Javadoc so it is a known, deliberate difference rather than something to be rediscovered — happy to file it as a follow-up if you would like it aligned.

5. Duplicated catch blocks — agreed, no change.

Verification

com.arcadedb.graph.**, com.arcadedb.engine.**, com.arcadedb.query.sql.parser.** plus CheckDatabaseExtendedTest: 1356 tests, 0 failures, BUILD SUCCESS.

@codacy-production

codacy-production Bot commented Aug 2, 2026 •

Copy link
Copy Markdown

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

🟢 Metrics 52 complexity

Metric Results
Complexity 52

View in Codacy

🟢 Coverage 84.29% diff coverage · -6.81% coverage variation

Metric Results
Coverage variation ✅ -6.81% coverage variation
Diff coverage ✅ 84.29% diff coverage

View coverage diff in Codacy

Coverage variation details
Coverable lines Covered lines Coverage
Common ancestor commit (4140df2) 151001 115507 76.49%
Head commit (fcb2958) 183178 (+32177) 127644 (+12137) 69.68% (-6.81%)

Coverage variation is the difference between the coverage for the head and common ancestor commits of the pull request branch: <coverage of head commit> - <coverage of common ancestor commit>

Diff coverage details
Coverable lines Covered lines Diff coverage
Pull request (#5710) 191 161 84.29%

Diff coverage is the percentage of lines that are covered by tests out of the coverable lines that the pull request added or modified: <covered lines added or modified>/<coverable lines added or modified> * 100%

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

@claude

claude Bot commented Aug 2, 2026

Copy link
Copy Markdown

Review: CHECK DATABASE RECORD <rid> (#5710)

Reviewed against CLAUDE.md/engine/CLAUDE.md. Overall this is high-quality work: the new scope is a genuinely useful escape hatch for the strict deleteVertex from #5707, the shared-consumer refactor of checkVertices/checkEdges avoids duplicating the connectivity logic, and the tests are unusually thorough (edge/vertex/document arms, cross-type grouping, missing-vs-corrupted distinction, the combination refusal, warning caps, and the genuinely-corrupted index-rebuild path). Fixing the vacuously-passing assertIntegrityClean (asserting on totalErrors, a field check database never emits) is a nice catch and the right kind of collateral cleanup.

A few points, all minor - nothing blocking.

1. Duplicated error-message string (maintainability)

The "cannot be combined with TYPE or BUCKET" message is written out verbatim in both CheckDatabaseStatement.executeSimple() and DatabaseChecker.check(). The two-layer guard itself is well-justified (the comments explain diagnosing the clause conflict before RID resolution), but the literal string is duplicated. Extracting it to a shared constant would keep the two messages from drifting.

2. Exception count collapses when every RID resolves to null (cosmetic)

In CheckDatabaseStatement, records.stream().map(toRecordId).collect(toCollection(LinkedHashSet::new)) collects into a LinkedHashSet<RID>. If several distinct RID expressions all resolve to null, they dedupe to a single null element, so setRecords reports "given 1 record(s) but none of them resolves to a RID" even when the user named, say, three. Purely a message-accuracy nit, but the "1" can mislead when debugging a multi-RID statement.

3. checkScopedDocuments grows corruptedRecords/totalCorruptedRecords uncapped

The graph arms go through addCorrupted(...), which honours maxCorrupted. The document arm (checkScopedDocuments) instead does corruptedRecords.add(rid) and increments totalCorruptedRecords directly, with no cap - only the retained warning is bounded via addScopedWarning. A record scope naming a large batch of unloadable documents would therefore let the corrupted set/total grow without bound, unlike every other path in this file. Low-risk in practice (documents rarely, and a caller hand-types the scope), but it's an inconsistency worth a comment or an equivalent cap.

4. Per-type-group edge rescan under multi-type scope (already partly documented)

reconnectEdges runs a full scan of every edge type once per checkVertices call, and checkScopedRecords calls checkVertices once per vertex-type group. So a scope spanning two vertex types pays two full edge-type sweeps in fix mode. The single-type case is correctly bounded (one sweep for N scoped vertices of one type), and the "edge scan is not bounded" caveat is already called out - just flagging that it multiplies by the number of distinct vertex types named, which the docs could state explicitly.

Verification note

I did not run the full reactor build; the PR states com.arcadedb.graph.**, com.arcadedb.engine.** and ...sql.parser.** pass (1335 tests, 0 failures). Imports check out (RecordNotFoundException, Consumer, Result, LinkedHashSet all added), and the RECORD grammar token already existed in the lexer and keyword rule, so the new clause introduces no identifier ambiguity.

Nice work - the design tradeoffs (skipping db-wide passes and orphan reclaim, refusing empty-resolved scopes rather than silently widening to a full-database scan, not treating a missing RID as corruption to avoid a needless index rebuild) are all sound and well-argued.

…t cannot know, bounded corrupted set

Review round: nothing blocking. All four points taken.

- The "cannot be combined with TYPE or BUCKET" text was written out in both the statement and the
  checker. The two-layer guard stays - the statement has to diagnose the clause conflict before the
  RIDs are resolved - but the wording now lives in one constant so the two cannot drift.
- Two messages quoted a record count the Set could not know: several RID expressions that all
  resolve to null collapse to a single null element, so "given 3 record(s)" was never reachable and
  "1 of the record(s) did not resolve" under-reported. Rather than plumb the original count through,
  both messages stop quoting a number, which is accurate whatever the input.
- checkScopedDocuments grew corruptedRecords without a bound while every other path honours
  maxCorrupted/maxWarnings. Bounded the same way, with the total still counting every occurrence.
- Documented that the edge sweep a rebuild needs runs once per distinct vertex TYPE named, not once
  per run: ten vertices of one type cost one sweep, one vertex of each of three types cost three.
@lvca

lvca commented Aug 2, 2026

Copy link
Copy Markdown
Member Author

Addressed in 0cc58f073. All four taken.

1. Duplicated error message — one owner. DatabaseChecker.RECORD_SCOPE_CONFLICT_ERROR, referenced by the statement. The two-layer guard stays (the statement must diagnose the clause conflict before RIDs are resolved, which is what the previous round fixed), but the wording can no longer drift.

2. Count collapses when several RIDs resolve to null — fixed by removing the count, not by plumbing one through. You are right that a Set dedupes every null into one element. The consequence is stronger than a nit: "given 3 record(s)" was never reachable, and the dropped-records warning always said "1" no matter how many were dropped. So any number those messages quoted was wrong by construction. Both now say "none of the records given" / "one or more of the records given", which is accurate whatever the input, and neither needs the original count threaded through the API.

3. checkScopedDocuments grew corruptedRecords uncapped — fixed. A real inconsistency: every other path in the file bounds the retained set while letting the total count on. The document arm now does the same.

4. Edge sweep multiplies by distinct vertex types — documented. Stated in both the records field Javadoc and the release note, with the concrete contrast so the cost is predictable: ten vertices of one type is one sweep; one vertex of each of three types is three.

Verification

com.arcadedb.graph.**, com.arcadedb.engine.**, com.arcadedb.query.sql.parser.** plus CheckDatabaseExtendedTest: 1356 tests, 0 failures, BUILD SUCCESS.

@claude

claude Bot commented Aug 2, 2026

Copy link
Copy Markdown

Review: CHECK DATABASE RECORD scope (#5710)

Reviewed the full diff against main. This is a high-quality PR - the feature is coherent, the design trade-offs are stated rather than hidden, and the test coverage is unusually complete.

What is well done

  • Semantics match the type-wide path. The scoped checkVertices/checkEdges reuse the identical per-record consumer (checkConnectivity / checkEndpoints) via the new Consumer<Record> extraction, so a scoped run and a type-wide run do the same work on the same record. Right way to avoid drift.
  • Missing-RID vs corrupted-RID distinction has teeth. Catching RecordNotFoundException separately (warn, never flag corrupted) matters: a corrupted record puts its bucket into affectedBuckets, and FIX then drops/rebuilds every index on that bucket - a full bucket scan, exactly the cost the scope exists to avoid. checkDatabaseRecordDoesNotTreatAMissingRidAsCorruption pins this, including the empty rebuiltIndexes assertion.
  • The empty-resolved-scope guard is in the right place. Refusing a non-empty-but-all-null scope in setRecords (the public API entry point, the last place that can distinguish "named records, none usable" from "named no records") prevents an explicitly-scoped call from silently widening to a full-database scan.
  • The vacuous-test fix is a genuine improvement. Issue5670EdgeDeleteDanglingBackRefTest.assertIntegrityClean was asserting totalErrors == 0 against a field check database never emits, so the null-tolerant reader turned it into 0 == 0 on every run. Switching to autoFix/invalidLinks/totalWarnings/totalCorruptedRecords plus a longProperty that fails loudly on a missing field is the correct fix.
  • Progress step accounting is careful. The currentStep-ownership-per-arm comment explains why a naive up-front bump double-counted document groups. stepTick() is invoked exactly once per record on every branch, matching progressBegin(scopedRecords.size()).

Points worth a second look (none blocking)

  1. Scoped vertex check collapses the type-wide "two passes" into one. The type-wide path runs a bucket scan (asVertex(true) on a freshly materialized immutable record) and a connectivity walk; the scoped path runs only checkConnectivity, materializing via lookupByRID rather than newImmutableRecord from the raw page view. Functionally equivalent for corruption detection (confirmed by checkDatabaseRecordFixRebuildsTheBucketIndexesOfAGenuinelyCorruptedRecord), but if a corruption shape ever only surfaces through raw-view materialization the scoped path would miss it. Likely fine given the scope purpose, worth being aware of.

  2. shrinkRecordBuffer is tightly coupled to on-disk page layout (PAGE_RECORD_TABLE_OFFSET, the record-size varint, zigzag(8)==16). Well-commented, but will break behaviorally (not at compile time) if the bucket page format changes. Acceptable for a regression test guarding a documented cost, just flagging maintenance cost.

  3. Double guard for the TYPE/BUCKET conflict in both CheckDatabaseStatement.executeSimple (before RID resolution) and DatabaseChecker.check(). Intentional defense-in-depth; the shared RECORD_SCOPE_CONFLICT_ERROR constant keeps the two messages in sync.

  4. deletedRecordsAfterFix not merged in the scoped graph arm - but the type-wide checkVertices/checkEdges do not merge it either (only checkBuckets/checkExternalProperties do, correctly skipped under scope). Consistent, not a gap.

Test coverage

Excellent - happy-path repair end-to-end, proof that RECORD is genuinely a scope, TYPE/BUCKET rejection, edge/document/no-type/cross-type arms, missing-vs-corrupt, warning cap with totals still counting, and the documented COMPRESS-not-scoped and index-rebuild-on-corruption costs. Parser test covers valid forms plus grammar-accepts-but-execution-rejects.

Style / conventions

Consistent: final on locals/params, imported names, assertThat(...).isTrue() style, Apache header on the new test, no System.out, no new dependencies.

Note: I could not run mvn in this sandboxed review environment, so I relied on the PR stated verification (1335 tests, 0 failures) plus a static read. Recommend a maintainer confirm the reactor build once more before merge.

Overall: LGTM. The items above are observations, not change requests.

Automated review by Claude.

@codecov

codecov Bot commented Aug 2, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 81.67539% with 35 lines in your changes missing coverage. Please review.
✅ Project coverage is 67.74%. Comparing base (4140df2) to head (fcb2958).
⚠️ Report is 13 commits behind head on main.

Files with missing lines Patch % Lines
.../java/com/arcadedb/graph/GraphDatabaseChecker.java 75.00% 19 Missing ⚠️
...main/java/com/arcadedb/engine/DatabaseChecker.java 85.58% 11 Missing and 5 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #5710      +/-   ##
==========================================
+ Coverage   67.68%   67.74%   +0.06%     
==========================================
  Files        1780     1780              
  Lines      151001   151132     +131     
  Branches    32029    32054      +25     
==========================================
+ Hits       102199   102379     +180     
+ Misses      35533    35469      -64     
- Partials    13269    13284      +15     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@lvca lvca self-assigned this Aug 2, 2026
@lvca lvca added this to the 26.8.1 milestone Aug 2, 2026
@lvca
lvca merged commit 56b47f4 into main Aug 2, 2026
59 checks passed
@lvca
lvca deleted the fix/5680-vertex-delete-ghost-edges branch August 2, 2026 16:10
@claude

claude Bot commented Aug 2, 2026

Copy link
Copy Markdown

Review: CHECK DATABASE RECORD <rid> (#5710)

Reviewed the full diff statically (couldn't run Maven in my environment, so this leans on the author's reported 1335 tests, 0 failures for the affected packages). Overall this is a careful, well-scoped, thoroughly tested change. Notes below, mostly affirmations plus a few minor points.

Strengths

  • The missing-RID vs. corruption distinction is the right call and is load-bearing. Treating a stale/typo'd RID as RecordNotFoundException -> a plain warning, rather than flagging it corrupted, avoids putting the bucket into affectedBuckets and triggering a full index drop/rebuild under FIX - exactly the cost the scope exists to avoid. checkDatabaseRecordDoesNotTreatAMissingRidAsCorruption pins it, including the rebuiltIndexes/autoFix giveaway. Good defensive design.
  • catch (Exception) rather than catch (Throwable) on the lookup correctly keeps a JVM Error from being recorded as record corruption (and from having FIX delete a healthy record), while the shared consumer still catches Throwable around the connectivity check to match the type-wide path. The asymmetry is deliberate and documented.
  • setRecords refusing an all-null resolution instead of silently widening to a full-DB scan is the correct place for the guard - it's the last point that can distinguish "named records, none usable" from "named no records". Nice that it's enforced at the public API boundary, not just the SQL layer, and tested directly.
  • Progress accounting is consistent: exactly one progressTick() per scoped RID (consumer ticks at entry; each catch arm ticks once), progressBegin sized to scopedRecords.size(), and the per-arm currentStep bump comment explains the double-count trap it avoids. totalSteps = scoped.size() + fix + compress lines up with what actually executes.
  • updateStats sums only Long values, so the separate warnings/corruptedRecords set merges in checkScopedRecords don't double-count totalWarnings/totalCorruptedRecords. Verified against checkDatabaseRecordReportsRidsItHadToDrop (expects totalWarnings == 1).
  • Test coverage is excellent - end-to-end repair, scope isolation, missing RID, edge RID, document RID, multi-type spanning, no-type RID, empty-resolution refusal, partial-drop reporting, COMPRESS warning, warning cap, and genuine-corruption index rebuild. The shrinkRecordBuffer helper to synthesize on-disk corruption (distinct from a missing record) is a particularly good touch.
  • Fixing the vacuously-passing assertIntegrityClean (asserting on a non-existent totalErrors field -> 0 == 0) is a real correctness win beyond the feature itself.

Minor points

  1. Doc formatting nit (docs/release-26.8.1.md): the last bullet of the new section is immediately followed by ## Copying a type... with no blank line. CommonMark/GFM will still render the heading (ATX headings can interrupt a paragraph), but every other section in the file has a blank line before its ##, so this is a stylistic inconsistency worth a one-line fix.

  2. RECORD + TYPE/BUCKET -> HTTP 400 is asserted only indirectly. The tests confirm an IllegalArgumentException with the expected message, and the PR body notes the HTTP layer maps that to 400, but there's no test exercising that mapping here. Probably fine to rely on existing HTTP-layer coverage, just flagging that the 400 claim isn't pinned in this PR.

  3. Type inference from bucket skips the explicit "record is of the right type" pass that the type-wide path runs (newImmutableRecord(type, ...).asVertex()). This is correct - the bucket deterministically owns the type, and asVertex(true)/asEdge(true) still validate the shape - so no action needed; noting it so a future reader doesn't mistake it for a gap.

  4. Comment density in DatabaseChecker/GraphDatabaseChecker is very high relative to the surrounding file. The rationale is genuinely useful (the missing-RID/corruption trade-off, the step-count trap), but a few of the multi-line asides could be tightened. Purely subjective; not blocking.

Nothing here rises to a correctness blocker from my read. Nice work - the scope boundaries (what it does and does NOT bound) are unusually well reasoned and the tests actually prove the "it's really a scope" property rather than just exercising the happy path.

lvca added a commit that referenced this pull request Aug 2, 2026
…airs it, and the conflict keeps its cause (#5767)

* fix(engine) #5764: a refused vertex delete names the command that repairs it, and the conflict keeps its cause

Follow-ups from #5680, whose two halves landed separately: #5707 made deleteVertex strict, #5710 added the
CHECK DATABASE RECORD scope. Neither is defective; the seams between them were never closed.

1. The repair advice now names the scoped command with the RID substituted. Split in two, which the report did
   not distinguish: the RECORD scope is the right answer only where the delete was REFUSED and the vertex is
   still there to repair. The force arms log AFTER the record is gone, so a scoped check would name a record
   that no longer exists - they keep the whole-database form and say what the scoped form buys when run first.
   And at the other-end conflict the broken list belongs to the NEIGHBOUR, so that is the RID named.
   The previously bare rethrow now carries the advice instead of only getEdgeHeadChunkForWrite's
   "concurrent commit in flight", which is the diagnosis for the transient case nobody ever sees.

2. ConcurrentModificationException gains the (message, cause) constructor NeedRetryException already declared.
   Eight sites across GraphEngine, EdgeLinkedList and StripedEdgeList kept the message and dropped the stack.
   A conflict is normally absorbed by the retry, so the run that surfaces one is the retry-exhausted run -
   exactly the one whose trace has to be diagnosable. The class on the wire is unchanged.

3. checkDocuments and checkScopedDocuments report a corrupt document identically. The type-wide arm added to
   the warning and corrupted-record SETS without touching totalWarnings/totalCorruptedRecords, called a
   document a "vertex" it was "removing" when nothing on that path removes anything, and honoured no
   maxWarnings cap at all. Both now go through shared addWarning/addCorrupted.

4. The scoped vertex check running one pass where the type-wide runs two is NOT a gap: lookupByRID IS the
   raw-view materialisation (same newImmutableRecord(type, rid, bucket.getRecord(rid).copyOfContent())) and
   checkConnectivity opens with the same asVertex(true) that forces the decode, while getRecord and scan
   resolve placeholders and multi-page chains identically. Documented and pinned by a test rather than
   "fixed" by adding a second pass, which would double the cost of the scope that exists to be cheap.

The advice tests execute what the message says: the command is parsed out of the raised exception and run
verbatim, then the delete must succeed - so a wrong RID or a shape the parser rejects fails here.

* review: state addCorrupted's dedup guarantee honestly, and drop the reflection in the advice test

Three points from the code review on #5767, all minor and none blocking.

1. addCorrupted's "a record flagged twice is one corrupted record" holds only while the retained set is under
   the cap. Behaviour kept (the contains() check past the cap is strictly better than an unconditional
   increment); the Javadoc now says exactly what survives the cap and why closing the gap is not worth it - the
   only exact fix is a second unbounded set of every RID ever counted, i.e. the memory the cap exists to refuse.
   Also recorded that neither caller reaches it: the bucket scan visits each RID once and the RECORD scope is a
   Set, so a duplicate cannot survive as far as groupRecordsByType.

2. The reviewer flagged that the retained set is capped with maxWarnings while the graph arms take a separate
   maxCorrupted. Documented rather than changed: DatabaseChecker has no maxCorrupted field, and checkScopedRecords
   already passes the remaining maxWarnings budget into the GraphDatabaseChecker parameter of that name. Two
   names, one setting.

3. The advice test reached the LogManager logger by reflecting on the private field. LogManager exposes a public
   getLogger() for precisely this case - a caller that swaps the logger and has to put the original back - so the
   reflection helper is gone along with its rename fragility.

* review: record the maxWarnings/maxCorrupted coupling at the field, and why the interpolated cause is not redundant

Three non-blocking observations from the second review on #5767. All three resolve to recording constraints a
future reader would otherwise have to rediscover - one of them by breaking a test.

1. maxWarnings caps the corrupted set while GraphDatabaseChecker takes a maxCorrupted for the SAME set. The
   method Javadoc said "two names, one setting"; the note now lives on the field, where a reader meets it, and
   adds the consequence the reviewer pointed out: retained corruptedRecords become affectedBuckets, so a record
   dropped past the cap does not get its bucket's indexes rebuilt under FIX.

2. The advice test swaps the process-wide LogManager logger. Restored in a finally and safe under sequential
   class execution, but it is shared state; documented as a latent constraint if class-level parallelism is ever
   enabled, along with why there is no per-invocation seam to capture through instead.

3. getEdgeHeadChunkForWrite and hasNextEdgeToDelete interpolate e.getMessage() AND keep e as the cause, which
   reads as redundant. It is not: the missing CHUNK's RID appears ONLY in the record-not-found message - the
   surrounding text names the vertex - and the top-level message is what a log line and an HTTP error body carry,
   while the cause chain surfaces only in a full trace. Removing it would drop the RID and fail
   Issue5670EdgeDeleteDanglingBackRefTest, which asserts on it. Commented at both sites.

* review: guard the advice against a message-less conflict, and record why otherEndOf can equal the vertex

Two of the three optional-polish points from the third review on #5767 (which was LGTM).

1. withRepairAdvice built the message as e.getMessage() + advice. Every conflict on this path carries a message
   today, but a message-less one would render "null. If it persists...", which reads as a bug in the advice rather
   than as a missing diagnosis. Falls back to toString(), i.e. the class name.

2. otherEndOf would answer vertexRID itself for a self-loop, which the method name reads as impossible. That is
   the correct answer - a self-loop has no other end and the list that failed to disconnect it IS the vertex's own
   - so the equality is documented rather than "simplified" away. Also recorded why there is no test: the caller
   runs only after both of the vertex's lists have been collected, and a self-loop is read out of those same two
   lists, so a list broken enough to fail the disconnection fails the collection first.

Not taken: the reviewer flagged that otherEndOf is computed before the force branch, so it runs on every retry of
a transient conflict. Both branches need the value, so moving it saves nothing - and the reviewer scoped it as
"for completeness" on an exception path where asEdge() is light.
robfrank pushed a commit that referenced this pull request Aug 14, 2026
…med records only (#5710)

* feat(engine) #5680: CHECK DATABASE RECORD <rid> checks and repairs named records only

CHECK DATABASE could be narrowed to a TYPE or a BUCKET, but not to a record. That is the sharp edge
of the strict vertex delete merged in #5707: a genuinely broken edge chain makes its vertex
undeletable until CHECK DATABASE ... FIX rebuilds the adjacency from the surviving edge records, and
the smallest way to ask for that repair was two full passes over the whole vertex type.

  CHECK DATABASE RECORD #12:3 FIX
  CHECK DATABASE RECORD #12:3, #12:9 FIX

The per-record work and the fix are identical to the type-wide run, including the edge-list rebuild;
only the enumeration changes. Combines with FIX and COMPRESS. Rejected when combined with TYPE or
BUCKET - RECORD is already the narrowest scope, so letting one silently win would run a check nobody
asked for and an intersection would be a third semantics nobody asked for. The database-wide passes
(buckets, external properties, indexes) and the orphaned-segment reclaim are skipped, since none can
be narrowed to a record.

Two costs it deliberately does not bound, documented rather than glossed: rebuilding an adjacency
needs every surviving edge that points at the vertex and no index maps endpoints back to edges, so
the scoped run saves the vertex passes but still scans the edge types; and a listed record that
turns out to be corrupted still has every index on its bucket rebuilt.

Also fixes a vacuously-passing assertion in Issue5670EdgeDeleteDanglingBackRefTest: it asserted
totalErrors == 0, and check database reports no such field, so the null-tolerant reader turned it
into 0 == 0 on every run whatever the check found. It now asserts on autoFix, invalidLinks,
totalWarnings and totalCorruptedRecords, and fails loudly on a field that does not exist.

* style(engine) #5680: strip trailing whitespace flagged by pre-commit

The setup job failed on a trailing-whitespace line an earlier edit left after a comment in the
RECORD-scoped edge branch. Removed, along with the stray blank line beside it.

* fix(engine) #5680: a missing RID is not corruption, and the scoped progress step stops double-counting

Two independent review passes on the RECORD scope, four actionable points between them.

- A listed RID that simply is not there was recorded through addCorrupted, which puts its BUCKET
  into affectedBuckets - so CHECK DATABASE RECORD ... FIX dropped and rebuilt every index on that
  bucket, a full bucket scan, over what is usually a mistyped or already-deleted RID. That is
  exactly the cost the scope exists to avoid, and the scope is meant to be hand-typed after a failed
  delete. RecordNotFoundException is now reported and never flagged corrupted, in all three arms
  (vertex, edge, document). Mutation-checked: restoring addCorrupted fails the new test with
  totalCorruptedRecords=1, autoFix=1.
- checkScopedRecords bumped currentStep for every group, but the document arm reaches stepBegin,
  which bumps it again - so a scope containing a plain document reported "step 3 of 2" against a
  totalSteps budgeted at one step per group. Each arm now owns its bump; the graph arms need it
  because they pass a fixed value into the nested GraphDatabaseChecker instead.
- setRecords and the statement's stream now drop null RIDs. Rid.toRecordId can answer null for a
  non-literal form, and groupRecordsByType would meet it as an NPE on getBucketId(). Not reachable
  through the RECORD #n:n literal grammar, but setRecords(Set<RID>) is public API.
- The parser test's cross-reference named a test class that no longer exists after the rescope.

Tests: four added for the branches that were unexercised - an edge-typed RID, a document-typed RID
(the branch the progress bug lived in), several RIDs spanning different types in one command, and a
missing-but-valid RID under FIX, which pins the first item above.

* fix(engine) #5680: a RECORD scope that resolves to nothing is refused, not widened to the whole database

Review round on the rescoped PR: LGTM with three minor points, two taken.

- CHECK DATABASE RECORD whose RIDs all resolve to null left the checker with an empty scope, which
  reads as "no scope" - so a command that explicitly named records quietly ran a FULL database
  check. The review judged this unreachable from SQL; it is not. Only the `#n:n` literal always
  resolves, while the non-literal {"@Rid": <expression>} form goes through an expression that can
  answer null, so `CHECK DATABASE RECORD {"@Rid": null}` reached it. Refused now, with a test that
  fails when the guard is stubbed out.
- A warning dropped for exceeding maxWarnings was discarded silently in the two scoped arms, where
  GraphDatabaseChecker.addWarning logs it. Extracted addScopedWarning so both behave the same:
  always counted, retained under the cap, logged when dropped.

Not taken: extracting the duplicated RecordNotFoundException/Exception catch blocks in the scoped
vertex and edge arms. They would need six accumulators threaded through a shared helper, which reads
worse than the duplication, and the surrounding type-wide methods already parallel each other the
same way - the review noted as much in raising it.

* fix(engine) #5680: the empty-scope guard moves into setRecords, so it protects the public API too

Review round: LGTM once the setRecords widening note was considered. Went one step further than the
note it asked for.

The guard against "a RECORD scope that resolves to nothing silently becomes a full database check"
lived only in the SQL layer, so an embedded caller reaching DatabaseChecker.setRecords directly
still hit the widening. The review reasoned that setRecords cannot tell "named records, none
usable" from "named no records" - true only because the SQL layer filtered the nulls out BEFORE
calling it. It no longer does: the nulls are passed through, setRecords owns both the filtering and
the refusal, and one guard now covers both entry points. A genuinely empty (or null) set still means
"no scope" and is accepted. Both are asserted.

Also: the document arm said "cannot be loaded, removing it", but nothing removes the record there -
corruptedRecords only drives the index rebuild. The review called the wording inherited and asked
for no action; it is new code here, so it now says what is true, with a note on why it diverges from
the type-wide checkDocuments (which also calls a document a "vertex").

Adds the warning-cap test the review asked for: 40 bogus RIDs against maxWarnings=10 retains 10
messages and still counts 40.

Not added: an execution test for RECORD ... FIX COMPRESS. The compress pass is shared, unscoped code
already covered elsewhere, and routing it through the scoped enumeration exercises no new branch.

* docs(engine) #5680: state the COMPRESS caveat, and pin the corrupted-record fix cost with a test

Review approved with three non-blocking notes; two taken.

- COMPRESS is unaffected by the RECORD scope and still compresses the whole database. It is opt-in,
  but it is the one clause that breaks the "naming a record bounds the cost" promise, so it now sits
  in the release notes beside the other two costs the scope does not bound.
- Added the test the review asked for: a listed record that is GENUINELY corrupted - not merely
  missing, not merely chain-broken - is flagged, and under FIX every index on its bucket is dropped
  and rebuilt. That turns the "RECORD bounds the check, not necessarily the fix" paragraph into an
  executable assertion, so a future change cannot quietly make a scoped repair either cheaper or
  more destructive than documented. Mutation-checked: dropping addCorrupted from the scoped arm
  fails it, and the injected corruption is the intended shape (buffer truncated to 8 bytes against
  the 25-byte vertex prefix), distinct from the missing-RID path.

Not taken: moving the RECORD-plus-TYPE/BUCKET rejection from check() into the setters. setTypes and
setRecords can be called in either order, so a setter-level guard would fire or not depending on
call order; check() is the only place that sees the whole configuration. The review called the
current placement acceptable.

* fix(engine) #5680: a dropped RID is reported, the no-type step is named, and a comment stops overselling

Review round: approve, all notes minor. Three taken.

- The "Exception, not Throwable" comment claimed more protection than the code gives. It guards the
  LOOKUP, but the shared per-record consumer still catches Throwable around the connectivity check
  itself - matching the type-wide path, which is why it stays. The comment now says exactly what is
  and is not covered rather than implying an Error can never be recorded as corruption.
- setRecords refused an all-null scope but dropped a PARTIAL one silently, so a caller who mistyped
  one of several RIDs got a clean report for a check that skipped it. Narrowing is still allowed -
  unlike widening it cannot do more than asked - but it now surfaces a warning naming how many were
  dropped, with a test.
- The no-type group advanced currentStep without ever naming the step, so a progress poller watched
  it go by blank. Routed through stepBegin like the document arm, which also owns the bump.

Not taken:

- Rejecting RECORD together with COMPRESS for symmetry with the TYPE/BUCKET rejection. The two are
  not the same case: RECORD plus TYPE/BUCKET has no coherent meaning, whereas "check this record,
  then compress the database" is perfectly meaningful - it is just not cheap. That is a
  documentation problem, and the release note now states it.
- @tag("slow") on the 200-edge tests. Measured rather than assumed: the whole class runs in 0.53s,
  and CLAUDE.md's criteria are multi-second elapsed or large batches. 200 tiny transactions under a
  second is neither.

* fix(engine) #5680: diagnose the clause conflict first, warn that COMPRESS is unscoped

Review round: refinements, not blockers. Four taken.

- CHECK DATABASE TYPE Hub RECORD {"@Rid": null} reported "none of them resolves to a RID", because
  the RIDs are resolved before check() reaches the combination guard - pointing the operator at the
  inner failure while the actual mistake is the clause combination. The statement now diagnoses the
  combination before resolving anything; check() keeps its own guard for programmatic callers.
- setRecords left droppedRecords stale on its early-return path. Harmless today, since the warning
  is only emitted under a record scope, but the field is public-API-visible state and should not
  depend on call order.
- COMPRESS now warns that it is not limited by the RECORD scope. The review suggested this as a
  middle ground between refusing the combination and staying silent, and it is a better answer than
  the documentation-only one: the command still does what it was asked, but an operator who named a
  record and expected a bounded run is told otherwise. Tested both ways, so the warning tracks the
  clause rather than the scope.
- Recorded, not fixed: the type-wide checkDocuments adds to the warnings/corrupted sets without
  incrementing the corresponding totals, so a corrupt document reports different totals depending on
  which path found it. The scoped arm here is the correct one; aligning the type-wide path is a
  behaviour change to a command this issue does not otherwise touch.

Agreed with the review that the duplicated catch blocks read better than threading six accumulators
through a shared helper; no change.

* fix(engine) #5680: one owner for the conflict message, no counts a Set cannot know, bounded corrupted set

Review round: nothing blocking. All four points taken.

- The "cannot be combined with TYPE or BUCKET" text was written out in both the statement and the
  checker. The two-layer guard stays - the statement has to diagnose the clause conflict before the
  RIDs are resolved - but the wording now lives in one constant so the two cannot drift.
- Two messages quoted a record count the Set could not know: several RID expressions that all
  resolve to null collapse to a single null element, so "given 3 record(s)" was never reachable and
  "1 of the record(s) did not resolve" under-reported. Rather than plumb the original count through,
  both messages stop quoting a number, which is accurate whatever the input.
- checkScopedDocuments grew corruptedRecords without a bound while every other path honours
  maxCorrupted/maxWarnings. Bounded the same way, with the total still counting every occurrence.
- Documented that the edge sweep a rebuild needs runs once per distinct vertex TYPE named, not once
  per run: ten vertices of one type cost one sweep, one vertex of each of three types cost three.

(cherry picked from commit 56b47f4)
robfrank pushed a commit that referenced this pull request Aug 14, 2026
…airs it, and the conflict keeps its cause (#5767)

* fix(engine) #5764: a refused vertex delete names the command that repairs it, and the conflict keeps its cause

Follow-ups from #5680, whose two halves landed separately: #5707 made deleteVertex strict, #5710 added the
CHECK DATABASE RECORD scope. Neither is defective; the seams between them were never closed.

1. The repair advice now names the scoped command with the RID substituted. Split in two, which the report did
   not distinguish: the RECORD scope is the right answer only where the delete was REFUSED and the vertex is
   still there to repair. The force arms log AFTER the record is gone, so a scoped check would name a record
   that no longer exists - they keep the whole-database form and say what the scoped form buys when run first.
   And at the other-end conflict the broken list belongs to the NEIGHBOUR, so that is the RID named.
   The previously bare rethrow now carries the advice instead of only getEdgeHeadChunkForWrite's
   "concurrent commit in flight", which is the diagnosis for the transient case nobody ever sees.

2. ConcurrentModificationException gains the (message, cause) constructor NeedRetryException already declared.
   Eight sites across GraphEngine, EdgeLinkedList and StripedEdgeList kept the message and dropped the stack.
   A conflict is normally absorbed by the retry, so the run that surfaces one is the retry-exhausted run -
   exactly the one whose trace has to be diagnosable. The class on the wire is unchanged.

3. checkDocuments and checkScopedDocuments report a corrupt document identically. The type-wide arm added to
   the warning and corrupted-record SETS without touching totalWarnings/totalCorruptedRecords, called a
   document a "vertex" it was "removing" when nothing on that path removes anything, and honoured no
   maxWarnings cap at all. Both now go through shared addWarning/addCorrupted.

4. The scoped vertex check running one pass where the type-wide runs two is NOT a gap: lookupByRID IS the
   raw-view materialisation (same newImmutableRecord(type, rid, bucket.getRecord(rid).copyOfContent())) and
   checkConnectivity opens with the same asVertex(true) that forces the decode, while getRecord and scan
   resolve placeholders and multi-page chains identically. Documented and pinned by a test rather than
   "fixed" by adding a second pass, which would double the cost of the scope that exists to be cheap.

The advice tests execute what the message says: the command is parsed out of the raised exception and run
verbatim, then the delete must succeed - so a wrong RID or a shape the parser rejects fails here.

* review: state addCorrupted's dedup guarantee honestly, and drop the reflection in the advice test

Three points from the code review on #5767, all minor and none blocking.

1. addCorrupted's "a record flagged twice is one corrupted record" holds only while the retained set is under
   the cap. Behaviour kept (the contains() check past the cap is strictly better than an unconditional
   increment); the Javadoc now says exactly what survives the cap and why closing the gap is not worth it - the
   only exact fix is a second unbounded set of every RID ever counted, i.e. the memory the cap exists to refuse.
   Also recorded that neither caller reaches it: the bucket scan visits each RID once and the RECORD scope is a
   Set, so a duplicate cannot survive as far as groupRecordsByType.

2. The reviewer flagged that the retained set is capped with maxWarnings while the graph arms take a separate
   maxCorrupted. Documented rather than changed: DatabaseChecker has no maxCorrupted field, and checkScopedRecords
   already passes the remaining maxWarnings budget into the GraphDatabaseChecker parameter of that name. Two
   names, one setting.

3. The advice test reached the LogManager logger by reflecting on the private field. LogManager exposes a public
   getLogger() for precisely this case - a caller that swaps the logger and has to put the original back - so the
   reflection helper is gone along with its rename fragility.

* review: record the maxWarnings/maxCorrupted coupling at the field, and why the interpolated cause is not redundant

Three non-blocking observations from the second review on #5767. All three resolve to recording constraints a
future reader would otherwise have to rediscover - one of them by breaking a test.

1. maxWarnings caps the corrupted set while GraphDatabaseChecker takes a maxCorrupted for the SAME set. The
   method Javadoc said "two names, one setting"; the note now lives on the field, where a reader meets it, and
   adds the consequence the reviewer pointed out: retained corruptedRecords become affectedBuckets, so a record
   dropped past the cap does not get its bucket's indexes rebuilt under FIX.

2. The advice test swaps the process-wide LogManager logger. Restored in a finally and safe under sequential
   class execution, but it is shared state; documented as a latent constraint if class-level parallelism is ever
   enabled, along with why there is no per-invocation seam to capture through instead.

3. getEdgeHeadChunkForWrite and hasNextEdgeToDelete interpolate e.getMessage() AND keep e as the cause, which
   reads as redundant. It is not: the missing CHUNK's RID appears ONLY in the record-not-found message - the
   surrounding text names the vertex - and the top-level message is what a log line and an HTTP error body carry,
   while the cause chain surfaces only in a full trace. Removing it would drop the RID and fail
   Issue5670EdgeDeleteDanglingBackRefTest, which asserts on it. Commented at both sites.

* review: guard the advice against a message-less conflict, and record why otherEndOf can equal the vertex

Two of the three optional-polish points from the third review on #5767 (which was LGTM).

1. withRepairAdvice built the message as e.getMessage() + advice. Every conflict on this path carries a message
   today, but a message-less one would render "null. If it persists...", which reads as a bug in the advice rather
   than as a missing diagnosis. Falls back to toString(), i.e. the class name.

2. otherEndOf would answer vertexRID itself for a self-loop, which the method name reads as impossible. That is
   the correct answer - a self-loop has no other end and the list that failed to disconnect it IS the vertex's own
   - so the equality is documented rather than "simplified" away. Also recorded why there is no test: the caller
   runs only after both of the vertex's lists have been collected, and a self-loop is read out of those same two
   lists, so a list broken enough to fail the disconnection fails the collection first.

Not taken: the reviewer flagged that otherEndOf is computed before the force branch, so it runs on every retry of
a transient conflict. Both branches need the value, so moving it saves nothing - and the reviewer scoped it as
"for completeness" on an exception path where asEdge() is light.

(cherry picked from commit c7349f8)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant