Repository navigation
feat(engine) #5680: CHECK DATABASE RECORD <rid> checks and repairs named records only - #5710
Conversation
|
Tick the box to add this pull request to the merge queue (same as
|
Up to standards ✅🟢 Issues
|
| Metric | Results |
|---|---|
| Complexity | 32 |
NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.
Review: #5680 vertex delete no longer outlives its edgesThorough, well-reasoned PR. I traced the strict/best-effort split through Strengths
Minor observations (non-blocking)
VerificationI reviewed against the engine's page-conflict model ( Overall: high quality, ship-ready modulo the minor notes above. |
|
Review: issue #5680 - vertex delete no longer outlives its edges Reviewed the full diff plus surrounding code in GraphEngine, DatabaseChecker, GraphDatabaseChecker, EdgeIterator/StripedEdgeList and the SQL wiring. This is a careful, well-reasoned fix and the write-up is exemplary: the mutation-checked tests, asserted preconditions, and the symmetry with the #5670 edge-side work all inspire confidence. Structurally everything lines up (edgeIteratorForWrite, allChains(true), getPreviousRID, the scoped checkVertices/checkEdges overloads, stats merge without double-counting). A few points, one worth a closer look. 1. isCorruptedBuffer tolerance can still leave ghost edges on a mid-chain corrupt buffer (medium) collectEdgesToDelete (GraphEngine.java:764-772) rethrows with the repair hint only when !isCorruptedBuffer(e). The Javadoc intent is that the tolerated case is a vertex whose own record buffer is corrupt, so the edge pointers cannot be read at all - i.e. nothing was collected. But isCorruptedBuffer keys purely on the exception type (SerializationException, IndexOutOfBoundsException, BufferUnderflowException, IllegalArgumentException, ...), not on where the failure happened. EdgeIterator.previousContainer() only converts RecordNotFoundException into a strict conflict; a chunk whose buffer fails to deserialize mid-walk throws one of the corrupt-buffer family instead. That escapes to this catch, is classified as a corrupted buffer, and is swallowed - after some edges were already added to edgesToDelete. The delete then proceeds with a partial collection and deletes the vertex, leaving every entry behind the corrupt chunk as a ghost edge, with no CHECK DATABASE RECORD ... FIX hint surfaced. That is precisely the outcome the PR sets out to prevent, just reached through a corrupt (rather than missing) mid-chain chunk. Concretely: a mid-chain EdgeSegment whose page bytes are damaged (not merely relinked away) -> edgeIteratorForWrite yields the head-side edges, then throws SerializationException/IndexOutOfBoundsException -> isCorruptedBuffer true -> swallowed -> vertex deleted, tail edges ghosted. Suggestion: gate the corrupt-buffer tolerance on edges == null (nothing walked yet), so the #4420/#4432 vertex-body-unreadable case stays tolerant while a failure that occurs after the walk started is treated as a genuinely broken chain and routed to enrichWithRepairHint. That preserves the invariant the whole PR rests on: once collection has started, an incomplete collection must never complete the delete. (Not a perfect split - a corrupt head chunk also leaves edges == null - but it closes the mid-chain hole, the one that silently strips edges after collecting some.) 2. enrichWithRepairHint drops the causal chain (low) new ConcurrentModificationException(hint) (around GraphEngine.java:830) does not chain cause, so the original stack trace is lost except for what was inlined into hint. Since that type extends NeedRetryException, consider preserving the cause for diagnosability on the run where the retry budget is finally exhausted and the exception does surface. 3. Minor / consistency notes
Strengths worth calling out
Overall: solid fix, good tests, aligned with the codebase page-conflict/retry model. Point 1 is the one I would like to see addressed or explicitly dismissed before merge. Reviewed by Claude Code. |
|
Round 2 addressed in 1. You are right that keying on the exception type alone does not express the intent. The tolerance is now scoped by construction rather than by classification: I preferred this to your suggested Caveat worth stating plainly: I could not build a fixture where the old classification actually stranded edges. I injected a mid-chain chunk whose 2. Causal chain dropped — fixed. 3. Verification: |
Code review: #5680 vertex delete no longer outlives its edgesReviewed the full diff (all 3 commits). This is a high-quality, well-reasoned change. The core fix is correct: routing The test suite is a real strength: 8 targeted tests, the mutation-check note (3 fail with the fix reverted, failing in the right way), preconditions that assert the chain really spans several chunks and the victim sits behind the hole, and message matching so a test cannot pass on an unrelated throw. Fixing the vacuously-passing A few minor points, none blocking: 1. 2. Breadth of 3. Forced-delete log message can misattribute. In the edge-delete loop, the 4. Hot-path cost on healthy deletes. Every vertex delete now does up to four 5. Overall: correct, well-tested, and the documentation/reasoning in the Javadoc and release notes is unusually thorough. Nice work. Reviewed with Claude Code. |
|
Round 3 addressed in 1. I deliberately left the shared per-record bodies ( 2. Breadth of
So the broad members are not reachable through this call. I kept the set aligned with the family 3. Log line could misattribute — fixed. You are right that 4. Hot-path cost — reduced. Acting on the flag rather than just noting it: 5. Grammar vs runtime rejection — agreed, keeping the runtime rejection for the tailored message. Verification: |
|
Code review: PR 5710 - strict vertex delete + CHECK DATABASE RECORD I read the full diff plus the surrounding GraphEngine, DatabaseChecker, GraphDatabaseChecker, the grammar/AST wiring and the new test. This is a careful, well-reasoned change - the fix mirrors the edge-side strictness from issues 5670/5678 symmetrically on the vertex side, the tolerances are deliberately scoped, and the Javadoc explains why at each decision point. The tests are strong: preconditions are asserted so they cannot pass vacuously, the mutation-check callout is credible, and the damaged-vs-missing chunk distinction is pinned. Nice work. A few observations, mostly minor. Correctness / behaviour
Tests
Things I checked that are fine
Overall this looks solid and safe to merge; items 1-3 are polish, not blockers. |
|
Round 4 addressed in 1. Repair hint lost from the message on the damaged-buffer path — fixed. You were right, and you were also right that the test would not have caught it: I checked the objection I had originally raised against wrapping (callers branch on the type) rather than trusting it: the corrupt-buffer family Same treatment added to the per-edge disconnect loop, naming the edge as well, since the broken list may be the one on the vertex at the other end. The test now pins the exception type, the 2. Forced delete leaves a neighbour-side dangling reference — documented. Added to both the 3. Global Verification: |
…med records only CHECK DATABASE could be narrowed to a TYPE or a BUCKET, but not to a record. That is the sharp edge of the strict vertex delete merged in #5707: a genuinely broken edge chain makes its vertex undeletable until CHECK DATABASE ... FIX rebuilds the adjacency from the surviving edge records, and the smallest way to ask for that repair was two full passes over the whole vertex type. CHECK DATABASE RECORD #12:3 FIX CHECK DATABASE RECORD #12:3, #12:9 FIX The per-record work and the fix are identical to the type-wide run, including the edge-list rebuild; only the enumeration changes. Combines with FIX and COMPRESS. Rejected when combined with TYPE or BUCKET - RECORD is already the narrowest scope, so letting one silently win would run a check nobody asked for and an intersection would be a third semantics nobody asked for. The database-wide passes (buckets, external properties, indexes) and the orphaned-segment reclaim are skipped, since none can be narrowed to a record. Two costs it deliberately does not bound, documented rather than glossed: rebuilding an adjacency needs every surviving edge that points at the vertex and no index maps endpoints back to edges, so the scoped run saves the vertex passes but still scans the edge types; and a listed record that turns out to be corrupted still has every index on its bucket rebuilt. Also fixes a vacuously-passing assertion in Issue5670EdgeDeleteDanglingBackRefTest: it asserted totalErrors == 0, and check database reports no such field, so the null-tolerant reader turned it into 0 == 0 on every run whatever the check found. It now asserts on autoFix, invalidLinks, totalWarnings and totalCorruptedRecords, and fails loudly on a field that does not exist.
c6e5de8 to
1e55f6c
Compare
The setup job failed on a trailing-whitespace line an earlier edit left after a comment in the RECORD-scoped edge branch. Removed, along with the stray blank line beside it.
Review:
|
Review:
|
…ogress step stops double-counting Two independent review passes on the RECORD scope, four actionable points between them. - A listed RID that simply is not there was recorded through addCorrupted, which puts its BUCKET into affectedBuckets - so CHECK DATABASE RECORD ... FIX dropped and rebuilt every index on that bucket, a full bucket scan, over what is usually a mistyped or already-deleted RID. That is exactly the cost the scope exists to avoid, and the scope is meant to be hand-typed after a failed delete. RecordNotFoundException is now reported and never flagged corrupted, in all three arms (vertex, edge, document). Mutation-checked: restoring addCorrupted fails the new test with totalCorruptedRecords=1, autoFix=1. - checkScopedRecords bumped currentStep for every group, but the document arm reaches stepBegin, which bumps it again - so a scope containing a plain document reported "step 3 of 2" against a totalSteps budgeted at one step per group. Each arm now owns its bump; the graph arms need it because they pass a fixed value into the nested GraphDatabaseChecker instead. - setRecords and the statement's stream now drop null RIDs. Rid.toRecordId can answer null for a non-literal form, and groupRecordsByType would meet it as an NPE on getBucketId(). Not reachable through the RECORD #n:n literal grammar, but setRecords(Set<RID>) is public API. - The parser test's cross-reference named a test class that no longer exists after the rescope. Tests: four added for the branches that were unexercised - an edge-typed RID, a document-typed RID (the branch the progress bug lived in), several RIDs spanning different types in one command, and a missing-but-valid RID under FIX, which pins the first item above.
|
Both review passes addressed in 1. A missing RID was treated as corruption — fixed, and this one had teeth.
Mutation-checked rather than assumed: restoring 2. Progress step double-count — fixed. Every group bumped 3. Null RIDs — hardened. 4. Stale cross-reference — fixed. The parser test named a class that no longer exists after the rescope. Test gaps — closed. Four added, for the branches nothing exercised: an edge-typed RID, a document-typed RID (the branch the progress bug lived in), several RIDs spanning different types in one command (the point of On the verification numbersBeing precise, since my earlier "1335 green" is worth qualifying: a sweep reported 3 failures ( |
Up to standards ✅🟢 Issues
|
| Metric | Results |
|---|---|
| Complexity | 44 |
NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.
Review:
|
…, not widened to the whole database
Review round on the rescoped PR: LGTM with three minor points, two taken.
- CHECK DATABASE RECORD whose RIDs all resolve to null left the checker with an empty scope, which
reads as "no scope" - so a command that explicitly named records quietly ran a FULL database
check. The review judged this unreachable from SQL; it is not. Only the `#n:n` literal always
resolves, while the non-literal {"@Rid": <expression>} form goes through an expression that can
answer null, so `CHECK DATABASE RECORD {"@Rid": null}` reached it. Refused now, with a test that
fails when the guard is stubbed out.
- A warning dropped for exceeding maxWarnings was discarded silently in the two scoped arms, where
GraphDatabaseChecker.addWarning logs it. Extracted addScopedWarning so both behave the same:
always counted, retained under the cap, logged when dropped.
Not taken: extracting the duplicated RecordNotFoundException/Exception catch blocks in the scoped
vertex and edge arms. They would need six accumulators threaded through a shared helper, which reads
worse than the duplication, and the surrounding type-wide methods already parallel each other the
same way - the review noted as much in raising it.
|
Addressed in 1. Silent fallback to a full-database check — fixed, and it was more reachable than we both thought. You judged this "reachable only through the public CHECK DATABASE RECORD {"@rid": null}and quietly became a whole-database scan. Now refused with a message naming how many records were given and that none resolved. Mutation-checked: stubbing the guard to Worth flagging since it moves the finding from "defensive only" to "a user can hit this", and on a large database the difference between the requested scope and a full check is not small. 2. Over-cap warnings dropped silently — fixed. Extracted 3. Duplicated catch blocks — not taken. A shared helper would need the six accumulators ( Verification
This run also settles the artifact I flagged last time: run as a single Maven invocation with a clean |
Review:
|
… protects the public API too Review round: LGTM once the setRecords widening note was considered. Went one step further than the note it asked for. The guard against "a RECORD scope that resolves to nothing silently becomes a full database check" lived only in the SQL layer, so an embedded caller reaching DatabaseChecker.setRecords directly still hit the widening. The review reasoned that setRecords cannot tell "named records, none usable" from "named no records" - true only because the SQL layer filtered the nulls out BEFORE calling it. It no longer does: the nulls are passed through, setRecords owns both the filtering and the refusal, and one guard now covers both entry points. A genuinely empty (or null) set still means "no scope" and is accepted. Both are asserted. Also: the document arm said "cannot be loaded, removing it", but nothing removes the record there - corruptedRecords only drives the index rebuild. The review called the wording inherited and asked for no action; it is new code here, so it now says what is true, with a note on why it diverges from the type-wide checkDocuments (which also calls a document a "vertex"). Adds the warning-cap test the review asked for: 40 bogus RIDs against maxWarnings=10 retains 10 messages and still counts 40. Not added: an execution test for RECORD ... FIX COMPRESS. The compress pass is shared, unscoped code already covered elsewhere, and routing it through the scoped enumeration exercises no new branch.
|
Addressed in 1. You suggested a Javadoc note on the grounds that the distinction "named records but none resolved" vs "named no records" "genuinely can't be reconstructed inside A genuinely empty set — and 2. Document arm's "removing it" — fixed. You judged this inherited and needing no action. It is inherited in 3. Not added: an execution test for Verification
|
|
Review: CHECK DATABASE RECORD (PR #5710) I read the full diff plus the surrounding What works well
Minor notes (non-blocking)
Verified, not an issue
Nice work - the comments carry the why at every non-obvious decision, which made this reviewable. Automated review by Claude. I could not run mvn in this environment, so the compile/test claims in the PR description were taken as reported; the above is a static read of the diff and surrounding code. |
…record fix cost with a test Review approved with three non-blocking notes; two taken. - COMPRESS is unaffected by the RECORD scope and still compresses the whole database. It is opt-in, but it is the one clause that breaks the "naming a record bounds the cost" promise, so it now sits in the release notes beside the other two costs the scope does not bound. - Added the test the review asked for: a listed record that is GENUINELY corrupted - not merely missing, not merely chain-broken - is flagged, and under FIX every index on its bucket is dropped and rebuilt. That turns the "RECORD bounds the check, not necessarily the fix" paragraph into an executable assertion, so a future change cannot quietly make a scoped repair either cheaper or more destructive than documented. Mutation-checked: dropping addCorrupted from the scoped arm fails it, and the injected corruption is the intended shape (buffer truncated to 8 bytes against the 25-byte vertex prefix), distinct from the missing-RID path. Not taken: moving the RECORD-plus-TYPE/BUCKET rejection from check() into the setters. setTypes and setRecords can be called in either order, so a setter-level guard would fire or not depending on call order; check() is the only place that sees the whole configuration. The review called the current placement acceptable.
|
Addressed in 1. 2. Corrupted-record
Mutation-checked: dropping 3. Setter-time rejection of Verification
|
|
Review: CHECK DATABASE RECORD (#5680 follow-up) Reviewed the full diff plus the surrounding DatabaseChecker / GraphDatabaseChecker flow. This is a genuinely high-quality change: the scoped enumeration is layered cleanly onto the existing per-record workers, the step/progress accounting is handled carefully, and the test suite is thorough (missing-vs-corrupt distinction, cross-type grouping, warning caps, the "is it really a scope?" test, and the anti-vacuous rewrite of assertIntegrityClean). The documentation of the costs the scope does not bound is exemplary. A few observations, all minor. Correctness / design
Minor
Style / conventions Consistent with the codebase: final on locals/params, imported types rather than FQNs, JSON handling untouched, tests use the assertThat(...) style from CLAUDE.md. One convention check: checkDatabaseRecordFixMakesTheVertexDeletableAgain and the multi-type test create 200 edges one-transaction-each. If these run noticeably long on CI, consider @tag("slow") per the project convention. Otherwise nothing to change. Nice work - the "costs this does NOT bound" honesty and the vacuous-test fix are the standout parts. Note: I reviewed the diff and surrounding code but did not re-run the build/test suite locally; the PR own verification run is the source for the 1335-tests-passing claim. |
…ed, and a comment stops overselling Review round: approve, all notes minor. Three taken. - The "Exception, not Throwable" comment claimed more protection than the code gives. It guards the LOOKUP, but the shared per-record consumer still catches Throwable around the connectivity check itself - matching the type-wide path, which is why it stays. The comment now says exactly what is and is not covered rather than implying an Error can never be recorded as corruption. - setRecords refused an all-null scope but dropped a PARTIAL one silently, so a caller who mistyped one of several RIDs got a clean report for a check that skipped it. Narrowing is still allowed - unlike widening it cannot do more than asked - but it now surfaces a warning naming how many were dropped, with a test. - The no-type group advanced currentStep without ever naming the step, so a progress poller watched it go by blank. Routed through stepBegin like the document arm, which also owns the bump. Not taken: - Rejecting RECORD together with COMPRESS for symmetry with the TYPE/BUCKET rejection. The two are not the same case: RECORD plus TYPE/BUCKET has no coherent meaning, whereas "check this record, then compress the database" is perfectly meaningful - it is just not cheap. That is a documentation problem, and the release note now states it. - @tag("slow") on the 200-edge tests. Measured rather than assumed: the whole class runs in 0.53s, and CLAUDE.md's criteria are multi-second elapsed or large batches. 200 tiny transactions under a second is neither.
|
Addressed in 1. The 2. Partial null RIDs dropped silently — now surfaced. You asked whether this was intended. It is: narrowing cannot do more than the caller asked for, unlike widening, so refusing would be too harsh. But silent was wrong — a caller who mistyped one of several RIDs got a clean report for a check that skipped it. 3. No-type group advanced the step counter without naming the step — fixed. Routed through 4. 5. Verification
|
|
Review: CHECK DATABASE RECORD (PR 5710) Reviewed the full diff against main. This is a well-scoped, carefully reasoned change: the new RECORD scope reuses the existing per-record workers (GraphDatabaseChecker.checkVertices/checkEdges gain a scopedRecords overload that only swaps the enumeration), the shared index-rebuild/compress tail is untouched, and the design trade-offs are documented both in code and in the release notes. Test coverage is genuinely thorough - end-to-end repair, scope isolation, missing-vs-corrupted distinction, multi-type grouping, warning caps, and mutation-checked assertions. Nice work. A few observations, mostly minor: Code quality / correctness
Consistency note (pre-existing, not introduced here)
Minor
Verification Reviewed the logic (step-counter accounting, warning-cap propagation via maxWarnings minus currentWarnings, updateStats double-count avoidance, getTypeByBucketId returning null for unknown buckets) but did not run the build locally. The PR reports 1335 tests passing across the affected packages, which matches what the change touches. Recommend confirming CI is green before merge. Overall: solid, defensible, and unusually well-documented. The points above are refinements, not blockers. (Automated review by Claude.) |
…RESS is unscoped
Review round: refinements, not blockers. Four taken.
- CHECK DATABASE TYPE Hub RECORD {"@Rid": null} reported "none of them resolves to a RID", because
the RIDs are resolved before check() reaches the combination guard - pointing the operator at the
inner failure while the actual mistake is the clause combination. The statement now diagnoses the
combination before resolving anything; check() keeps its own guard for programmatic callers.
- setRecords left droppedRecords stale on its early-return path. Harmless today, since the warning
is only emitted under a record scope, but the field is public-API-visible state and should not
depend on call order.
- COMPRESS now warns that it is not limited by the RECORD scope. The review suggested this as a
middle ground between refusing the combination and staying silent, and it is a better answer than
the documentation-only one: the command still does what it was asked, but an operator who named a
record and expected a bounded run is told otherwise. Tested both ways, so the warning tracks the
clause rather than the scope.
- Recorded, not fixed: the type-wide checkDocuments adds to the warnings/corrupted sets without
incrementing the corresponding totals, so a corrupt document reports different totals depending on
which path found it. The scoped arm here is the correct one; aligning the type-wide path is a
behaviour change to a command this issue does not otherwise touch.
Agreed with the review that the duplicated catch blocks read better than threading six accumulators
through a shared helper; no change.
|
Addressed in 1. Wrong error for 2. 3. 4. Scoped vs type-wide document totals — recorded, not silently diverged. Agreed the scoped arm is the more correct one. I have not changed the type-wide path: that would alter what 5. Duplicated catch blocks — agreed, no change. Verification
|
Up to standards ✅🟢 Issues
|
| Metric | Results |
|---|---|
| Complexity | 52 |
🟢 Coverage 84.29% diff coverage · -6.81% coverage variation
Metric Results Coverage variation ✅ -6.81% coverage variation Diff coverage ✅ 84.29% diff coverage Coverage variation details
Coverable lines Covered lines Coverage Common ancestor commit (4140df2) 151001 115507 76.49% Head commit (fcb2958) 183178 (+32177) 127644 (+12137) 69.68% (-6.81%) Coverage variation is the difference between the coverage for the head and common ancestor commits of the pull request branch:
<coverage of head commit> - <coverage of common ancestor commit>Diff coverage details
Coverable lines Covered lines Diff coverage Pull request (#5710) 191 161 84.29% Diff coverage is the percentage of lines that are covered by tests out of the coverable lines that the pull request added or modified:
<covered lines added or modified>/<coverable lines added or modified> * 100%
NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.
Review:
|
…t cannot know, bounded corrupted set Review round: nothing blocking. All four points taken. - The "cannot be combined with TYPE or BUCKET" text was written out in both the statement and the checker. The two-layer guard stays - the statement has to diagnose the clause conflict before the RIDs are resolved - but the wording now lives in one constant so the two cannot drift. - Two messages quoted a record count the Set could not know: several RID expressions that all resolve to null collapse to a single null element, so "given 3 record(s)" was never reachable and "1 of the record(s) did not resolve" under-reported. Rather than plumb the original count through, both messages stop quoting a number, which is accurate whatever the input. - checkScopedDocuments grew corruptedRecords without a bound while every other path honours maxCorrupted/maxWarnings. Bounded the same way, with the total still counting every occurrence. - Documented that the edge sweep a rebuild needs runs once per distinct vertex TYPE named, not once per run: ten vertices of one type cost one sweep, one vertex of each of three types cost three.
|
Addressed in 1. Duplicated error message — one owner. 2. Count collapses when several RIDs resolve to null — fixed by removing the count, not by plumbing one through. You are right that a 3. 4. Edge sweep multiplies by distinct vertex types — documented. Stated in both the Verification
|
Review: CHECK DATABASE RECORD scope (#5710)Reviewed the full diff against What is well done
Points worth a second look (none blocking)
Test coverageExcellent - happy-path repair end-to-end, proof that Style / conventionsConsistent: Note: I could not run Overall: LGTM. The items above are observations, not change requests. Automated review by Claude. |
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #5710 +/- ##
==========================================
+ Coverage 67.68% 67.74% +0.06%
==========================================
Files 1780 1780
Lines 151001 151132 +131
Branches 32029 32054 +25
==========================================
+ Hits 102199 102379 +180
+ Misses 35533 35469 -64
- Partials 13269 13284 +15 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Review:
|
…airs it, and the conflict keeps its cause (#5767) * fix(engine) #5764: a refused vertex delete names the command that repairs it, and the conflict keeps its cause Follow-ups from #5680, whose two halves landed separately: #5707 made deleteVertex strict, #5710 added the CHECK DATABASE RECORD scope. Neither is defective; the seams between them were never closed. 1. The repair advice now names the scoped command with the RID substituted. Split in two, which the report did not distinguish: the RECORD scope is the right answer only where the delete was REFUSED and the vertex is still there to repair. The force arms log AFTER the record is gone, so a scoped check would name a record that no longer exists - they keep the whole-database form and say what the scoped form buys when run first. And at the other-end conflict the broken list belongs to the NEIGHBOUR, so that is the RID named. The previously bare rethrow now carries the advice instead of only getEdgeHeadChunkForWrite's "concurrent commit in flight", which is the diagnosis for the transient case nobody ever sees. 2. ConcurrentModificationException gains the (message, cause) constructor NeedRetryException already declared. Eight sites across GraphEngine, EdgeLinkedList and StripedEdgeList kept the message and dropped the stack. A conflict is normally absorbed by the retry, so the run that surfaces one is the retry-exhausted run - exactly the one whose trace has to be diagnosable. The class on the wire is unchanged. 3. checkDocuments and checkScopedDocuments report a corrupt document identically. The type-wide arm added to the warning and corrupted-record SETS without touching totalWarnings/totalCorruptedRecords, called a document a "vertex" it was "removing" when nothing on that path removes anything, and honoured no maxWarnings cap at all. Both now go through shared addWarning/addCorrupted. 4. The scoped vertex check running one pass where the type-wide runs two is NOT a gap: lookupByRID IS the raw-view materialisation (same newImmutableRecord(type, rid, bucket.getRecord(rid).copyOfContent())) and checkConnectivity opens with the same asVertex(true) that forces the decode, while getRecord and scan resolve placeholders and multi-page chains identically. Documented and pinned by a test rather than "fixed" by adding a second pass, which would double the cost of the scope that exists to be cheap. The advice tests execute what the message says: the command is parsed out of the raised exception and run verbatim, then the delete must succeed - so a wrong RID or a shape the parser rejects fails here. * review: state addCorrupted's dedup guarantee honestly, and drop the reflection in the advice test Three points from the code review on #5767, all minor and none blocking. 1. addCorrupted's "a record flagged twice is one corrupted record" holds only while the retained set is under the cap. Behaviour kept (the contains() check past the cap is strictly better than an unconditional increment); the Javadoc now says exactly what survives the cap and why closing the gap is not worth it - the only exact fix is a second unbounded set of every RID ever counted, i.e. the memory the cap exists to refuse. Also recorded that neither caller reaches it: the bucket scan visits each RID once and the RECORD scope is a Set, so a duplicate cannot survive as far as groupRecordsByType. 2. The reviewer flagged that the retained set is capped with maxWarnings while the graph arms take a separate maxCorrupted. Documented rather than changed: DatabaseChecker has no maxCorrupted field, and checkScopedRecords already passes the remaining maxWarnings budget into the GraphDatabaseChecker parameter of that name. Two names, one setting. 3. The advice test reached the LogManager logger by reflecting on the private field. LogManager exposes a public getLogger() for precisely this case - a caller that swaps the logger and has to put the original back - so the reflection helper is gone along with its rename fragility. * review: record the maxWarnings/maxCorrupted coupling at the field, and why the interpolated cause is not redundant Three non-blocking observations from the second review on #5767. All three resolve to recording constraints a future reader would otherwise have to rediscover - one of them by breaking a test. 1. maxWarnings caps the corrupted set while GraphDatabaseChecker takes a maxCorrupted for the SAME set. The method Javadoc said "two names, one setting"; the note now lives on the field, where a reader meets it, and adds the consequence the reviewer pointed out: retained corruptedRecords become affectedBuckets, so a record dropped past the cap does not get its bucket's indexes rebuilt under FIX. 2. The advice test swaps the process-wide LogManager logger. Restored in a finally and safe under sequential class execution, but it is shared state; documented as a latent constraint if class-level parallelism is ever enabled, along with why there is no per-invocation seam to capture through instead. 3. getEdgeHeadChunkForWrite and hasNextEdgeToDelete interpolate e.getMessage() AND keep e as the cause, which reads as redundant. It is not: the missing CHUNK's RID appears ONLY in the record-not-found message - the surrounding text names the vertex - and the top-level message is what a log line and an HTTP error body carry, while the cause chain surfaces only in a full trace. Removing it would drop the RID and fail Issue5670EdgeDeleteDanglingBackRefTest, which asserts on it. Commented at both sites. * review: guard the advice against a message-less conflict, and record why otherEndOf can equal the vertex Two of the three optional-polish points from the third review on #5767 (which was LGTM). 1. withRepairAdvice built the message as e.getMessage() + advice. Every conflict on this path carries a message today, but a message-less one would render "null. If it persists...", which reads as a bug in the advice rather than as a missing diagnosis. Falls back to toString(), i.e. the class name. 2. otherEndOf would answer vertexRID itself for a self-loop, which the method name reads as impossible. That is the correct answer - a self-loop has no other end and the list that failed to disconnect it IS the vertex's own - so the equality is documented rather than "simplified" away. Also recorded why there is no test: the caller runs only after both of the vertex's lists have been collected, and a self-loop is read out of those same two lists, so a list broken enough to fail the disconnection fails the collection first. Not taken: the reviewer flagged that otherEndOf is computed before the force branch, so it runs on every retry of a transient conflict. Both branches need the value, so moving it saves nothing - and the reviewer scoped it as "for completeness" on an exception path where asEdge() is light.
…med records only (#5710) * feat(engine) #5680: CHECK DATABASE RECORD <rid> checks and repairs named records only CHECK DATABASE could be narrowed to a TYPE or a BUCKET, but not to a record. That is the sharp edge of the strict vertex delete merged in #5707: a genuinely broken edge chain makes its vertex undeletable until CHECK DATABASE ... FIX rebuilds the adjacency from the surviving edge records, and the smallest way to ask for that repair was two full passes over the whole vertex type. CHECK DATABASE RECORD #12:3 FIX CHECK DATABASE RECORD #12:3, #12:9 FIX The per-record work and the fix are identical to the type-wide run, including the edge-list rebuild; only the enumeration changes. Combines with FIX and COMPRESS. Rejected when combined with TYPE or BUCKET - RECORD is already the narrowest scope, so letting one silently win would run a check nobody asked for and an intersection would be a third semantics nobody asked for. The database-wide passes (buckets, external properties, indexes) and the orphaned-segment reclaim are skipped, since none can be narrowed to a record. Two costs it deliberately does not bound, documented rather than glossed: rebuilding an adjacency needs every surviving edge that points at the vertex and no index maps endpoints back to edges, so the scoped run saves the vertex passes but still scans the edge types; and a listed record that turns out to be corrupted still has every index on its bucket rebuilt. Also fixes a vacuously-passing assertion in Issue5670EdgeDeleteDanglingBackRefTest: it asserted totalErrors == 0, and check database reports no such field, so the null-tolerant reader turned it into 0 == 0 on every run whatever the check found. It now asserts on autoFix, invalidLinks, totalWarnings and totalCorruptedRecords, and fails loudly on a field that does not exist. * style(engine) #5680: strip trailing whitespace flagged by pre-commit The setup job failed on a trailing-whitespace line an earlier edit left after a comment in the RECORD-scoped edge branch. Removed, along with the stray blank line beside it. * fix(engine) #5680: a missing RID is not corruption, and the scoped progress step stops double-counting Two independent review passes on the RECORD scope, four actionable points between them. - A listed RID that simply is not there was recorded through addCorrupted, which puts its BUCKET into affectedBuckets - so CHECK DATABASE RECORD ... FIX dropped and rebuilt every index on that bucket, a full bucket scan, over what is usually a mistyped or already-deleted RID. That is exactly the cost the scope exists to avoid, and the scope is meant to be hand-typed after a failed delete. RecordNotFoundException is now reported and never flagged corrupted, in all three arms (vertex, edge, document). Mutation-checked: restoring addCorrupted fails the new test with totalCorruptedRecords=1, autoFix=1. - checkScopedRecords bumped currentStep for every group, but the document arm reaches stepBegin, which bumps it again - so a scope containing a plain document reported "step 3 of 2" against a totalSteps budgeted at one step per group. Each arm now owns its bump; the graph arms need it because they pass a fixed value into the nested GraphDatabaseChecker instead. - setRecords and the statement's stream now drop null RIDs. Rid.toRecordId can answer null for a non-literal form, and groupRecordsByType would meet it as an NPE on getBucketId(). Not reachable through the RECORD #n:n literal grammar, but setRecords(Set<RID>) is public API. - The parser test's cross-reference named a test class that no longer exists after the rescope. Tests: four added for the branches that were unexercised - an edge-typed RID, a document-typed RID (the branch the progress bug lived in), several RIDs spanning different types in one command, and a missing-but-valid RID under FIX, which pins the first item above. * fix(engine) #5680: a RECORD scope that resolves to nothing is refused, not widened to the whole database Review round on the rescoped PR: LGTM with three minor points, two taken. - CHECK DATABASE RECORD whose RIDs all resolve to null left the checker with an empty scope, which reads as "no scope" - so a command that explicitly named records quietly ran a FULL database check. The review judged this unreachable from SQL; it is not. Only the `#n:n` literal always resolves, while the non-literal {"@Rid": <expression>} form goes through an expression that can answer null, so `CHECK DATABASE RECORD {"@Rid": null}` reached it. Refused now, with a test that fails when the guard is stubbed out. - A warning dropped for exceeding maxWarnings was discarded silently in the two scoped arms, where GraphDatabaseChecker.addWarning logs it. Extracted addScopedWarning so both behave the same: always counted, retained under the cap, logged when dropped. Not taken: extracting the duplicated RecordNotFoundException/Exception catch blocks in the scoped vertex and edge arms. They would need six accumulators threaded through a shared helper, which reads worse than the duplication, and the surrounding type-wide methods already parallel each other the same way - the review noted as much in raising it. * fix(engine) #5680: the empty-scope guard moves into setRecords, so it protects the public API too Review round: LGTM once the setRecords widening note was considered. Went one step further than the note it asked for. The guard against "a RECORD scope that resolves to nothing silently becomes a full database check" lived only in the SQL layer, so an embedded caller reaching DatabaseChecker.setRecords directly still hit the widening. The review reasoned that setRecords cannot tell "named records, none usable" from "named no records" - true only because the SQL layer filtered the nulls out BEFORE calling it. It no longer does: the nulls are passed through, setRecords owns both the filtering and the refusal, and one guard now covers both entry points. A genuinely empty (or null) set still means "no scope" and is accepted. Both are asserted. Also: the document arm said "cannot be loaded, removing it", but nothing removes the record there - corruptedRecords only drives the index rebuild. The review called the wording inherited and asked for no action; it is new code here, so it now says what is true, with a note on why it diverges from the type-wide checkDocuments (which also calls a document a "vertex"). Adds the warning-cap test the review asked for: 40 bogus RIDs against maxWarnings=10 retains 10 messages and still counts 40. Not added: an execution test for RECORD ... FIX COMPRESS. The compress pass is shared, unscoped code already covered elsewhere, and routing it through the scoped enumeration exercises no new branch. * docs(engine) #5680: state the COMPRESS caveat, and pin the corrupted-record fix cost with a test Review approved with three non-blocking notes; two taken. - COMPRESS is unaffected by the RECORD scope and still compresses the whole database. It is opt-in, but it is the one clause that breaks the "naming a record bounds the cost" promise, so it now sits in the release notes beside the other two costs the scope does not bound. - Added the test the review asked for: a listed record that is GENUINELY corrupted - not merely missing, not merely chain-broken - is flagged, and under FIX every index on its bucket is dropped and rebuilt. That turns the "RECORD bounds the check, not necessarily the fix" paragraph into an executable assertion, so a future change cannot quietly make a scoped repair either cheaper or more destructive than documented. Mutation-checked: dropping addCorrupted from the scoped arm fails it, and the injected corruption is the intended shape (buffer truncated to 8 bytes against the 25-byte vertex prefix), distinct from the missing-RID path. Not taken: moving the RECORD-plus-TYPE/BUCKET rejection from check() into the setters. setTypes and setRecords can be called in either order, so a setter-level guard would fire or not depending on call order; check() is the only place that sees the whole configuration. The review called the current placement acceptable. * fix(engine) #5680: a dropped RID is reported, the no-type step is named, and a comment stops overselling Review round: approve, all notes minor. Three taken. - The "Exception, not Throwable" comment claimed more protection than the code gives. It guards the LOOKUP, but the shared per-record consumer still catches Throwable around the connectivity check itself - matching the type-wide path, which is why it stays. The comment now says exactly what is and is not covered rather than implying an Error can never be recorded as corruption. - setRecords refused an all-null scope but dropped a PARTIAL one silently, so a caller who mistyped one of several RIDs got a clean report for a check that skipped it. Narrowing is still allowed - unlike widening it cannot do more than asked - but it now surfaces a warning naming how many were dropped, with a test. - The no-type group advanced currentStep without ever naming the step, so a progress poller watched it go by blank. Routed through stepBegin like the document arm, which also owns the bump. Not taken: - Rejecting RECORD together with COMPRESS for symmetry with the TYPE/BUCKET rejection. The two are not the same case: RECORD plus TYPE/BUCKET has no coherent meaning, whereas "check this record, then compress the database" is perfectly meaningful - it is just not cheap. That is a documentation problem, and the release note now states it. - @tag("slow") on the 200-edge tests. Measured rather than assumed: the whole class runs in 0.53s, and CLAUDE.md's criteria are multi-second elapsed or large batches. 200 tiny transactions under a second is neither. * fix(engine) #5680: diagnose the clause conflict first, warn that COMPRESS is unscoped Review round: refinements, not blockers. Four taken. - CHECK DATABASE TYPE Hub RECORD {"@Rid": null} reported "none of them resolves to a RID", because the RIDs are resolved before check() reaches the combination guard - pointing the operator at the inner failure while the actual mistake is the clause combination. The statement now diagnoses the combination before resolving anything; check() keeps its own guard for programmatic callers. - setRecords left droppedRecords stale on its early-return path. Harmless today, since the warning is only emitted under a record scope, but the field is public-API-visible state and should not depend on call order. - COMPRESS now warns that it is not limited by the RECORD scope. The review suggested this as a middle ground between refusing the combination and staying silent, and it is a better answer than the documentation-only one: the command still does what it was asked, but an operator who named a record and expected a bounded run is told otherwise. Tested both ways, so the warning tracks the clause rather than the scope. - Recorded, not fixed: the type-wide checkDocuments adds to the warnings/corrupted sets without incrementing the corresponding totals, so a corrupt document reports different totals depending on which path found it. The scoped arm here is the correct one; aligning the type-wide path is a behaviour change to a command this issue does not otherwise touch. Agreed with the review that the duplicated catch blocks read better than threading six accumulators through a shared helper; no change. * fix(engine) #5680: one owner for the conflict message, no counts a Set cannot know, bounded corrupted set Review round: nothing blocking. All four points taken. - The "cannot be combined with TYPE or BUCKET" text was written out in both the statement and the checker. The two-layer guard stays - the statement has to diagnose the clause conflict before the RIDs are resolved - but the wording now lives in one constant so the two cannot drift. - Two messages quoted a record count the Set could not know: several RID expressions that all resolve to null collapse to a single null element, so "given 3 record(s)" was never reachable and "1 of the record(s) did not resolve" under-reported. Rather than plumb the original count through, both messages stop quoting a number, which is accurate whatever the input. - checkScopedDocuments grew corruptedRecords without a bound while every other path honours maxCorrupted/maxWarnings. Bounded the same way, with the total still counting every occurrence. - Documented that the edge sweep a rebuild needs runs once per distinct vertex TYPE named, not once per run: ten vertices of one type cost one sweep, one vertex of each of three types cost three. (cherry picked from commit 56b47f4)
…airs it, and the conflict keeps its cause (#5767) * fix(engine) #5764: a refused vertex delete names the command that repairs it, and the conflict keeps its cause Follow-ups from #5680, whose two halves landed separately: #5707 made deleteVertex strict, #5710 added the CHECK DATABASE RECORD scope. Neither is defective; the seams between them were never closed. 1. The repair advice now names the scoped command with the RID substituted. Split in two, which the report did not distinguish: the RECORD scope is the right answer only where the delete was REFUSED and the vertex is still there to repair. The force arms log AFTER the record is gone, so a scoped check would name a record that no longer exists - they keep the whole-database form and say what the scoped form buys when run first. And at the other-end conflict the broken list belongs to the NEIGHBOUR, so that is the RID named. The previously bare rethrow now carries the advice instead of only getEdgeHeadChunkForWrite's "concurrent commit in flight", which is the diagnosis for the transient case nobody ever sees. 2. ConcurrentModificationException gains the (message, cause) constructor NeedRetryException already declared. Eight sites across GraphEngine, EdgeLinkedList and StripedEdgeList kept the message and dropped the stack. A conflict is normally absorbed by the retry, so the run that surfaces one is the retry-exhausted run - exactly the one whose trace has to be diagnosable. The class on the wire is unchanged. 3. checkDocuments and checkScopedDocuments report a corrupt document identically. The type-wide arm added to the warning and corrupted-record SETS without touching totalWarnings/totalCorruptedRecords, called a document a "vertex" it was "removing" when nothing on that path removes anything, and honoured no maxWarnings cap at all. Both now go through shared addWarning/addCorrupted. 4. The scoped vertex check running one pass where the type-wide runs two is NOT a gap: lookupByRID IS the raw-view materialisation (same newImmutableRecord(type, rid, bucket.getRecord(rid).copyOfContent())) and checkConnectivity opens with the same asVertex(true) that forces the decode, while getRecord and scan resolve placeholders and multi-page chains identically. Documented and pinned by a test rather than "fixed" by adding a second pass, which would double the cost of the scope that exists to be cheap. The advice tests execute what the message says: the command is parsed out of the raised exception and run verbatim, then the delete must succeed - so a wrong RID or a shape the parser rejects fails here. * review: state addCorrupted's dedup guarantee honestly, and drop the reflection in the advice test Three points from the code review on #5767, all minor and none blocking. 1. addCorrupted's "a record flagged twice is one corrupted record" holds only while the retained set is under the cap. Behaviour kept (the contains() check past the cap is strictly better than an unconditional increment); the Javadoc now says exactly what survives the cap and why closing the gap is not worth it - the only exact fix is a second unbounded set of every RID ever counted, i.e. the memory the cap exists to refuse. Also recorded that neither caller reaches it: the bucket scan visits each RID once and the RECORD scope is a Set, so a duplicate cannot survive as far as groupRecordsByType. 2. The reviewer flagged that the retained set is capped with maxWarnings while the graph arms take a separate maxCorrupted. Documented rather than changed: DatabaseChecker has no maxCorrupted field, and checkScopedRecords already passes the remaining maxWarnings budget into the GraphDatabaseChecker parameter of that name. Two names, one setting. 3. The advice test reached the LogManager logger by reflecting on the private field. LogManager exposes a public getLogger() for precisely this case - a caller that swaps the logger and has to put the original back - so the reflection helper is gone along with its rename fragility. * review: record the maxWarnings/maxCorrupted coupling at the field, and why the interpolated cause is not redundant Three non-blocking observations from the second review on #5767. All three resolve to recording constraints a future reader would otherwise have to rediscover - one of them by breaking a test. 1. maxWarnings caps the corrupted set while GraphDatabaseChecker takes a maxCorrupted for the SAME set. The method Javadoc said "two names, one setting"; the note now lives on the field, where a reader meets it, and adds the consequence the reviewer pointed out: retained corruptedRecords become affectedBuckets, so a record dropped past the cap does not get its bucket's indexes rebuilt under FIX. 2. The advice test swaps the process-wide LogManager logger. Restored in a finally and safe under sequential class execution, but it is shared state; documented as a latent constraint if class-level parallelism is ever enabled, along with why there is no per-invocation seam to capture through instead. 3. getEdgeHeadChunkForWrite and hasNextEdgeToDelete interpolate e.getMessage() AND keep e as the cause, which reads as redundant. It is not: the missing CHUNK's RID appears ONLY in the record-not-found message - the surrounding text names the vertex - and the top-level message is what a log line and an HTTP error body carry, while the cause chain surfaces only in a full trace. Removing it would drop the RID and fail Issue5670EdgeDeleteDanglingBackRefTest, which asserts on it. Commented at both sites. * review: guard the advice against a message-less conflict, and record why otherEndOf can equal the vertex Two of the three optional-polish points from the third review on #5767 (which was LGTM). 1. withRepairAdvice built the message as e.getMessage() + advice. Every conflict on this path carries a message today, but a message-less one would render "null. If it persists...", which reads as a bug in the advice rather than as a missing diagnosis. Falls back to toString(), i.e. the class name. 2. otherEndOf would answer vertexRID itself for a self-loop, which the method name reads as impossible. That is the correct answer - a self-loop has no other end and the list that failed to disconnect it IS the vertex's own - so the equality is documented rather than "simplified" away. Also recorded why there is no test: the caller runs only after both of the vertex's lists have been collected, and a self-loop is read out of those same two lists, so a list broken enough to fail the disconnection fails the collection first. Not taken: the reviewer flagged that otherEndOf is computed before the force branch, so it runs on every retry of a transient conflict. Both branches need the value, so moving it saves nothing - and the reviewer scoped it as "for completeness" on an exception path where asEdge() is light. (cherry picked from commit c7349f8)
Follow-up to #5680, now that #5707 has merged the strict vertex delete.
This PR was rescoped. It originally carried its own fix for #5680; #5707 landed an equivalent (and in places wider) fix while this was in review, so everything duplicated there has been dropped and this branch was rebuilt on top of
main. What remains is the partmaindoes not have.Why
RECORD#5707 made an ordinary
deleteVertexstrict: an edge list it cannot walk is a retryable conflict rather than a short collection, and a genuinely broken chain therefore makes its vertex undeletable untilCHECK DATABASE ... FIXrebuilds the adjacency from the surviving edge records.That makes
CHECK DATABASE ... FIXthe mandated escape hatch — but it could not be aimed. The narrowest scope wasTYPEorBUCKET, so recovering one vertex meant two full passes over its entire vertex type.The per-record work and the fix are identical to the type-wide run, including the edge-list rebuild. Only the enumeration changes: a lookup per listed RID instead of two passes over every bucket of the type.
Design decisions worth flagging
Rejected, not silently resolved, when combined with
TYPE/BUCKET. The grammar accepts the combination; execution refuses it with anIllegalArgumentException(which the HTTP layer already maps to 400).RECORDis already the narrowest scope, so letting one win silently would run a check nobody asked for, and an intersection would be a third semantics nobody asked for either. Enforcing it in the grammar would be stricter but would lose the tailored message.Database-wide passes skipped. Buckets, external properties and indexes cannot be narrowed to a record, and paying for them would defeat the point of naming one. The orphaned-segment reclaim is skipped for a stronger reason: it builds its reachable set by walking every vertex, so under any partial scope it would misclassify unwalked vertices' segments as orphans.
Two costs it does NOT bound, stated rather than glossed:
RECORDbounds the check, not necessarily the fix.A test that was passing vacuously
Issue5670EdgeDeleteDanglingBackRefTest.assertIntegrityCleanassertedtotalErrors == 0, andcheck databasereports nototalErrorsfield — the null-tolerant reader turned it into0 == 0on every run, whatever the check found. It now asserts onautoFix,invalidLinks,totalWarningsandtotalCorruptedRecords, andlongPropertyfails loudly on a field that does not exist rather than silently answering 0. Included here because it guards the same area.Tests
CheckDatabaseRecordScopeTest:checkDatabaseRecordFixMakesTheVertexDeletableAgaincheckDatabaseRecordVisitsOnlyTheNamedRecordRECORDis a scope — naming a healthy vertex leaves another vertex's broken chain untouched, so a "scoped" run that quietly fell back to a type scan would failcheckDatabaseRecordRejectsBeingCombinedWithTypeOrBucketTYPEandBUCKET, plus each clause still working alonecheckDatabaseRecordReportsARidBelongingToNoTypePlus parser coverage in
CheckDatabaseStatementTestParserTest.Verification
com.arcadedb.graph.**,com.arcadedb.engine.**andcom.arcadedb.query.sql.parser.**: 1335 tests, 0 failures — includingIssue5680VertexDeleteEdgeCollectionTestfrom #5707, so the rebase did not disturb the merged fix.DatabaseChecker'sgetPageSizeForNewFile()change from #5713 is preserved (it landed in the same method this PR touches).🤖 Generated with Claude Code
https://claude.ai/code/session_01Puky78GhudfgTHPtYSuFPm