Repository navigation
fix(engine) #5680: an unreadable edge list retries the vertex delete instead of losing its edges - #5707
Conversation
…instead of losing its edges deleteVertex collected the edges to remove through the best-effort getEdgeHeadChunk with the whole walk wrapped in catch (Exception), so any part of the list that could not be read at that instant was taken for "nothing to remove here" and the vertex record was deleted on top of that empty or partial view: the edges outlived their endpoint, with a back-reference left on an innocent neighbour. Three routes reached that outcome - an unreadable head chunk, an unreadable hop mid-chain (everything behind the hole dropped), and, on a promoted super-node, a stripe chain the read walk is allowed to skip, which lost a whole stripe with no exception behind it at all. None of them is a fact about the graph; they are the transient publication window #5670 was about, which deleteEdge has answered with a retry since #5678. The collection now reads the list the way a removal must: the head through getEdgeHeadChunkForWrite, the walk through the new EdgeLinkedList.edgeIteratorForRemoval (strict over stripe chains), and the chain hops mapped to a retryable ConcurrentModificationException. force is the single escape hatch and now works end to end: it also absorbs the conflict raised while disconnecting a collected edge from the vertex at its OTHER end, which previously let a broken neighbour block a forced delete. Draining the leftover chunk records stays best-effort in both modes on purpose - by then a miss loses orphaned chunks, never a reference. The heads are also read from the transaction's own vertex instance (#5660), closing a fourth silent-loss route that needed no race. Strictness costs no repairability: CHECK DATABASE ... FIX rebuilds an unloadable chain from the surviving edge records and is never blocked by the delete being blocked.
Up to standards ✅🟢 Issues
|
| Metric | Results |
|---|---|
| Complexity | 0 |
NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.
|
Tick the box to add this pull request to the merge queue (same as
|
|
Review: issue 5680 strict edge collection on deleteVertex I reviewed the full diff (GraphEngine, EdgeLinkedList, StripedEdgeList, the new test, release notes) plus the surrounding call sites (LocalDatabase.deleteRecordNoLock, getEdgeHeadChunkForWrite, allChains, chainsForNeighbour). This is a well-reasoned fix for a genuine silent data-loss bug, and it slots cleanly into the retry contract established by issues 5670/5678. Overall: strong, ship-worthy. Notes below are mostly confirmations plus a couple of small observations. What is correct and worth calling out
Observations (non-blocking)
Style / conventions
Nice work - the trade-off analysis in the description (why strictness does not cost repairability, and the three rejected alternatives) made this easy to review with confidence. |
…lerated, and where the striped conflict is raised Review follow-ups on #5707, both documentation. The blanket catch (Exception) left in collectEdgesToDelete is the residue once the transient window is split off into the retryable branch: an undecodable buffer, which no retry can fix and which is tolerated even when force is false. That reads as inconsistent with the strictness around it until you follow where such a record comes from - LocalDatabase.deleteRecordNoLock catches the same exception family around the index cleanup and proceeds WITHOUT raising its force flag, so a corrupt-buffer vertex arrives here with force == false and failing would make it undeletable again (#4420, #4432). Its cost is larger than a single tolerated dangling entry, so the warning now names it and points at CHECK DATABASE. hasNextEdgeToDelete's javadoc claimed only a chain hop surfaces through it, which is true on the classic layout but not the whole story on a promoted super-node: StripedEdgeList.edgeIteratorForRemoval resolves every stripe head while BUILDING the iterator, so an unloadable stripe raises there instead. Same try, same handling - the note keeps a future reader from reading the helper as the only strict point.
|
Thanks - addressed both documentary observations in a95aa6f. Notes on each: 1. Residual silent-loss path under hard corruption. Correct reading, and I dug into whether it should become strict rather than documented. It should not, and the reason is concrete: So the boundary stays, but you are right that its cost is larger than the tolerated single dangling entry (everything behind the damage is dropped, and the vertex still goes). The catch now says so explicitly, and its WARNING names the case and points at 2. Classic vs striped: where the CME originates. Fair - the javadoc sentence was accurate for the classic layout and misleading as a general claim. Added a note that 3. Documentation only, no behaviour change. Re-verified: |
…ex-strict-collection # Conflicts: # engine/src/main/java/com/arcadedb/graph/GraphEngine.java
Review: #5680 strict edge collection on vertex deleteI read the full diff, the reworked What's right
Discussion points (none blocking)
Style/nits
Nice work. The fix is correct, the failing-on- |
…/tolerant split actually keys on Second review follow-ups on #5707. A self-loop is the one shape both collection walks yield, so it lands in the delete list twice and the second removal runs against a chain the first already cleaned. That is unchanged by this fix and resolves quietly, but the removal walk is stricter now and a self-loop is the shape most likely to surprise a later change here, so it gets a test rather than an argument. The two catch arms in collectEdgesToDelete are easy to read as transient-versus-permanent, which is wrong and would make the next reader draw the line in the wrong place. They key on what a MISS COSTS: a chunk that cannot be found retries because deleting the vertex on a short list loses references, and that is applied to a genuinely lost chunk too - no retry recovers it, so the delete now fails once the retries are spent instead of quietly completing. Stated explicitly, since it is a deliberate behaviour change for the lost-chunk case and not only for the transient one.
|
Thanks - second pass addressed in 11f4805. Point by point: 1. Explicit about the genuinely-lost chunk. Agreed, and worth more than a footnote: the split between the two catch arms is easy to read as transient vs permanent, which is the wrong axis and would lead the next reader to draw the line in the wrong place. It keys on what a miss costs. A chunk that cannot be FOUND retries because the alternative - deleting the vertex on a short list - loses references, and that is applied to a genuinely lost chunk too, which no retry recovers: that case now fails the delete once the retries are spent rather than quietly completing it. A chunk that cannot be DECODED takes the other arm for the 2. Self-loop test. Added 3. Whole-list materialization on super-nodes. Agreed and not changed. It is unchanged from the previous implementation, and the two-phase collect-then-delete exists precisely because the removals mutate the chain the walk is standing on - streaming it would mean walking a list while 4. Retry count in the concurrency test. No change - as you note the 5-minute ceiling caps the tail, and the budget is deliberately far beyond what the contention needs so that any throwable reaching the counter is the bug rather than a tolerated retry. Verified: |
|
Review: #5707 - vertex delete no longer loses its own edges I traced the change through the real call paths rather than just the diff. Short version: this is a correct, well-scoped fix with excellent tests and unusually good in-code documentation. A few notes below, only one of which is more than a nit, and none block merge. What I verified holds up
One thing worth a second look (non-blocking) The trailing The residual risk is that the same blanket catch would also mask a genuine programming error (e.g. an NPE or IllegalStateException from an unrelated future change) as "corrupted chunk, delete anyway" - i.e. reintroduce a silent-edge-loss path of exactly the kind this PR exists to eliminate. Since the comment already enumerates the expected exception types, consider catching that named set explicitly (or asserting on it) so an unexpected type propagates instead of being silently absorbed. Given the documented constraint this is a judgment call, not a defect. Minor
Style / conventions Consistent with the codebase: Nice work - the design rationale captured inline (strict-where-a-miss-loses-a-reference vs tolerant-where-it-loses-garbage, plus the rejected alternatives) will save the next maintainer a lot of guessing. |
…s, and scope the concurrent test to what this fix owns Third review follow-up on #5707, plus a flake the follow-up exposed. The trailing catch (Exception) in collectEdgesToDelete would have absorbed an NPE or an IllegalStateException from a future change as "corrupted chunk, delete anyway" - reintroducing exactly the silent-edge-loss path this issue exists to remove. It now names the shapes it tolerates: the decode family LocalDatabase already uses for the same purpose, plus ClassCastException and SchemaException, which are the two further shapes a corrupt EDGE LIST adds (a head RID naming a record that is not an edge segment, an edge bucket whose type is gone). Anything else propagates. While re-running, concurrentVertexDeletionAndEdgeAppendingLeaveNoEdgeBehind proved flaky (~1 in 3), and the cause is not this fix - it reproduces identically with the fix reverted. An edge an APPENDER commits while its hub is being deleted can outlive that hub: the survivor has an existing out vertex and an in naming the deleted hub, so it never existed when the deleter's collection walk ran and no vertex-side strictness can prevent it. That is a separate defect on the append path, reported separately. The test now asserts the invariant this fix actually owns - every edge that was IN the hub's list when the deleter started must be gone with it, i.e. the fixture edges - instead of "no LINK edge survives at all", which folded in the append-side defect. The round-start precondition moves from a global LINK count to a per-hub degree for the same reason, so a leftover from an earlier round cannot make a healthy round look wrong, and the end-of-run integrity check is dropped with the reason stated inline (the non-racing fixtures still assert it). 10 consecutive green runs.
|
Third pass addressed in cab5a0a - and it turned up something worth flagging separately. Closed catch (your one substantive point). Agreed, and implemented: absorbing an NPE or
A finding, not a fix: It is not this fix - it reproduces identically with the fix reverted (3 failures in 6 runs on the parent commit). Diagnosed by dumping the survivor: An edge an appender commits while its hub is being deleted can outlive that hub. The survivor has a live So the test now asserts the invariant this fix owns - every edge that was in the hub list when the deleter started must go with it, i.e. the fixture edges - instead of "no LINK edge survives at all", which quietly folded in the append-side defect. The round-start precondition moves from a global Verified: 10 consecutive green runs of the test class, |
|
Review: #5680 strict edge-list collection on vertex delete I read the four commits end to end (GraphEngine, EdgeLinkedList, StripedEdgeList, the new test, and the LocalDatabase.deleteRecordNoLock caller). This is a careful, high-quality fix that closes a real correctness hole with a coherent principle behind it ("strict where a miss loses a reference, tolerant where a miss loses only garbage"). Feedback below, mostly confirmations with a few minor considerations. What holds up well
Minor considerations (non-blocking)
Verification I did not independently re-run the suite; relying on the reported results (engine 10743/0, server 950/0, Issue5680 8/8 x3). Given the visible-behaviour change (DELETE VERTEX can now raise ConcurrentModificationException), the release-note entry and the server auto-retry coverage are the right mitigations, and the explicit-transaction contract note is accurate. Nice work - the reasoning is sound and the tests pin exactly the three routes plus the force gap. |
…ce in the tolerated set Fourth review follow-up on #5707, comment only. IllegalArgumentException is the loosest member of the closed catch list and the one a future change is most likely to throw for a non-corruption reason. It stays, because Binary raises it ("Invalid position", Binary.java:179) for a content offset that decodes past the end of the buffer - a genuine corruption shape, and the reason LocalDatabase carries it in the same family - and dropping it would make a vertex with that damage undeletable again. Naming the origin means the periodic second look the review asks for can be done without re-deriving it, and narrowing it later is a decision with the evidence attached rather than a guess.
|
Fourth pass - all three points were flagged non-blocking, and I agree with that classification. One is now recorded in the code (07a769c, comment only); the other two I am deliberately not changing, with reasons. 1. Breadth of the decode catch arm, 2. Comment-to-code ratio. Not changing. You called it a taste note and I agree it is dense, but the engine 3. Verified after the change: That closes every point raised across the four passes. Recapping what was not implemented and why, so it is on the record in one place: the whole-list materialization for super-nodes (unchanged from before this PR; streaming it means walking a list while Separately, and not a review point: this work surfaced a pre-existing defect on the append path - an edge committed by |
|
Review of #5707 - vertex delete retries an unreadable edge list instead of losing its edges I traced the change end to end (collectEdgesToDelete -> EdgeLinkedList.edgeIteratorForRemoval -> EdgeIterator / EdgeIteratorFilter / IteratorFilterBase.getPrevious for the classic chain, and StripedEdgeList.addChain(strict=true) for the striped layout) and the behaviour matches every claim in the description. This is a careful, well-scoped fix. Notes below. What holds up under inspection
Test coverage Strong. 8 tests with 3 that fail on main, deterministic corruption fixtures that assert their own preconditions (degree, chunk count, promotion) so they cannot pass on an unrelated throw, plus the self-loop control and the @tag('slow') concurrent invariant test scoped to what this fix owns. The honest call-out that the super-node end-to-end test is vacuous (raises with or without the fix) and the switch to asserting the two walks side by side is exactly the right way to keep it meaningful. Minor observations (non-blocking)
Overall this reads as a correct, well-tested fix that closes four distinct silent-loss routes and makes force do what it always claimed. Nice work. Reviewed with Claude Code. |
Up to standards ✅🟢 Issues
|
| Metric | Results |
|---|---|
| Complexity | 0 |
🟢 Coverage 90.74% diff coverage · -6.28% coverage variation
Metric Results Coverage variation ✅ -6.28% coverage variation Diff coverage ✅ 90.74% diff coverage Coverage variation details
Coverable lines Covered lines Coverage Common ancestor commit (ffe7865) 150555 113861 75.63% Head commit (07a769c) 175578 (+25023) 121764 (+7903) 69.35% (-6.28%) Coverage variation is the difference between the coverage for the head and common ancestor commits of the pull request branch:
<coverage of head commit> - <coverage of common ancestor commit>Diff coverage details
Coverable lines Covered lines Diff coverage Pull request (#5707) 54 49 90.74% Diff coverage is the percentage of lines that are covered by tests out of the coverable lines that the pull request added or modified:
<covered lines added or modified>/<coverable lines added or modified> * 100%
NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #5707 +/- ##
============================================
+ Coverage 66.74% 67.63% +0.89%
============================================
Files 1774 1729 -45
Lines 150555 143554 -7001
Branches 31938 30791 -1147
============================================
- Hits 100487 97093 -3394
+ Misses 36732 33745 -2987
+ Partials 13336 12716 -620 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
…med records only CHECK DATABASE could be narrowed to a TYPE or a BUCKET, but not to a record. That is the sharp edge of the strict vertex delete merged in #5707: a genuinely broken edge chain makes its vertex undeletable until CHECK DATABASE ... FIX rebuilds the adjacency from the surviving edge records, and the smallest way to ask for that repair was two full passes over the whole vertex type. CHECK DATABASE RECORD #12:3 FIX CHECK DATABASE RECORD #12:3, #12:9 FIX The per-record work and the fix are identical to the type-wide run, including the edge-list rebuild; only the enumeration changes. Combines with FIX and COMPRESS. Rejected when combined with TYPE or BUCKET - RECORD is already the narrowest scope, so letting one silently win would run a check nobody asked for and an intersection would be a third semantics nobody asked for. The database-wide passes (buckets, external properties, indexes) and the orphaned-segment reclaim are skipped, since none can be narrowed to a record. Two costs it deliberately does not bound, documented rather than glossed: rebuilding an adjacency needs every surviving edge that points at the vertex and no index maps endpoints back to edges, so the scoped run saves the vertex passes but still scans the edge types; and a listed record that turns out to be corrupted still has every index on its bucket rebuilt. Also fixes a vacuously-passing assertion in Issue5670EdgeDeleteDanglingBackRefTest: it asserted totalErrors == 0, and check database reports no such field, so the null-tolerant reader turned it into 0 == 0 on every run whatever the check found. It now asserts on autoFix, invalidLinks, totalWarnings and totalCorruptedRecords, and fails loudly on a field that does not exist.
…med records only (#5710) * feat(engine) #5680: CHECK DATABASE RECORD <rid> checks and repairs named records only CHECK DATABASE could be narrowed to a TYPE or a BUCKET, but not to a record. That is the sharp edge of the strict vertex delete merged in #5707: a genuinely broken edge chain makes its vertex undeletable until CHECK DATABASE ... FIX rebuilds the adjacency from the surviving edge records, and the smallest way to ask for that repair was two full passes over the whole vertex type. CHECK DATABASE RECORD #12:3 FIX CHECK DATABASE RECORD #12:3, #12:9 FIX The per-record work and the fix are identical to the type-wide run, including the edge-list rebuild; only the enumeration changes. Combines with FIX and COMPRESS. Rejected when combined with TYPE or BUCKET - RECORD is already the narrowest scope, so letting one silently win would run a check nobody asked for and an intersection would be a third semantics nobody asked for. The database-wide passes (buckets, external properties, indexes) and the orphaned-segment reclaim are skipped, since none can be narrowed to a record. Two costs it deliberately does not bound, documented rather than glossed: rebuilding an adjacency needs every surviving edge that points at the vertex and no index maps endpoints back to edges, so the scoped run saves the vertex passes but still scans the edge types; and a listed record that turns out to be corrupted still has every index on its bucket rebuilt. Also fixes a vacuously-passing assertion in Issue5670EdgeDeleteDanglingBackRefTest: it asserted totalErrors == 0, and check database reports no such field, so the null-tolerant reader turned it into 0 == 0 on every run whatever the check found. It now asserts on autoFix, invalidLinks, totalWarnings and totalCorruptedRecords, and fails loudly on a field that does not exist. * style(engine) #5680: strip trailing whitespace flagged by pre-commit The setup job failed on a trailing-whitespace line an earlier edit left after a comment in the RECORD-scoped edge branch. Removed, along with the stray blank line beside it. * fix(engine) #5680: a missing RID is not corruption, and the scoped progress step stops double-counting Two independent review passes on the RECORD scope, four actionable points between them. - A listed RID that simply is not there was recorded through addCorrupted, which puts its BUCKET into affectedBuckets - so CHECK DATABASE RECORD ... FIX dropped and rebuilt every index on that bucket, a full bucket scan, over what is usually a mistyped or already-deleted RID. That is exactly the cost the scope exists to avoid, and the scope is meant to be hand-typed after a failed delete. RecordNotFoundException is now reported and never flagged corrupted, in all three arms (vertex, edge, document). Mutation-checked: restoring addCorrupted fails the new test with totalCorruptedRecords=1, autoFix=1. - checkScopedRecords bumped currentStep for every group, but the document arm reaches stepBegin, which bumps it again - so a scope containing a plain document reported "step 3 of 2" against a totalSteps budgeted at one step per group. Each arm now owns its bump; the graph arms need it because they pass a fixed value into the nested GraphDatabaseChecker instead. - setRecords and the statement's stream now drop null RIDs. Rid.toRecordId can answer null for a non-literal form, and groupRecordsByType would meet it as an NPE on getBucketId(). Not reachable through the RECORD #n:n literal grammar, but setRecords(Set<RID>) is public API. - The parser test's cross-reference named a test class that no longer exists after the rescope. Tests: four added for the branches that were unexercised - an edge-typed RID, a document-typed RID (the branch the progress bug lived in), several RIDs spanning different types in one command, and a missing-but-valid RID under FIX, which pins the first item above. * fix(engine) #5680: a RECORD scope that resolves to nothing is refused, not widened to the whole database Review round on the rescoped PR: LGTM with three minor points, two taken. - CHECK DATABASE RECORD whose RIDs all resolve to null left the checker with an empty scope, which reads as "no scope" - so a command that explicitly named records quietly ran a FULL database check. The review judged this unreachable from SQL; it is not. Only the `#n:n` literal always resolves, while the non-literal {"@Rid": <expression>} form goes through an expression that can answer null, so `CHECK DATABASE RECORD {"@Rid": null}` reached it. Refused now, with a test that fails when the guard is stubbed out. - A warning dropped for exceeding maxWarnings was discarded silently in the two scoped arms, where GraphDatabaseChecker.addWarning logs it. Extracted addScopedWarning so both behave the same: always counted, retained under the cap, logged when dropped. Not taken: extracting the duplicated RecordNotFoundException/Exception catch blocks in the scoped vertex and edge arms. They would need six accumulators threaded through a shared helper, which reads worse than the duplication, and the surrounding type-wide methods already parallel each other the same way - the review noted as much in raising it. * fix(engine) #5680: the empty-scope guard moves into setRecords, so it protects the public API too Review round: LGTM once the setRecords widening note was considered. Went one step further than the note it asked for. The guard against "a RECORD scope that resolves to nothing silently becomes a full database check" lived only in the SQL layer, so an embedded caller reaching DatabaseChecker.setRecords directly still hit the widening. The review reasoned that setRecords cannot tell "named records, none usable" from "named no records" - true only because the SQL layer filtered the nulls out BEFORE calling it. It no longer does: the nulls are passed through, setRecords owns both the filtering and the refusal, and one guard now covers both entry points. A genuinely empty (or null) set still means "no scope" and is accepted. Both are asserted. Also: the document arm said "cannot be loaded, removing it", but nothing removes the record there - corruptedRecords only drives the index rebuild. The review called the wording inherited and asked for no action; it is new code here, so it now says what is true, with a note on why it diverges from the type-wide checkDocuments (which also calls a document a "vertex"). Adds the warning-cap test the review asked for: 40 bogus RIDs against maxWarnings=10 retains 10 messages and still counts 40. Not added: an execution test for RECORD ... FIX COMPRESS. The compress pass is shared, unscoped code already covered elsewhere, and routing it through the scoped enumeration exercises no new branch. * docs(engine) #5680: state the COMPRESS caveat, and pin the corrupted-record fix cost with a test Review approved with three non-blocking notes; two taken. - COMPRESS is unaffected by the RECORD scope and still compresses the whole database. It is opt-in, but it is the one clause that breaks the "naming a record bounds the cost" promise, so it now sits in the release notes beside the other two costs the scope does not bound. - Added the test the review asked for: a listed record that is GENUINELY corrupted - not merely missing, not merely chain-broken - is flagged, and under FIX every index on its bucket is dropped and rebuilt. That turns the "RECORD bounds the check, not necessarily the fix" paragraph into an executable assertion, so a future change cannot quietly make a scoped repair either cheaper or more destructive than documented. Mutation-checked: dropping addCorrupted from the scoped arm fails it, and the injected corruption is the intended shape (buffer truncated to 8 bytes against the 25-byte vertex prefix), distinct from the missing-RID path. Not taken: moving the RECORD-plus-TYPE/BUCKET rejection from check() into the setters. setTypes and setRecords can be called in either order, so a setter-level guard would fire or not depending on call order; check() is the only place that sees the whole configuration. The review called the current placement acceptable. * fix(engine) #5680: a dropped RID is reported, the no-type step is named, and a comment stops overselling Review round: approve, all notes minor. Three taken. - The "Exception, not Throwable" comment claimed more protection than the code gives. It guards the LOOKUP, but the shared per-record consumer still catches Throwable around the connectivity check itself - matching the type-wide path, which is why it stays. The comment now says exactly what is and is not covered rather than implying an Error can never be recorded as corruption. - setRecords refused an all-null scope but dropped a PARTIAL one silently, so a caller who mistyped one of several RIDs got a clean report for a check that skipped it. Narrowing is still allowed - unlike widening it cannot do more than asked - but it now surfaces a warning naming how many were dropped, with a test. - The no-type group advanced currentStep without ever naming the step, so a progress poller watched it go by blank. Routed through stepBegin like the document arm, which also owns the bump. Not taken: - Rejecting RECORD together with COMPRESS for symmetry with the TYPE/BUCKET rejection. The two are not the same case: RECORD plus TYPE/BUCKET has no coherent meaning, whereas "check this record, then compress the database" is perfectly meaningful - it is just not cheap. That is a documentation problem, and the release note now states it. - @tag("slow") on the 200-edge tests. Measured rather than assumed: the whole class runs in 0.53s, and CLAUDE.md's criteria are multi-second elapsed or large batches. 200 tiny transactions under a second is neither. * fix(engine) #5680: diagnose the clause conflict first, warn that COMPRESS is unscoped Review round: refinements, not blockers. Four taken. - CHECK DATABASE TYPE Hub RECORD {"@Rid": null} reported "none of them resolves to a RID", because the RIDs are resolved before check() reaches the combination guard - pointing the operator at the inner failure while the actual mistake is the clause combination. The statement now diagnoses the combination before resolving anything; check() keeps its own guard for programmatic callers. - setRecords left droppedRecords stale on its early-return path. Harmless today, since the warning is only emitted under a record scope, but the field is public-API-visible state and should not depend on call order. - COMPRESS now warns that it is not limited by the RECORD scope. The review suggested this as a middle ground between refusing the combination and staying silent, and it is a better answer than the documentation-only one: the command still does what it was asked, but an operator who named a record and expected a bounded run is told otherwise. Tested both ways, so the warning tracks the clause rather than the scope. - Recorded, not fixed: the type-wide checkDocuments adds to the warnings/corrupted sets without incrementing the corresponding totals, so a corrupt document reports different totals depending on which path found it. The scoped arm here is the correct one; aligning the type-wide path is a behaviour change to a command this issue does not otherwise touch. Agreed with the review that the duplicated catch blocks read better than threading six accumulators through a shared helper; no change. * fix(engine) #5680: one owner for the conflict message, no counts a Set cannot know, bounded corrupted set Review round: nothing blocking. All four points taken. - The "cannot be combined with TYPE or BUCKET" text was written out in both the statement and the checker. The two-layer guard stays - the statement has to diagnose the clause conflict before the RIDs are resolved - but the wording now lives in one constant so the two cannot drift. - Two messages quoted a record count the Set could not know: several RID expressions that all resolve to null collapse to a single null element, so "given 3 record(s)" was never reachable and "1 of the record(s) did not resolve" under-reported. Rather than plumb the original count through, both messages stop quoting a number, which is accurate whatever the input. - checkScopedDocuments grew corruptedRecords without a bound while every other path honours maxCorrupted/maxWarnings. Bounded the same way, with the total still counting every occurrence. - Documented that the edge sweep a rebuild needs runs once per distinct vertex TYPE named, not once per run: ten vertices of one type cost one sweep, one vertex of each of three types cost three.
…airs it, and the conflict keeps its cause (#5767) * fix(engine) #5764: a refused vertex delete names the command that repairs it, and the conflict keeps its cause Follow-ups from #5680, whose two halves landed separately: #5707 made deleteVertex strict, #5710 added the CHECK DATABASE RECORD scope. Neither is defective; the seams between them were never closed. 1. The repair advice now names the scoped command with the RID substituted. Split in two, which the report did not distinguish: the RECORD scope is the right answer only where the delete was REFUSED and the vertex is still there to repair. The force arms log AFTER the record is gone, so a scoped check would name a record that no longer exists - they keep the whole-database form and say what the scoped form buys when run first. And at the other-end conflict the broken list belongs to the NEIGHBOUR, so that is the RID named. The previously bare rethrow now carries the advice instead of only getEdgeHeadChunkForWrite's "concurrent commit in flight", which is the diagnosis for the transient case nobody ever sees. 2. ConcurrentModificationException gains the (message, cause) constructor NeedRetryException already declared. Eight sites across GraphEngine, EdgeLinkedList and StripedEdgeList kept the message and dropped the stack. A conflict is normally absorbed by the retry, so the run that surfaces one is the retry-exhausted run - exactly the one whose trace has to be diagnosable. The class on the wire is unchanged. 3. checkDocuments and checkScopedDocuments report a corrupt document identically. The type-wide arm added to the warning and corrupted-record SETS without touching totalWarnings/totalCorruptedRecords, called a document a "vertex" it was "removing" when nothing on that path removes anything, and honoured no maxWarnings cap at all. Both now go through shared addWarning/addCorrupted. 4. The scoped vertex check running one pass where the type-wide runs two is NOT a gap: lookupByRID IS the raw-view materialisation (same newImmutableRecord(type, rid, bucket.getRecord(rid).copyOfContent())) and checkConnectivity opens with the same asVertex(true) that forces the decode, while getRecord and scan resolve placeholders and multi-page chains identically. Documented and pinned by a test rather than "fixed" by adding a second pass, which would double the cost of the scope that exists to be cheap. The advice tests execute what the message says: the command is parsed out of the raised exception and run verbatim, then the delete must succeed - so a wrong RID or a shape the parser rejects fails here. * review: state addCorrupted's dedup guarantee honestly, and drop the reflection in the advice test Three points from the code review on #5767, all minor and none blocking. 1. addCorrupted's "a record flagged twice is one corrupted record" holds only while the retained set is under the cap. Behaviour kept (the contains() check past the cap is strictly better than an unconditional increment); the Javadoc now says exactly what survives the cap and why closing the gap is not worth it - the only exact fix is a second unbounded set of every RID ever counted, i.e. the memory the cap exists to refuse. Also recorded that neither caller reaches it: the bucket scan visits each RID once and the RECORD scope is a Set, so a duplicate cannot survive as far as groupRecordsByType. 2. The reviewer flagged that the retained set is capped with maxWarnings while the graph arms take a separate maxCorrupted. Documented rather than changed: DatabaseChecker has no maxCorrupted field, and checkScopedRecords already passes the remaining maxWarnings budget into the GraphDatabaseChecker parameter of that name. Two names, one setting. 3. The advice test reached the LogManager logger by reflecting on the private field. LogManager exposes a public getLogger() for precisely this case - a caller that swaps the logger and has to put the original back - so the reflection helper is gone along with its rename fragility. * review: record the maxWarnings/maxCorrupted coupling at the field, and why the interpolated cause is not redundant Three non-blocking observations from the second review on #5767. All three resolve to recording constraints a future reader would otherwise have to rediscover - one of them by breaking a test. 1. maxWarnings caps the corrupted set while GraphDatabaseChecker takes a maxCorrupted for the SAME set. The method Javadoc said "two names, one setting"; the note now lives on the field, where a reader meets it, and adds the consequence the reviewer pointed out: retained corruptedRecords become affectedBuckets, so a record dropped past the cap does not get its bucket's indexes rebuilt under FIX. 2. The advice test swaps the process-wide LogManager logger. Restored in a finally and safe under sequential class execution, but it is shared state; documented as a latent constraint if class-level parallelism is ever enabled, along with why there is no per-invocation seam to capture through instead. 3. getEdgeHeadChunkForWrite and hasNextEdgeToDelete interpolate e.getMessage() AND keep e as the cause, which reads as redundant. It is not: the missing CHUNK's RID appears ONLY in the record-not-found message - the surrounding text names the vertex - and the top-level message is what a log line and an HTTP error body carry, while the cause chain surfaces only in a full trace. Removing it would drop the RID and fail Issue5670EdgeDeleteDanglingBackRefTest, which asserts on it. Commented at both sites. * review: guard the advice against a message-less conflict, and record why otherEndOf can equal the vertex Two of the three optional-polish points from the third review on #5767 (which was LGTM). 1. withRepairAdvice built the message as e.getMessage() + advice. Every conflict on this path carries a message today, but a message-less one would render "null. If it persists...", which reads as a bug in the advice rather than as a missing diagnosis. Falls back to toString(), i.e. the class name. 2. otherEndOf would answer vertexRID itself for a self-loop, which the method name reads as impossible. That is the correct answer - a self-loop has no other end and the list that failed to disconnect it IS the vertex's own - so the equality is documented rather than "simplified" away. Also recorded why there is no test: the caller runs only after both of the vertex's lists have been collected, and a self-loop is read out of those same two lists, so a list broken enough to fail the disconnection fails the collection first. Not taken: the reviewer flagged that otherEndOf is computed before the force branch, so it runs on every retry of a transient conflict. Both branches need the value, so moving it saves nothing - and the reviewer scoped it as "for completeness" on an exception path where asEdge() is light.
…instead of losing its edges (#5707) * fix(engine) #5680: an unreadable edge list retries the vertex delete instead of losing its edges deleteVertex collected the edges to remove through the best-effort getEdgeHeadChunk with the whole walk wrapped in catch (Exception), so any part of the list that could not be read at that instant was taken for "nothing to remove here" and the vertex record was deleted on top of that empty or partial view: the edges outlived their endpoint, with a back-reference left on an innocent neighbour. Three routes reached that outcome - an unreadable head chunk, an unreadable hop mid-chain (everything behind the hole dropped), and, on a promoted super-node, a stripe chain the read walk is allowed to skip, which lost a whole stripe with no exception behind it at all. None of them is a fact about the graph; they are the transient publication window #5670 was about, which deleteEdge has answered with a retry since #5678. The collection now reads the list the way a removal must: the head through getEdgeHeadChunkForWrite, the walk through the new EdgeLinkedList.edgeIteratorForRemoval (strict over stripe chains), and the chain hops mapped to a retryable ConcurrentModificationException. force is the single escape hatch and now works end to end: it also absorbs the conflict raised while disconnecting a collected edge from the vertex at its OTHER end, which previously let a broken neighbour block a forced delete. Draining the leftover chunk records stays best-effort in both modes on purpose - by then a miss loses orphaned chunks, never a reference. The heads are also read from the transaction's own vertex instance (#5660), closing a fourth silent-loss route that needed no race. Strictness costs no repairability: CHECK DATABASE ... FIX rebuilds an unloadable chain from the surviving edge records and is never blocked by the delete being blocked. * docs(engine) #5680: state why a chunk that cannot be DECODED stays tolerated, and where the striped conflict is raised Review follow-ups on #5707, both documentation. The blanket catch (Exception) left in collectEdgesToDelete is the residue once the transient window is split off into the retryable branch: an undecodable buffer, which no retry can fix and which is tolerated even when force is false. That reads as inconsistent with the strictness around it until you follow where such a record comes from - LocalDatabase.deleteRecordNoLock catches the same exception family around the index cleanup and proceeds WITHOUT raising its force flag, so a corrupt-buffer vertex arrives here with force == false and failing would make it undeletable again (#4420, #4432). Its cost is larger than a single tolerated dangling entry, so the warning now names it and points at CHECK DATABASE. hasNextEdgeToDelete's javadoc claimed only a chain hop surfaces through it, which is true on the classic layout but not the whole story on a promoted super-node: StripedEdgeList.edgeIteratorForRemoval resolves every stripe head while BUILDING the iterator, so an unloadable stripe raises there instead. Same try, same handling - the note keeps a future reader from reading the helper as the only strict point. * test(engine) #5680: pin the self-loop delete, and say what the strict/tolerant split actually keys on Second review follow-ups on #5707. A self-loop is the one shape both collection walks yield, so it lands in the delete list twice and the second removal runs against a chain the first already cleaned. That is unchanged by this fix and resolves quietly, but the removal walk is stricter now and a self-loop is the shape most likely to surprise a later change here, so it gets a test rather than an argument. The two catch arms in collectEdgesToDelete are easy to read as transient-versus-permanent, which is wrong and would make the next reader draw the line in the wrong place. They key on what a MISS COSTS: a chunk that cannot be found retries because deleting the vertex on a short list loses references, and that is applied to a genuinely lost chunk too - no retry recovers it, so the delete now fails once the retries are spent instead of quietly completing. Stated explicitly, since it is a deliberate behaviour change for the lost-chunk case and not only for the transient one. * fix(engine) #5680: close the tolerant catch to named corruption shapes, and scope the concurrent test to what this fix owns Third review follow-up on #5707, plus a flake the follow-up exposed. The trailing catch (Exception) in collectEdgesToDelete would have absorbed an NPE or an IllegalStateException from a future change as "corrupted chunk, delete anyway" - reintroducing exactly the silent-edge-loss path this issue exists to remove. It now names the shapes it tolerates: the decode family LocalDatabase already uses for the same purpose, plus ClassCastException and SchemaException, which are the two further shapes a corrupt EDGE LIST adds (a head RID naming a record that is not an edge segment, an edge bucket whose type is gone). Anything else propagates. While re-running, concurrentVertexDeletionAndEdgeAppendingLeaveNoEdgeBehind proved flaky (~1 in 3), and the cause is not this fix - it reproduces identically with the fix reverted. An edge an APPENDER commits while its hub is being deleted can outlive that hub: the survivor has an existing out vertex and an in naming the deleted hub, so it never existed when the deleter's collection walk ran and no vertex-side strictness can prevent it. That is a separate defect on the append path, reported separately. The test now asserts the invariant this fix actually owns - every edge that was IN the hub's list when the deleter started must be gone with it, i.e. the fixture edges - instead of "no LINK edge survives at all", which folded in the append-side defect. The round-start precondition moves from a global LINK count to a per-hub degree for the same reason, so a leftover from an earlier round cannot make a healthy round look wrong, and the end-of-run integrity check is dropped with the reason stated inline (the non-racing fixtures still assert it). 10 consecutive green runs. * docs(engine) #5680: record why IllegalArgumentException earns its place in the tolerated set Fourth review follow-up on #5707, comment only. IllegalArgumentException is the loosest member of the closed catch list and the one a future change is most likely to throw for a non-corruption reason. It stays, because Binary raises it ("Invalid position", Binary.java:179) for a content offset that decodes past the end of the buffer - a genuine corruption shape, and the reason LocalDatabase carries it in the same family - and dropping it would make a vertex with that damage undeletable again. Naming the origin means the periodic second look the review asks for can be done without re-deriving it, and narrowing it later is a decision with the evidence attached rather than a guess. (cherry picked from commit 324eca1)
…med records only (#5710) * feat(engine) #5680: CHECK DATABASE RECORD <rid> checks and repairs named records only CHECK DATABASE could be narrowed to a TYPE or a BUCKET, but not to a record. That is the sharp edge of the strict vertex delete merged in #5707: a genuinely broken edge chain makes its vertex undeletable until CHECK DATABASE ... FIX rebuilds the adjacency from the surviving edge records, and the smallest way to ask for that repair was two full passes over the whole vertex type. CHECK DATABASE RECORD #12:3 FIX CHECK DATABASE RECORD #12:3, #12:9 FIX The per-record work and the fix are identical to the type-wide run, including the edge-list rebuild; only the enumeration changes. Combines with FIX and COMPRESS. Rejected when combined with TYPE or BUCKET - RECORD is already the narrowest scope, so letting one silently win would run a check nobody asked for and an intersection would be a third semantics nobody asked for. The database-wide passes (buckets, external properties, indexes) and the orphaned-segment reclaim are skipped, since none can be narrowed to a record. Two costs it deliberately does not bound, documented rather than glossed: rebuilding an adjacency needs every surviving edge that points at the vertex and no index maps endpoints back to edges, so the scoped run saves the vertex passes but still scans the edge types; and a listed record that turns out to be corrupted still has every index on its bucket rebuilt. Also fixes a vacuously-passing assertion in Issue5670EdgeDeleteDanglingBackRefTest: it asserted totalErrors == 0, and check database reports no such field, so the null-tolerant reader turned it into 0 == 0 on every run whatever the check found. It now asserts on autoFix, invalidLinks, totalWarnings and totalCorruptedRecords, and fails loudly on a field that does not exist. * style(engine) #5680: strip trailing whitespace flagged by pre-commit The setup job failed on a trailing-whitespace line an earlier edit left after a comment in the RECORD-scoped edge branch. Removed, along with the stray blank line beside it. * fix(engine) #5680: a missing RID is not corruption, and the scoped progress step stops double-counting Two independent review passes on the RECORD scope, four actionable points between them. - A listed RID that simply is not there was recorded through addCorrupted, which puts its BUCKET into affectedBuckets - so CHECK DATABASE RECORD ... FIX dropped and rebuilt every index on that bucket, a full bucket scan, over what is usually a mistyped or already-deleted RID. That is exactly the cost the scope exists to avoid, and the scope is meant to be hand-typed after a failed delete. RecordNotFoundException is now reported and never flagged corrupted, in all three arms (vertex, edge, document). Mutation-checked: restoring addCorrupted fails the new test with totalCorruptedRecords=1, autoFix=1. - checkScopedRecords bumped currentStep for every group, but the document arm reaches stepBegin, which bumps it again - so a scope containing a plain document reported "step 3 of 2" against a totalSteps budgeted at one step per group. Each arm now owns its bump; the graph arms need it because they pass a fixed value into the nested GraphDatabaseChecker instead. - setRecords and the statement's stream now drop null RIDs. Rid.toRecordId can answer null for a non-literal form, and groupRecordsByType would meet it as an NPE on getBucketId(). Not reachable through the RECORD #n:n literal grammar, but setRecords(Set<RID>) is public API. - The parser test's cross-reference named a test class that no longer exists after the rescope. Tests: four added for the branches that were unexercised - an edge-typed RID, a document-typed RID (the branch the progress bug lived in), several RIDs spanning different types in one command, and a missing-but-valid RID under FIX, which pins the first item above. * fix(engine) #5680: a RECORD scope that resolves to nothing is refused, not widened to the whole database Review round on the rescoped PR: LGTM with three minor points, two taken. - CHECK DATABASE RECORD whose RIDs all resolve to null left the checker with an empty scope, which reads as "no scope" - so a command that explicitly named records quietly ran a FULL database check. The review judged this unreachable from SQL; it is not. Only the `#n:n` literal always resolves, while the non-literal {"@Rid": <expression>} form goes through an expression that can answer null, so `CHECK DATABASE RECORD {"@Rid": null}` reached it. Refused now, with a test that fails when the guard is stubbed out. - A warning dropped for exceeding maxWarnings was discarded silently in the two scoped arms, where GraphDatabaseChecker.addWarning logs it. Extracted addScopedWarning so both behave the same: always counted, retained under the cap, logged when dropped. Not taken: extracting the duplicated RecordNotFoundException/Exception catch blocks in the scoped vertex and edge arms. They would need six accumulators threaded through a shared helper, which reads worse than the duplication, and the surrounding type-wide methods already parallel each other the same way - the review noted as much in raising it. * fix(engine) #5680: the empty-scope guard moves into setRecords, so it protects the public API too Review round: LGTM once the setRecords widening note was considered. Went one step further than the note it asked for. The guard against "a RECORD scope that resolves to nothing silently becomes a full database check" lived only in the SQL layer, so an embedded caller reaching DatabaseChecker.setRecords directly still hit the widening. The review reasoned that setRecords cannot tell "named records, none usable" from "named no records" - true only because the SQL layer filtered the nulls out BEFORE calling it. It no longer does: the nulls are passed through, setRecords owns both the filtering and the refusal, and one guard now covers both entry points. A genuinely empty (or null) set still means "no scope" and is accepted. Both are asserted. Also: the document arm said "cannot be loaded, removing it", but nothing removes the record there - corruptedRecords only drives the index rebuild. The review called the wording inherited and asked for no action; it is new code here, so it now says what is true, with a note on why it diverges from the type-wide checkDocuments (which also calls a document a "vertex"). Adds the warning-cap test the review asked for: 40 bogus RIDs against maxWarnings=10 retains 10 messages and still counts 40. Not added: an execution test for RECORD ... FIX COMPRESS. The compress pass is shared, unscoped code already covered elsewhere, and routing it through the scoped enumeration exercises no new branch. * docs(engine) #5680: state the COMPRESS caveat, and pin the corrupted-record fix cost with a test Review approved with three non-blocking notes; two taken. - COMPRESS is unaffected by the RECORD scope and still compresses the whole database. It is opt-in, but it is the one clause that breaks the "naming a record bounds the cost" promise, so it now sits in the release notes beside the other two costs the scope does not bound. - Added the test the review asked for: a listed record that is GENUINELY corrupted - not merely missing, not merely chain-broken - is flagged, and under FIX every index on its bucket is dropped and rebuilt. That turns the "RECORD bounds the check, not necessarily the fix" paragraph into an executable assertion, so a future change cannot quietly make a scoped repair either cheaper or more destructive than documented. Mutation-checked: dropping addCorrupted from the scoped arm fails it, and the injected corruption is the intended shape (buffer truncated to 8 bytes against the 25-byte vertex prefix), distinct from the missing-RID path. Not taken: moving the RECORD-plus-TYPE/BUCKET rejection from check() into the setters. setTypes and setRecords can be called in either order, so a setter-level guard would fire or not depending on call order; check() is the only place that sees the whole configuration. The review called the current placement acceptable. * fix(engine) #5680: a dropped RID is reported, the no-type step is named, and a comment stops overselling Review round: approve, all notes minor. Three taken. - The "Exception, not Throwable" comment claimed more protection than the code gives. It guards the LOOKUP, but the shared per-record consumer still catches Throwable around the connectivity check itself - matching the type-wide path, which is why it stays. The comment now says exactly what is and is not covered rather than implying an Error can never be recorded as corruption. - setRecords refused an all-null scope but dropped a PARTIAL one silently, so a caller who mistyped one of several RIDs got a clean report for a check that skipped it. Narrowing is still allowed - unlike widening it cannot do more than asked - but it now surfaces a warning naming how many were dropped, with a test. - The no-type group advanced currentStep without ever naming the step, so a progress poller watched it go by blank. Routed through stepBegin like the document arm, which also owns the bump. Not taken: - Rejecting RECORD together with COMPRESS for symmetry with the TYPE/BUCKET rejection. The two are not the same case: RECORD plus TYPE/BUCKET has no coherent meaning, whereas "check this record, then compress the database" is perfectly meaningful - it is just not cheap. That is a documentation problem, and the release note now states it. - @tag("slow") on the 200-edge tests. Measured rather than assumed: the whole class runs in 0.53s, and CLAUDE.md's criteria are multi-second elapsed or large batches. 200 tiny transactions under a second is neither. * fix(engine) #5680: diagnose the clause conflict first, warn that COMPRESS is unscoped Review round: refinements, not blockers. Four taken. - CHECK DATABASE TYPE Hub RECORD {"@Rid": null} reported "none of them resolves to a RID", because the RIDs are resolved before check() reaches the combination guard - pointing the operator at the inner failure while the actual mistake is the clause combination. The statement now diagnoses the combination before resolving anything; check() keeps its own guard for programmatic callers. - setRecords left droppedRecords stale on its early-return path. Harmless today, since the warning is only emitted under a record scope, but the field is public-API-visible state and should not depend on call order. - COMPRESS now warns that it is not limited by the RECORD scope. The review suggested this as a middle ground between refusing the combination and staying silent, and it is a better answer than the documentation-only one: the command still does what it was asked, but an operator who named a record and expected a bounded run is told otherwise. Tested both ways, so the warning tracks the clause rather than the scope. - Recorded, not fixed: the type-wide checkDocuments adds to the warnings/corrupted sets without incrementing the corresponding totals, so a corrupt document reports different totals depending on which path found it. The scoped arm here is the correct one; aligning the type-wide path is a behaviour change to a command this issue does not otherwise touch. Agreed with the review that the duplicated catch blocks read better than threading six accumulators through a shared helper; no change. * fix(engine) #5680: one owner for the conflict message, no counts a Set cannot know, bounded corrupted set Review round: nothing blocking. All four points taken. - The "cannot be combined with TYPE or BUCKET" text was written out in both the statement and the checker. The two-layer guard stays - the statement has to diagnose the clause conflict before the RIDs are resolved - but the wording now lives in one constant so the two cannot drift. - Two messages quoted a record count the Set could not know: several RID expressions that all resolve to null collapse to a single null element, so "given 3 record(s)" was never reachable and "1 of the record(s) did not resolve" under-reported. Rather than plumb the original count through, both messages stop quoting a number, which is accurate whatever the input. - checkScopedDocuments grew corruptedRecords without a bound while every other path honours maxCorrupted/maxWarnings. Bounded the same way, with the total still counting every occurrence. - Documented that the edge sweep a rebuild needs runs once per distinct vertex TYPE named, not once per run: ten vertices of one type cost one sweep, one vertex of each of three types cost three. (cherry picked from commit 56b47f4)
…airs it, and the conflict keeps its cause (#5767) * fix(engine) #5764: a refused vertex delete names the command that repairs it, and the conflict keeps its cause Follow-ups from #5680, whose two halves landed separately: #5707 made deleteVertex strict, #5710 added the CHECK DATABASE RECORD scope. Neither is defective; the seams between them were never closed. 1. The repair advice now names the scoped command with the RID substituted. Split in two, which the report did not distinguish: the RECORD scope is the right answer only where the delete was REFUSED and the vertex is still there to repair. The force arms log AFTER the record is gone, so a scoped check would name a record that no longer exists - they keep the whole-database form and say what the scoped form buys when run first. And at the other-end conflict the broken list belongs to the NEIGHBOUR, so that is the RID named. The previously bare rethrow now carries the advice instead of only getEdgeHeadChunkForWrite's "concurrent commit in flight", which is the diagnosis for the transient case nobody ever sees. 2. ConcurrentModificationException gains the (message, cause) constructor NeedRetryException already declared. Eight sites across GraphEngine, EdgeLinkedList and StripedEdgeList kept the message and dropped the stack. A conflict is normally absorbed by the retry, so the run that surfaces one is the retry-exhausted run - exactly the one whose trace has to be diagnosable. The class on the wire is unchanged. 3. checkDocuments and checkScopedDocuments report a corrupt document identically. The type-wide arm added to the warning and corrupted-record SETS without touching totalWarnings/totalCorruptedRecords, called a document a "vertex" it was "removing" when nothing on that path removes anything, and honoured no maxWarnings cap at all. Both now go through shared addWarning/addCorrupted. 4. The scoped vertex check running one pass where the type-wide runs two is NOT a gap: lookupByRID IS the raw-view materialisation (same newImmutableRecord(type, rid, bucket.getRecord(rid).copyOfContent())) and checkConnectivity opens with the same asVertex(true) that forces the decode, while getRecord and scan resolve placeholders and multi-page chains identically. Documented and pinned by a test rather than "fixed" by adding a second pass, which would double the cost of the scope that exists to be cheap. The advice tests execute what the message says: the command is parsed out of the raised exception and run verbatim, then the delete must succeed - so a wrong RID or a shape the parser rejects fails here. * review: state addCorrupted's dedup guarantee honestly, and drop the reflection in the advice test Three points from the code review on #5767, all minor and none blocking. 1. addCorrupted's "a record flagged twice is one corrupted record" holds only while the retained set is under the cap. Behaviour kept (the contains() check past the cap is strictly better than an unconditional increment); the Javadoc now says exactly what survives the cap and why closing the gap is not worth it - the only exact fix is a second unbounded set of every RID ever counted, i.e. the memory the cap exists to refuse. Also recorded that neither caller reaches it: the bucket scan visits each RID once and the RECORD scope is a Set, so a duplicate cannot survive as far as groupRecordsByType. 2. The reviewer flagged that the retained set is capped with maxWarnings while the graph arms take a separate maxCorrupted. Documented rather than changed: DatabaseChecker has no maxCorrupted field, and checkScopedRecords already passes the remaining maxWarnings budget into the GraphDatabaseChecker parameter of that name. Two names, one setting. 3. The advice test reached the LogManager logger by reflecting on the private field. LogManager exposes a public getLogger() for precisely this case - a caller that swaps the logger and has to put the original back - so the reflection helper is gone along with its rename fragility. * review: record the maxWarnings/maxCorrupted coupling at the field, and why the interpolated cause is not redundant Three non-blocking observations from the second review on #5767. All three resolve to recording constraints a future reader would otherwise have to rediscover - one of them by breaking a test. 1. maxWarnings caps the corrupted set while GraphDatabaseChecker takes a maxCorrupted for the SAME set. The method Javadoc said "two names, one setting"; the note now lives on the field, where a reader meets it, and adds the consequence the reviewer pointed out: retained corruptedRecords become affectedBuckets, so a record dropped past the cap does not get its bucket's indexes rebuilt under FIX. 2. The advice test swaps the process-wide LogManager logger. Restored in a finally and safe under sequential class execution, but it is shared state; documented as a latent constraint if class-level parallelism is ever enabled, along with why there is no per-invocation seam to capture through instead. 3. getEdgeHeadChunkForWrite and hasNextEdgeToDelete interpolate e.getMessage() AND keep e as the cause, which reads as redundant. It is not: the missing CHUNK's RID appears ONLY in the record-not-found message - the surrounding text names the vertex - and the top-level message is what a log line and an HTTP error body carry, while the cause chain surfaces only in a full trace. Removing it would drop the RID and fail Issue5670EdgeDeleteDanglingBackRefTest, which asserts on it. Commented at both sites. * review: guard the advice against a message-less conflict, and record why otherEndOf can equal the vertex Two of the three optional-polish points from the third review on #5767 (which was LGTM). 1. withRepairAdvice built the message as e.getMessage() + advice. Every conflict on this path carries a message today, but a message-less one would render "null. If it persists...", which reads as a bug in the advice rather than as a missing diagnosis. Falls back to toString(), i.e. the class name. 2. otherEndOf would answer vertexRID itself for a self-loop, which the method name reads as impossible. That is the correct answer - a self-loop has no other end and the list that failed to disconnect it IS the vertex's own - so the equality is documented rather than "simplified" away. Also recorded why there is no test: the caller runs only after both of the vertex's lists have been collected, and a self-loop is read out of those same two lists, so a list broken enough to fail the disconnection fails the collection first. Not taken: the reviewer flagged that otherEndOf is computed before the force branch, so it runs on every retry of a transient conflict. Both branches need the value, so moving it saves nothing - and the reviewer scoped it as "for completeness" on an exception path where asEdge() is light. (cherry picked from commit c7349f8)
Fixes #5680.
The bug is real, and it is wider than the issue describes
GraphEngine.deleteVertexcollects the edges to remove by walking the vertex's own OUT and IN lists, then deleteseach collected edge, then deletes the vertex record. The collection read the head through the best-effort
getEdgeHeadChunkand wrapped the whole walk incatch (Exception), so any part of the list that could not beread at that instant was taken for "nothing to remove here" and the vertex record was deleted on top of that empty
or partial view. Reproduced deterministically on
main-A --E--> BwithA's OUT head chunk removed:Three distinct routes reach that outcome, not one. The issue named the first:
getEdgeHeadChunkanswersnull, nothing is collected.Iterator.hasNext()raisesRecordNotFoundExceptionout of the walk loop intothe blanket catch. Everything behind the hole is dropped; everything already seen is kept, so the delete
proceeds on a partial view that looks complete.
exception behind it at all.
StripedEdgeList.edgeIteratorcomposes one iterator per stripe chain throughallChains(false), and a chain whose head cannot be read is legitimately skipped there: that is thedocumented best-effort READ contract. Used as a removal walk it silently lost a whole stripe's worth of edges.
None of these is a fact about the graph. They are facts about timing - the same window #5670 was about: a commit
publishes its pages one at a time and the reader takes no commit lock, so a vertex page can expose a head RID
before that head's own page is visible, and a chunk emptied by another transaction is relinked out from under a
walker.
deleteEdgehas answered that window with a retry since #5678; the vertex path never did.A fourth, unrelated to concurrency, was found while writing the fix: the collection read the edge-list heads off
the caller's vertex handle rather than the instance the transaction holds (#5660). A handle obtained before an
append in the same transaction still names the previous head, so the newest edges were invisible to the walk - and
the vertex was deleted anyway. Same defect, no race required.
The fix
Strict by default,
forceas the single escape hatch, andforcemade to work end to end.deleteVertex(v, force=false)- the collection reads the list the way a removal must: the head through thestrict
getEdgeHeadChunkForWrite, the walk through a newEdgeLinkedList.edgeIteratorForRemoval()(identical toedgeIterator()on the classic layout; onStripedEdgeListit composesallChains(**true**)so a stripe chaincannot be skipped), and the chain hops through a helper that maps an unreadable hop to a retryable
ConcurrentModificationException. Thecatch (Exception)no longer swallowsNeedRetryException.deleteVertex(v, force=true)- every one of those conflicts is absorbed and logged, including the one raisedwhile disconnecting a collected edge from the vertex at its OTHER end. That last one is the gap recorded in the
issue's second comment: since fix(engine) #5670: an unreadable edge-list chunk retries the delete instead of skipping it #5678 made the neighbour's removal strict, a broken neighbour blocked a forced
delete exactly as it blocked an ordinary one - precisely what
forceexists to override.getMostUpdatedVertex(vertex), closing the fourth route above.One line is deliberately not strict, in either mode: draining the leftover chunk records at the end. By then
every collected edge has been disconnected from both endpoints and the vertex record is about to go, so a chunk that
cannot be read there costs orphaned chunk records - garbage
CHECK DATABASEreclaims - never a surviving reference.The rule the fix follows is strict where a miss loses a reference, tolerant where a miss loses only garbage, which
is finer than the blanket "strict when
force == false" the issue proposed.Also left tolerant, in both modes: a single entry whose edge record cannot be resolved. It is skipped, the walk
keeps every other entry, so the cost is that one already-dangling pointer rather than the whole remaining list -
the same way
EdgeIteratorFilterandCHECK DATABASEtreat such an entry. Escalating it would make every vertexcarrying one pre-existing dangling pointer undeletable, which is a strictly larger change than this window
justifies.
Why strictness does not cost repairability (the trade-off the issue asked to decide)
The issue framed the open question as "what should an ordinary
deleteVertexdo when the chain is genuinelybroken", since a transient and a lost chunk are indistinguishable at that moment. Two things settled it:
forceis not the operator's answer, and never was. It is not reachable from SQL or the API: it is setinternally by
LocalDatabase.deleteRecordNoLockonly whenbucket.isChunkChainBrokenconfirms the record's ownmulti-page chain is broken. A broken edge list never sets it. So "keep tolerance under
force" gives usersnothing on its own - it only fixes the internal forced-delete path (which this PR does fix, see above).
CHECK DATABASE ... FIXalready is the answer, and it is reachable. It rebuilds an unloadable chain from thesurviving edge records - head chunk, mid-chain, classic and striped - and is pinned by
GraphDatabaseCheckerChainRebuildTest. It reads edge lists through the best-effort reader, so it is never blockedby the delete being blocked. After the repair the delete goes through normally.
So a genuinely broken graph is still cleanable without
force; it just takes the repair command first, and thedelete stops silently manufacturing ghost edges in the meantime.
Alternatives considered and rejected
LocalBucket.isChunkChainBrokendoes for the record chain. It does nottransfer. There, the
ConcurrentModificationExceptioncomes from page-version validation, so a version-blindstructural walk genuinely tells contention from breakage. Here the failure is "record not found", which reads
identically for a chunk that is invisible-for-now and one that is lost; a re-probe returns the same answer.
final one). Attractive - it keeps a broken graph deletable without any repair step - but it makes correctness
depend on
arcadedb.txRetries, gives an explicitbegin/commitcaller no retries at all (attempt 0 is the lastone, so it is exactly today's behaviour), and quietly reintroduces the ghost-edge outcome under a new name.
flight, an unreadable chunk is genuinely lost. Rejected as deadlock-prone - acquiring a file commit lock outside
commit ordering, on the delete path - for a case the repair command already covers.
Visible behaviour change
vertex.delete()/DELETE VERTEXcan now raise aConcurrentModificationExceptionwhere it previously"succeeded". That is a
NeedRetryException, sodatabase.transaction(...)and the server's auto-retry forsingle-request commands absorb it; a client-managed explicit transaction spanning several requests sees it and
should retry, the same contract concurrent updates have always had. This is the vertex-side half of the change
#5678 already documented for the neighbour side. Noted in the release notes.
Tests
Issue5680VertexDeleteEdgeCollectionTest- 8 tests; the 3 that pin the defect fail on currentmain, all 8 passwith the fix. The other 5 are the controls that keep the fix honest.
maindeletingAVertexWhoseOwnHeadChunkIsUnreadableReportsAConflictRatherThanLosingItsEdgesdeletingAVertexWhoseChainIsBrokenMidWayReportsAConflictRatherThanLosingTheEdgesBehindTheHoledeletingAPromotedSuperNodeWhoseStripeChainIsUnreadableReportsAConflictaForcedDeleteIsNotBlockedByAnUnreadableNeighbourEdgeListforcegap from comment 2aForcedDeleteStillRemovesAVertexWhoseOwnEdgeListIsUnreadableforcetolerance keptaForcedDeleteOfAHealthyVertexStillDisconnectsItsNeighboursforceis not "skip the work"anOrdinaryDeleteOfAHealthyVertexRemovesEveryEdgeOnBothSidesconcurrentVertexDeletionAndEdgeAppendingLeaveNoEdgeBehind@Tag("slow"); the production shapeEach deterministic test asserts its preconditions before breaking anything (the degree is right, the chain really
spans several chunks, the hub really was promoted to the striped layout), so none can pass on an unrelated throw.
The super-node one is worth calling out because the obvious version of it is vacuous: an end-to-end
DELETE VERTEXon that fixture raisesConcurrentModificationExceptionwith or without the fix, because theper-edge disconnection reaches the same broken chain through
chainsForNeighbour, which has always been strict. Soit asserts the two walks over the same broken list side by side instead - the READ one still under-reports the
degree in silence (unchanged, deliberately), the REMOVAL one raises naming the broken chunk RID - and only then the
end-to-end delete. Verified by flipping
allChains(true)back tofalse: the test fails.concurrentVertexDeletionAndEdgeAppendingLeaveNoEdgeBehindasserts the invariant rather than reproducing the race:every edge in the fixture points at a hub, so once every hub is deleted no edge record may survive. It also checks
that the appends really did land against hubs being deleted (otherwise the rounds only measured a sequential
delete) and that the fixture was non-empty before each round, so the invariants cannot hold vacuously.
Verification
Issue5680VertexDeleteEdgeCollectionTest: 8/8, three consecutive runs.enginesuite (benchmarks excluded): see below.10743 tests, 0 failures, 0 errors, 23 skipped (21:49 min).
servermodule suite (benchmarks excluded): 950 tests, 0 failures, 0 errors.