Skip to content

fix(engine) #5680: an unreadable edge list retries the vertex delete instead of losing its edges - #5707

Merged
lvca merged 6 commits into
mainfrom
issue-5680-deletevertex-strict-collection
Aug 1, 2026
Merged

lvca merged 6 commits into
mainfrom
issue-5680-deletevertex-strict-collection

Conversation

@lvca

@lvca lvca commented Aug 1, 2026 •

Copy link
Copy Markdown
Member

Fixes #5680.

The bug is real, and it is wider than the issue describes

GraphEngine.deleteVertex collects the edges to remove by walking the vertex's own OUT and IN lists, then deletes
each collected edge, then deletes the vertex record. The collection read the head through the best-effort
getEdgeHeadChunk and wrapped the whole walk in catch (Exception), so any part of the list that could not be
read at that instant was taken for "nothing to remove here" and the vertex record was deleted on top of that empty
or partial view. Reproduced deterministically on main - A --E--> B with A's OUT head chunk removed:

DELETE VERTEX with own unreadable OUT list SUCCEEDED
vertex A still exists = false
EDGE record still exists = true      <-- edge outlived its OUT vertex
B in-degree           = 1            <-- B still points at it

Three distinct routes reach that outcome, not one. The issue named the first:

  1. Unreadable head chunk - getEdgeHeadChunk answers null, nothing is collected.
  2. Unreadable hop mid-chain - Iterator.hasNext() raises RecordNotFoundException out of the walk loop into
    the blanket catch. Everything behind the hole is dropped; everything already seen is kept, so the delete
    proceeds on a partial view that looks complete.
  3. A skipped stripe chain on a promoted super-node (Adaptive striped/sharded edge list for super-node write throughput (#5148 follow-up) #5156) - the widest of the three and the only one with no
    exception behind it at all. StripedEdgeList.edgeIterator composes one iterator per stripe chain through
    allChains(false), and a chain whose head cannot be read is legitimately skipped there: that is the
    documented best-effort READ contract. Used as a removal walk it silently lost a whole stripe's worth of edges.

None of these is a fact about the graph. They are facts about timing - the same window #5670 was about: a commit
publishes its pages one at a time and the reader takes no commit lock, so a vertex page can expose a head RID
before that head's own page is visible, and a chunk emptied by another transaction is relinked out from under a
walker. deleteEdge has answered that window with a retry since #5678; the vertex path never did.

A fourth, unrelated to concurrency, was found while writing the fix: the collection read the edge-list heads off
the caller's vertex handle rather than the instance the transaction holds (#5660). A handle obtained before an
append in the same transaction still names the previous head, so the newest edges were invisible to the walk - and
the vertex was deleted anyway. Same defect, no race required.

The fix

Strict by default, force as the single escape hatch, and force made to work end to end.

  • deleteVertex(v, force=false) - the collection reads the list the way a removal must: the head through the
    strict getEdgeHeadChunkForWrite, the walk through a new EdgeLinkedList.edgeIteratorForRemoval() (identical to
    edgeIterator() on the classic layout; on StripedEdgeList it composes allChains(**true**) so a stripe chain
    cannot be skipped), and the chain hops through a helper that maps an unreadable hop to a retryable
    ConcurrentModificationException. The catch (Exception) no longer swallows NeedRetryException.
  • deleteVertex(v, force=true) - every one of those conflicts is absorbed and logged, including the one raised
    while disconnecting a collected edge from the vertex at its OTHER end
    . That last one is the gap recorded in the
    issue's second comment: since fix(engine) #5670: an unreadable edge-list chunk retries the delete instead of skipping it #5678 made the neighbour's removal strict, a broken neighbour blocked a forced
    delete exactly as it blocked an ordinary one - precisely what force exists to override.
  • The heads are read from getMostUpdatedVertex(vertex), closing the fourth route above.

One line is deliberately not strict, in either mode: draining the leftover chunk records at the end. By then
every collected edge has been disconnected from both endpoints and the vertex record is about to go, so a chunk that
cannot be read there costs orphaned chunk records - garbage CHECK DATABASE reclaims - never a surviving reference.
The rule the fix follows is strict where a miss loses a reference, tolerant where a miss loses only garbage, which
is finer than the blanket "strict when force == false" the issue proposed.

Also left tolerant, in both modes: a single entry whose edge record cannot be resolved. It is skipped, the walk
keeps every other entry, so the cost is that one already-dangling pointer rather than the whole remaining list -
the same way EdgeIteratorFilter and CHECK DATABASE treat such an entry. Escalating it would make every vertex
carrying one pre-existing dangling pointer undeletable, which is a strictly larger change than this window
justifies.

Why strictness does not cost repairability (the trade-off the issue asked to decide)

The issue framed the open question as "what should an ordinary deleteVertex do when the chain is genuinely
broken", since a transient and a lost chunk are indistinguishable at that moment. Two things settled it:

  • force is not the operator's answer, and never was. It is not reachable from SQL or the API: it is set
    internally by LocalDatabase.deleteRecordNoLock only when bucket.isChunkChainBroken confirms the record's own
    multi-page chain is broken. A broken edge list never sets it. So "keep tolerance under force" gives users
    nothing on its own - it only fixes the internal forced-delete path (which this PR does fix, see above).
  • CHECK DATABASE ... FIX already is the answer, and it is reachable. It rebuilds an unloadable chain from the
    surviving edge records - head chunk, mid-chain, classic and striped - and is pinned by
    GraphDatabaseCheckerChainRebuildTest. It reads edge lists through the best-effort reader, so it is never blocked
    by the delete being blocked. After the repair the delete goes through normally.

So a genuinely broken graph is still cleanable without force; it just takes the repair command first, and the
delete stops silently manufacturing ghost edges in the meantime.

Alternatives considered and rejected

  • Disambiguate structurally, the way LocalBucket.isChunkChainBroken does for the record chain. It does not
    transfer. There, the ConcurrentModificationException comes from page-version validation, so a version-blind
    structural walk genuinely tells contention from breakage. Here the failure is "record not found", which reads
    identically for a chunk that is invisible-for-now and one that is lost; a re-probe returns the same answer.
  • Make the last retry attempt tolerant (expose the attempt number on the transaction, go best-effort on the
    final one). Attractive - it keeps a broken graph deletable without any repair step - but it makes correctness
    depend on arcadedb.txRetries, gives an explicit begin/commit caller no retries at all (attempt 0 is the last
    one, so it is exactly today's behaviour), and quietly reintroduces the ghost-edge outcome under a new name.
  • Confirm by taking the file commit lock and re-reading. This is a true disambiguation: with no commit in
    flight, an unreadable chunk is genuinely lost. Rejected as deadlock-prone - acquiring a file commit lock outside
    commit ordering, on the delete path - for a case the repair command already covers.

Visible behaviour change

vertex.delete() / DELETE VERTEX can now raise a ConcurrentModificationException where it previously
"succeeded". That is a NeedRetryException, so database.transaction(...) and the server's auto-retry for
single-request commands absorb it; a client-managed explicit transaction spanning several requests sees it and
should retry, the same contract concurrent updates have always had. This is the vertex-side half of the change
#5678 already documented for the neighbour side. Noted in the release notes.

Tests

Issue5680VertexDeleteEdgeCollectionTest - 8 tests; the 3 that pin the defect fail on current main, all 8 pass
with the fix.
The other 5 are the controls that keep the fix honest.

Test Shape On main
deletingAVertexWhoseOwnHeadChunkIsUnreadableReportsAConflictRatherThanLosingItsEdges route 1, deterministic fails
deletingAVertexWhoseChainIsBrokenMidWayReportsAConflictRatherThanLosingTheEdgesBehindTheHole route 2, deterministic fails
deletingAPromotedSuperNodeWhoseStripeChainIsUnreadableReportsAConflict route 3, deterministic passes for the wrong reason (see below)
aForcedDeleteIsNotBlockedByAnUnreadableNeighbourEdgeList the force gap from comment 2 fails
aForcedDeleteStillRemovesAVertexWhoseOwnEdgeListIsUnreadable control: force tolerance kept passes
aForcedDeleteOfAHealthyVertexStillDisconnectsItsNeighbours control: force is not "skip the work" passes
anOrdinaryDeleteOfAHealthyVertexRemovesEveryEdgeOnBothSides control: intact graph untouched passes
concurrentVertexDeletionAndEdgeAppendingLeaveNoEdgeBehind @Tag("slow"); the production shape passes

Each deterministic test asserts its preconditions before breaking anything (the degree is right, the chain really
spans several chunks, the hub really was promoted to the striped layout), so none can pass on an unrelated throw.

The super-node one is worth calling out because the obvious version of it is vacuous: an end-to-end
DELETE VERTEX on that fixture raises ConcurrentModificationException with or without the fix, because the
per-edge disconnection reaches the same broken chain through chainsForNeighbour, which has always been strict. So
it asserts the two walks over the same broken list side by side instead - the READ one still under-reports the
degree in silence (unchanged, deliberately), the REMOVAL one raises naming the broken chunk RID - and only then the
end-to-end delete. Verified by flipping allChains(true) back to false: the test fails.

concurrentVertexDeletionAndEdgeAppendingLeaveNoEdgeBehind asserts the invariant rather than reproducing the race:
every edge in the fixture points at a hub, so once every hub is deleted no edge record may survive. It also checks
that the appends really did land against hubs being deleted (otherwise the rounds only measured a sequential
delete) and that the fixture was non-empty before each round, so the invariants cannot hold vacuously.

Verification

  • Issue5680VertexDeleteEdgeCollectionTest: 8/8, three consecutive runs.
  • Full engine suite (benchmarks excluded): see below.
    10743 tests, 0 failures, 0 errors, 23 skipped (21:49 min).
  • server module suite (benchmarks excluded): 950 tests, 0 failures, 0 errors.

…instead of losing its edges

deleteVertex collected the edges to remove through the best-effort getEdgeHeadChunk with the whole
walk wrapped in catch (Exception), so any part of the list that could not be read at that instant
was taken for "nothing to remove here" and the vertex record was deleted on top of that empty or
partial view: the edges outlived their endpoint, with a back-reference left on an innocent
neighbour. Three routes reached that outcome - an unreadable head chunk, an unreadable hop mid-chain
(everything behind the hole dropped), and, on a promoted super-node, a stripe chain the read walk is
allowed to skip, which lost a whole stripe with no exception behind it at all. None of them is a
fact about the graph; they are the transient publication window #5670 was about, which deleteEdge
has answered with a retry since #5678.

The collection now reads the list the way a removal must: the head through getEdgeHeadChunkForWrite,
the walk through the new EdgeLinkedList.edgeIteratorForRemoval (strict over stripe chains), and the
chain hops mapped to a retryable ConcurrentModificationException. force is the single escape hatch
and now works end to end: it also absorbs the conflict raised while disconnecting a collected edge
from the vertex at its OTHER end, which previously let a broken neighbour block a forced delete.
Draining the leftover chunk records stays best-effort in both modes on purpose - by then a miss
loses orphaned chunks, never a reference. The heads are also read from the transaction's own vertex
instance (#5660), closing a fourth silent-loss route that needed no race.

Strictness costs no repairability: CHECK DATABASE ... FIX rebuilds an unloadable chain from the
surviving edge records and is never blocked by the delete being blocked.
@codacy-production

Copy link
Copy Markdown

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

🟢 Metrics 0 complexity

Metric Results
Complexity 0

View in Codacy

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

@mergify

mergify Bot commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

Tick the box to add this pull request to the merge queue (same as @mergifyio queue).

  • Queue this pull request

@claude

claude Bot commented Aug 1, 2026

Copy link
Copy Markdown

Review: issue 5680 strict edge collection on deleteVertex

I reviewed the full diff (GraphEngine, EdgeLinkedList, StripedEdgeList, the new test, release notes) plus the surrounding call sites (LocalDatabase.deleteRecordNoLock, getEdgeHeadChunkForWrite, allChains, chainsForNeighbour). This is a well-reasoned fix for a genuine silent data-loss bug, and it slots cleanly into the retry contract established by issues 5670/5678. Overall: strong, ship-worthy. Notes below are mostly confirmations plus a couple of small observations.

What is correct and worth calling out

  • Retryability is real, not accidental. hasNextEdgeToDelete and getEdgeHeadChunkForWrite throw com.arcadedb.exception.ConcurrentModificationException (verified the import at GraphEngine.java:31), which extends NeedRetryException, so database.transaction(...) and the server auto-retry absorb it. The catch ordering in collectEdgesToDelete (specific NeedRetryException before the blanket catch (Exception)) is right, since CME is a subtype.
  • The force plumbing lines up end to end. LocalDatabase.deleteRecordNoLock only sets force=true after bucket.isChunkChainBroken(record) confirms the record body chain is broken (LocalBucket.java:1750), never for a broken edge list. So an unreadable edge list correctly rethrows to retry on the ordinary path, and the forced path now also absorbs the neighbour-side conflict (GraphEngine.java:662-671) that issue 5678 had made strict. aForcedDeleteIsNotBlockedByAnUnreadableNeighbourEdgeList pins exactly that gap.
  • The 5660 fix (getMostUpdatedVertex) is a good catch and independent of concurrency: reading heads off a stale handle after an in-transaction append is the same collect-nothing-delete-anyway defect without a race.
  • Striped strictness is consistent. edgeIteratorForRemoval() to allChains(true) matches the always-strict rationale already documented on chainsForNeighbour (StripedEdgeList.java:453-457). The base EdgeLinkedList.edgeIteratorForRemoval() delegating to edgeIterator() keeps classic-layout behavior identical, so the change is scoped precisely to where a skipped chain would lose references.
  • Test coverage is excellent: deterministic reproductions for all three routes (fail on main), the two force controls, an intact-graph control, and the @tag("slow") concurrent invariant test that guards against the strict read wedging a real workload. Preconditions are asserted before corrupting the fixture, so none can pass on an unrelated throw.

Observations (non-blocking)

  1. Residual silent-loss path under hard corruption. In collectEdgesToDelete, the blanket catch (Exception) (GraphEngine.java:723-728) still swallows a non-RecordNotFoundException / non-NeedRetryException thrown mid-walk (e.g. a SerializationException/BufferUnderflowException from a genuinely corrupted chunk body) and proceeds to delete the vertex on a partial view - the very shape this PR fights, just for hard corruption rather than the transient window. Not a regression (the old code swallowed everything, including the transient case), and arguably the intended tolerant-where-the-miss-is-a-pre-existing-dangling-entry boundary. But unlike the deliberately-tolerated single dangling edge record, a corrupt chunk here drops everything behind it silently even with force=false. Might be worth a one-line comment on that catch (Exception) distinguishing transient (strict, handled above) from structural corruption (still tolerated here on purpose), so a future reader does not mistake it for an oversight.

  2. Classic vs striped: where the CME originates. On the classic layout the conflict surfaces from hasNextEdgeToDelete mid-walk; on the striped layout edgeIteratorForRemoval() builds allChains(true) eagerly, so the CME is thrown at iterator construction. Both are inside the try in collectEdgesToDelete, so both are handled identically - just noting the asymmetry, since the hasNextEdgeToDelete javadoc (Only the CHAIN HOP surfaces here) is accurate for classic but the striped head-chunk conflict never reaches that helper. Purely documentary.

  3. deleteRemainingChunks reloads via best-effort getEdgeHeadChunk(mostUpdatedVertex, ...). Correct and intentionally tolerant (leftovers are reclaimed by CHECK DATABASE), and using mostUpdatedVertex is if anything fresher than the old code using the original handle. No action needed.

Style / conventions

  • Matches the surrounding code (final params, brace-less single-statement if, text blocks for log messages). No System.out, no new dependencies, Claude not credited as author. Consistent with CLAUDE.md.
  • I could not run mvn in this sandbox to independently reproduce the build, but the change compiles logically (only StripedEdgeList overrides EdgeLinkedList, so the new method needs no other implementors) and the PR reports the full engine suite (10743 tests) and server suite green.

Nice work - the trade-off analysis in the description (why strictness does not cost repairability, and the three rejected alternatives) made this easy to review with confidence.

…lerated, and where the striped conflict is raised

Review follow-ups on #5707, both documentation.

The blanket catch (Exception) left in collectEdgesToDelete is the residue once the transient window is
split off into the retryable branch: an undecodable buffer, which no retry can fix and which is
tolerated even when force is false. That reads as inconsistent with the strictness around it until you
follow where such a record comes from - LocalDatabase.deleteRecordNoLock catches the same exception
family around the index cleanup and proceeds WITHOUT raising its force flag, so a corrupt-buffer vertex
arrives here with force == false and failing would make it undeletable again (#4420, #4432). Its cost is
larger than a single tolerated dangling entry, so the warning now names it and points at CHECK DATABASE.

hasNextEdgeToDelete's javadoc claimed only a chain hop surfaces through it, which is true on the classic
layout but not the whole story on a promoted super-node: StripedEdgeList.edgeIteratorForRemoval resolves
every stripe head while BUILDING the iterator, so an unloadable stripe raises there instead. Same try,
same handling - the note keeps a future reader from reading the helper as the only strict point.
@lvca

lvca commented Aug 1, 2026

Copy link
Copy Markdown
Member Author

Thanks - addressed both documentary observations in a95aa6f. Notes on each:

1. Residual silent-loss path under hard corruption. Correct reading, and I dug into whether it should become strict rather than documented. It should not, and the reason is concrete: LocalDatabase.deleteRecordNoLock catches exactly that exception family (SerializationException | NegativeArraySizeException | BufferUnderflowException | IndexOutOfBoundsException | IllegalArgumentException) around the index cleanup and proceeds without setting forceBrokenChainDelete - it raises that flag only in the ConcurrentModificationException branch, after isChunkChainBroken confirms a broken chunk chain. So a vertex whose buffer is corrupt reaches deleteVertex with force == false, and failing there would make it undeletable again - the "records that cannot be deleted" complaint #4420 and #4432 fixed. force is not an escape hatch for this shape because nothing sets it for this shape.

So the boundary stays, but you are right that its cost is larger than the tolerated single dangling entry (everything behind the damage is dropped, and the vertex still goes). The catch now says so explicitly, and its WARNING names the case and points at CHECK DATABASE ... FIX rather than the previous generic "Error on deleting ... edges".

2. Classic vs striped: where the CME originates. Fair - the javadoc sentence was accurate for the classic layout and misleading as a general claim. Added a note that StripedEdgeList.edgeIteratorForRemoval resolves every stripe head eagerly at iterator construction, so an unloadable stripe raises there rather than on a hop, and that both land in the same try in collectEdgesToDelete.

3. deleteRemainingChunks - agreed, no action taken.

Documentation only, no behaviour change. Re-verified: Issue5680VertexDeleteEdgeCollectionTest 8/8, and the com.arcadedb.graph package plus Issue4420TolerantDeleteTest / Issue4432CorruptVertexDeleteTest - the two that pin the tolerance discussed in point 1 - 345 tests, 0 failures.

…ex-strict-collection

# Conflicts:
#	engine/src/main/java/com/arcadedb/graph/GraphEngine.java
@claude

claude Bot commented Aug 1, 2026

Copy link
Copy Markdown

Review: #5680 strict edge collection on vertex delete

I read the full diff, the reworked GraphEngine.deleteVertex path, collectEdgesToDelete/hasNextEdgeToDelete/deleteRemainingChunks, the StripedEdgeList.edgeIteratorForRemoval strict composition, and the new test. Overall this is a strong, well-reasoned fix for a real data-loss bug, and it lands consistently with the #5670/#5678 pattern already established on deleteEdge. The write-up and the inline Javadoc make the intent unusually easy to verify.

What's right

  • Root cause correctly identified, fix symmetric with the neighbour-side change. The three read routes (unreadable head, mid-chain hole, skipped stripe chain) all funnel through getEdgeHeadChunkForWrite + edgeIteratorForRemoval + hasNextEdgeToDelete, so a transient miss becomes a retryable ConcurrentModificationException instead of a silent partial delete. Exception hierarchy checks out: ConcurrentModificationException extends NeedRetryException, so database.transaction(...) retry and the server auto-retry absorb it, while RecordNotFoundException (not a NeedRetryException) is handled separately - the catch ordering in both the collection loop and the per-edge loop is correct.
  • force end-to-end is a genuine gap closed. Absorbing the NeedRetryException raised while disconnecting a collected edge from the vertex at its other end is the right fix for the "broken neighbour blocks a forced delete" case, and aForcedDeleteIsNotBlockedByAnUnreadableNeighbourEdgeList pins it (fails on main).
  • Cypher: a relationship pattern with a pinned target expands the source's whole edge list #5660 read-from-getMostUpdatedVertex is a good catch, applied consistently with the rest of the file's edge-list-head reads.
  • Test coverage is excellent. Three deterministic reproductions that fail on main, controls that keep the fix honest, and a preconditioned concurrent invariant test. The super-node test asserting the read vs. removal walk side-by-side (rather than a vacuous end-to-end delete) is a nice touch, as is verifying appendsLanded > 0 so the concurrency test can't pass vacuously.

Discussion points (none blocking)

  1. catch (final Exception e) tolerating decode failures even when force == false. The second follow-up commit documents this well (the deleteRecordNoLock path arrives here with force == false, so failing would re-introduce the "undeletable record" bug of java.lang.NegativeArraySizeException on transaction commit #4420/PostCommandHandler: Cannot execute command #4432). The strict-vs-tolerant split rests entirely on the premise that RecordNotFoundException is always transient while a decode exception is always permanent. A genuinely lost chunk (hard-deleted, not merely invisible) also surfaces as RecordNotFoundException and will now retry-then-fail rather than tolerate - the intended contract (repair via CHECK DATABASE), but worth being explicit that this is a deliberate behavior change for that case too, not only the transient one.

  2. Self-loop edges (A --> A) are collected twice - once from A's OUT list and once from A's IN list - so edgesToDelete contains the same RID twice. The second edge.asEdge().delete() throws RecordNotFoundException and is swallowed, so it's harmless and matches pre-existing behavior. Worth one explicit test asserting a self-loop vertex deletes cleanly, since the removal walk is now stricter and self-loops are the shape most likely to surprise a future change here.

  3. Whole-list materialization for super-nodes - edgesToDelete holds every edge in memory before deleting. Unchanged from the previous implementation (not a regression), but on a promoted super-node with millions of edges this is a large ArrayList<Identifiable>. Out of scope here; noting given the GC/lightweight-collection mantra.

  4. concurrentVertexDeletionAndEdgeAppendingLeaveNoEdgeBehind uses 10,000 retries with a 1ms delay under a 5-minute bound and is @Tag("slow") - good defensive setup. The deleteFailures == 0 invariant is right; just be aware the high retry count trades a hang risk for a long tail on a loaded CI runner, which the 5-minute ceiling correctly caps.

Style/nits

  • Javadoc is extremely detailed, which fits this module's documented convention (engine CLAUDE.md favors recording non-obvious rationale) - keep it, just noting the density is intentional.
  • final usage, import-not-FQN, and the assertThat(...).isTrue() test style all follow CLAUDE.md conventions.

Nice work. The fix is correct, the failing-on-main tests prove it, and the tolerant/strict boundary is drawn at the right place (strict where a miss loses a reference, tolerant where it loses only garbage).

…/tolerant split actually keys on

Second review follow-ups on #5707.

A self-loop is the one shape both collection walks yield, so it lands in the delete list twice and the
second removal runs against a chain the first already cleaned. That is unchanged by this fix and
resolves quietly, but the removal walk is stricter now and a self-loop is the shape most likely to
surprise a later change here, so it gets a test rather than an argument.

The two catch arms in collectEdgesToDelete are easy to read as transient-versus-permanent, which is
wrong and would make the next reader draw the line in the wrong place. They key on what a MISS COSTS: a
chunk that cannot be found retries because deleting the vertex on a short list loses references, and
that is applied to a genuinely lost chunk too - no retry recovers it, so the delete now fails once the
retries are spent instead of quietly completing. Stated explicitly, since it is a deliberate behaviour
change for the lost-chunk case and not only for the transient one.
@lvca

lvca commented Aug 1, 2026

Copy link
Copy Markdown
Member Author

Thanks - second pass addressed in 11f4805. Point by point:

1. Explicit about the genuinely-lost chunk. Agreed, and worth more than a footnote: the split between the two catch arms is easy to read as transient vs permanent, which is the wrong axis and would lead the next reader to draw the line in the wrong place. It keys on what a miss costs. A chunk that cannot be FOUND retries because the alternative - deleting the vertex on a short list - loses references, and that is applied to a genuinely lost chunk too, which no retry recovers: that case now fails the delete once the retries are spent rather than quietly completing it. A chunk that cannot be DECODED takes the other arm for the #4420/#4432 reason, not because it is less permanent. Spelled out on collectEdgesToDelete.

2. Self-loop test. Added deletingAVertexWithASelfLoopRemovesItCleanly. You are right that it is the shape most likely to surprise a later change - it is the only one both walks yield, so it enters edgesToDelete twice and the second removal runs against a chain the first already cleaned. It asserts the precondition (the loop is reachable OUT and IN, so the duplicate really arises), that the vertex and the edge both go, and that check database is clean.

3. Whole-list materialization on super-nodes. Agreed and not changed. It is unchanged from the previous implementation, and the two-phase collect-then-delete exists precisely because the removals mutate the chain the walk is standing on - streaming it would mean walking a list while deleteEdge relinks and deletes chunks underneath, which is the class of problem this PR is fixing. Worth its own issue rather than a change smuggled into a correctness fix.

4. Retry count in the concurrency test. No change - as you note the 5-minute ceiling caps the tail, and the budget is deliberately far beyond what the contention needs so that any throwable reaching the counter is the bug rather than a tolerated retry.

Verified: Issue5680VertexDeleteEdgeCollectionTest 9/9, com.arcadedb.graph package 373 tests, 0 failures.

@claude

claude Bot commented Aug 1, 2026

Copy link
Copy Markdown

Review: #5707 - vertex delete no longer loses its own edges

I traced the change through the real call paths rather than just the diff. Short version: this is a correct, well-scoped fix with excellent tests and unusually good in-code documentation. A few notes below, only one of which is more than a nit, and none block merge.

What I verified holds up

  • Retry classification is sound. Both GraphEngine and StripedEdgeList import the ArcadeDB com.arcadedb.exception.ConcurrentModificationException (a NeedRetryException), not the java.util one. That is the linchpin of the whole design: the catch (NeedRetryException) arm in collectEdgesToDelete sits before the blanket catch (Exception), so a transient conflict is rethrown (or absorbed under force) and never falls through to the tolerant arm. Easy to get wrong, done right.
  • The hasNextEdgeToDelete javadoc is accurate. In EdgeIterator.hasNext() a dangling edge record is skipped internally (continue), while a chunk-hop miss escapes through getPrevious(). So a RecordNotFoundException reaching hasNextEdgeToDelete really is always a chunk hop, and the dangling-single-entry tolerance really is handled one level down. The comment matches the code.
  • Striped removal is genuinely strict. edgeIteratorForRemoval() -> allChains(true) -> addChain(strict=true) raises a retryable CME on an unreadable stripe head while building the iterator, instead of the READ path silent skip. The side-by-side assertion in deletingAPromotedSuperNodeWhoseStripeChainIsUnreadableReportsAConflict pins exactly that difference, which is what keeps that test non-vacuous.
  • Self-loop double-collection is safe. The edge lands in edgesToDelete from both OUT and IN; the second edge.asEdge().delete() hits catch (RecordNotFoundException) and is ignored. The dedicated test locks this in.
  • force end to end. The NeedRetryException arm in the per-edge disconnect loop closes the neighbour-blocks-forced-delete gap from the second comment on fix(engine) #5670: an unreadable edge-list chunk retries the delete instead of skipping it #5678, and aForcedDeleteIsNotBlockedByAnUnreadableNeighbourEdgeList covers it.

One thing worth a second look (non-blocking)

The trailing catch (final Exception e) in collectEdgesToDelete still tolerates any non-retryable throwable when force == false and deletes the vertex anyway on a partial edge view. The comment justifies this well for the decode family (SerializationException, BufferUnderflowException, NegativeArraySizeException) and the #4420/#4432 constraint that deleteRecordNoLock reaches here with force == false for a corrupt buffer - that reasoning is convincing.

The residual risk is that the same blanket catch would also mask a genuine programming error (e.g. an NPE or IllegalStateException from an unrelated future change) as "corrupted chunk, delete anyway" - i.e. reintroduce a silent-edge-loss path of exactly the kind this PR exists to eliminate. Since the comment already enumerates the expected exception types, consider catching that named set explicitly (or asserting on it) so an unexpected type propagates instead of being silently absorbed. Given the documented constraint this is a judgment call, not a defect.

Minor

  • deleteVertex(vertex, force) calls getMostUpdatedVertex up front; callers that already hold the most-updated instance pay a redundant cache lookup. Negligible, just noting it.
  • The concurrent test is correctly @Tag("slow") and self-guards against vacuous passes (appendsLanded > 0, non-empty precondition per round) - matches the CLAUDE.md guidance nicely.

Style / conventions

Consistent with the codebase: final used throughout, class-imported (no FQNs), assertThat(...).isTrue() style, Apache headers present, no stray System.out, no new dependencies. Release note included.

Nice work - the design rationale captured inline (strict-where-a-miss-loses-a-reference vs tolerant-where-it-loses-garbage, plus the rejected alternatives) will save the next maintainer a lot of guessing.

…s, and scope the concurrent test to what this fix owns

Third review follow-up on #5707, plus a flake the follow-up exposed.

The trailing catch (Exception) in collectEdgesToDelete would have absorbed an NPE or an
IllegalStateException from a future change as "corrupted chunk, delete anyway" - reintroducing exactly
the silent-edge-loss path this issue exists to remove. It now names the shapes it tolerates: the decode
family LocalDatabase already uses for the same purpose, plus ClassCastException and SchemaException,
which are the two further shapes a corrupt EDGE LIST adds (a head RID naming a record that is not an
edge segment, an edge bucket whose type is gone). Anything else propagates.

While re-running, concurrentVertexDeletionAndEdgeAppendingLeaveNoEdgeBehind proved flaky (~1 in 3), and
the cause is not this fix - it reproduces identically with the fix reverted. An edge an APPENDER commits
while its hub is being deleted can outlive that hub: the survivor has an existing out vertex and an in
naming the deleted hub, so it never existed when the deleter's collection walk ran and no vertex-side
strictness can prevent it. That is a separate defect on the append path, reported separately.

The test now asserts the invariant this fix actually owns - every edge that was IN the hub's list when
the deleter started must be gone with it, i.e. the fixture edges - instead of "no LINK edge survives at
all", which folded in the append-side defect. The round-start precondition moves from a global LINK
count to a per-hub degree for the same reason, so a leftover from an earlier round cannot make a healthy
round look wrong, and the end-of-run integrity check is dropped with the reason stated inline (the
non-racing fixtures still assert it). 10 consecutive green runs.
@lvca

lvca commented Aug 1, 2026

Copy link
Copy Markdown
Member Author

Third pass addressed in cab5a0a - and it turned up something worth flagging separately.

Closed catch (your one substantive point). Agreed, and implemented: absorbing an NPE or IllegalStateException as "corrupted chunk, delete anyway" would reintroduce exactly the silent-edge-loss path this PR removes. The arm now names what it tolerates - the decode family LocalDatabase uses for the same purpose (SerializationException, NegativeArraySizeException, BufferUnderflowException, IndexOutOfBoundsException, IllegalArgumentException) plus ClassCastException and SchemaException, the two further shapes a corrupt edge list adds on top (a head RID naming a record that is not an edge segment; an edge bucket whose type is gone). Everything else propagates, and the comment says to add a shape with its reason rather than widen the catch.

getMostUpdatedVertex redundant lookup. Not changed. It is a HashMap get on a path that is about to do record I/O, and making it conditional is what turned the #5660 invariant into a coincidence in the first place - one method owns the rule.


A finding, not a fix: concurrentVertexDeletionAndEdgeAppendingLeaveNoEdgeBehind was flaky (~1 in 3) and I chased it rather than retrying it away.

It is not this fix - it reproduces identically with the fix reverted (3 failures in 6 runs on the parent commit). Diagnosed by dumping the survivor:

GHOST edge=#65:70 out=#43:490 outExists=true in=#1:1 inExists=false inIsHub=true

An edge an appender commits while its hub is being deleted can outlive that hub. The survivor has a live out vertex and an in naming the deleted hub, so it did not exist when the deleter collection walk ran - no amount of vertex-side strictness can prevent it. That is a separate pre-existing defect on the append path (newEdge racing deleteVertex), and I would rather report it than fold it into this PR.

So the test now asserts the invariant this fix owns - every edge that was in the hub list when the deleter started must go with it, i.e. the fixture edges - instead of "no LINK edge survives at all", which quietly folded in the append-side defect. The round-start precondition moves from a global LINK count to a per-hub degree so a leftover from an earlier round cannot make a healthy round look wrong, and assertIntegrityClean() is dropped from this one test with the reason inline (the non-racing fixtures still assert integrity).

Verified: 10 consecutive green runs of the test class, com.arcadedb.graph 373 tests / 0 failures.

@claude

claude Bot commented Aug 1, 2026

Copy link
Copy Markdown

Review: #5680 strict edge-list collection on vertex delete

I read the four commits end to end (GraphEngine, EdgeLinkedList, StripedEdgeList, the new test, and the LocalDatabase.deleteRecordNoLock caller). This is a careful, high-quality fix that closes a real correctness hole with a coherent principle behind it ("strict where a miss loses a reference, tolerant where a miss loses only garbage"). Feedback below, mostly confirmations with a few minor considerations.

What holds up well

  • The strict/tolerant split lands in the right places. getEdgeHeadChunkForWrite + edgeIteratorForRemoval() + hasNextEdgeToDelete() cover the three loss routes, and keeping deleteRemainingChunks best-effort in both modes is correct: by that point every collected edge is disconnected, so a miss orphans chunk records (reclaimable) rather than a reference.
  • Exception plumbing is right. Verified ConcurrentModificationException extends NeedRetryException, so the strict conflicts flow through the NeedRetryException arm in collectEdgesToDelete and the per-edge loop, while RecordNotFoundException extends ArcadeDBException (not NeedRetryException), so the dangling-edge-record case is correctly caught separately and stays tolerated. Catch ordering (NeedRetryException before the decode family) is correct.
  • StripedEdgeList.edgeIteratorForRemoval() using allChains(true) matches the existing chainsForNeighbour reasoning exactly; the eager head resolution raising in addChain(strict=true) is the same conflict, handled in the same try. StripedEdgeList is the only EdgeLinkedList subclass, so no other override is missing.
  • The caller path is consistent. In LocalDatabase.deleteRecordNoLock, a CME from an unreadable edge list leaves isChunkChainBroken(record) false (the record body is intact), so it rethrows into the retry machinery rather than force-deleting - exactly the intended "transient retries, genuinely-broken needs CHECK DATABASE" behaviour, confirming the claim that force is not user-reachable for a broken edge list.
  • Test coverage is exemplary. Preconditions asserted before corruption, controls that keep the fix honest, the side-by-side read-vs-removal walk on the super-node (guarding against a vacuous pass), and the honest scoping of the concurrent test to fixture edges with the separate append-path defect documented rather than papered over.

Minor considerations (non-blocking)

  1. Breadth of the decode catch arm. Catching IllegalArgumentException | IndexOutOfBoundsException alongside the true decode shapes is broad - both are easy for a future change to throw for a non-corruption reason, and they would be silently absorbed as "corrupted chunk, delete anyway", quietly re-opening the very loss this PR removes. The inline comment argues the case well and the closed-list philosophy is the right instinct; just flagging that IllegalArgumentException is the member most likely to catch a programming error rather than a broken buffer. Worth a periodic second look if that arm ever starts firing in the field.

  2. Comment-to-code ratio. The javadoc/inline prose is essay-length (the deleteVertex + collectEdgesToDelete block is far more comment than code). It is genuinely informative and consistent with the 5670/5678 house style, so this is a taste note, not a request - but the density carries a drift risk as the surrounding code evolves.

  3. getMostUpdatedVertex(vertex) is now always called on the delete path (previously the raw handle was used). Correct for the 5660 route and the cost is negligible; just noting the extra lookup is now unconditional.

Verification

I did not independently re-run the suite; relying on the reported results (engine 10743/0, server 950/0, Issue5680 8/8 x3). Given the visible-behaviour change (DELETE VERTEX can now raise ConcurrentModificationException), the release-note entry and the server auto-retry coverage are the right mitigations, and the explicit-transaction contract note is accurate.

Nice work - the reasoning is sound and the tests pin exactly the three routes plus the force gap.

…ce in the tolerated set

Fourth review follow-up on #5707, comment only.

IllegalArgumentException is the loosest member of the closed catch list and the one a future change is
most likely to throw for a non-corruption reason. It stays, because Binary raises it ("Invalid
position", Binary.java:179) for a content offset that decodes past the end of the buffer - a genuine
corruption shape, and the reason LocalDatabase carries it in the same family - and dropping it would
make a vertex with that damage undeletable again. Naming the origin means the periodic second look the
review asks for can be done without re-deriving it, and narrowing it later is a decision with the
evidence attached rather than a guess.
@lvca

lvca commented Aug 1, 2026

Copy link
Copy Markdown
Member Author

Fourth pass - all three points were flagged non-blocking, and I agree with that classification. One is now recorded in the code (07a769c, comment only); the other two I am deliberately not changing, with reasons.

1. Breadth of the decode catch arm, IllegalArgumentException in particular. Fair, and it is the right member to single out. It stays, and I verified why rather than asserting it: Binary raises it at Binary.java:179 - "Invalid position " + index + " (size=" + buffer.limit() + ")" - for a content offset that decodes past the end of the buffer. That is a genuine corruption shape, not a caller error, and it is precisely why LocalDatabase.deleteRecordNoLock carries it in the same family; dropping it would make a vertex with that damage undeletable again, which is the #4420/#4432 regression. Narrowing it by message text would be brittle. What I have done is name that origin in the comment and mark it explicitly as the member to re-examine first if the arm ever fires in the field, so your periodic second look can be done with the evidence attached instead of re-derived.

2. Comment-to-code ratio. Not changing. You called it a taste note and I agree it is dense, but the engine CLAUDE.md explicitly asks for the non-obvious rationale to live next to the code, and this is a method whose whole difficulty is which failures deserve tolerance - the part that cannot be recovered by reading the statements. The drift risk is real; the mitigation is that each block names the concrete thing it depends on (LocalDatabase.deleteRecordNoLocks force flag, Binary.javas "Invalid position", chainsForNeighbour), so a change that invalidates one is visible rather than silent.

3. getMostUpdatedVertex now unconditional. Not changing, same reason as the previous pass: it is a HashMap get on a path about to do record I/O, and making it conditional is exactly what let the #5660 defect exist - one method owns the rule, and the read either always goes through it or it is a coincidence again.

Verified after the change: com.arcadedb.graph 373 tests, 0 failures.

That closes every point raised across the four passes. Recapping what was not implemented and why, so it is on the record in one place: the whole-list materialization for super-nodes (unchanged from before this PR; streaming it means walking a list while deleteEdge relinks chunks underneath, which is the class of problem this PR fixes - it wants its own issue), the concurrency-test retry budget (deliberately oversized so any throwable reaching the counter is a bug, with the 5-minute ceiling capping the tail), and the two above.

Separately, and not a review point: this work surfaced a pre-existing defect on the append path - an edge committed by newEdge while its target vertex is being deleted can outlive it (out alive, in naming the deleted hub). It reproduces with this fix reverted, so it is out of scope here, and the concurrent test is scoped around it rather than over it.

@claude

claude Bot commented Aug 1, 2026

Copy link
Copy Markdown

Review of #5707 - vertex delete retries an unreadable edge list instead of losing its edges

I traced the change end to end (collectEdgesToDelete -> EdgeLinkedList.edgeIteratorForRemoval -> EdgeIterator / EdgeIteratorFilter / IteratorFilterBase.getPrevious for the classic chain, and StripedEdgeList.addChain(strict=true) for the striped layout) and the behaviour matches every claim in the description. This is a careful, well-scoped fix. Notes below.

What holds up under inspection

  • Strict/tolerant boundary is drawn where a miss actually costs a reference. hasNextEdgeToDelete maps only a chunk-hop RecordNotFoundException (raised from MutableEdgeSegment.getPrevious via lookupByRID with loadContent=true) to a retryable ConcurrentModificationException. A dangling edge record is already swallowed earlier - EdgeIterator.hasNext skips it, and EdgeIteratorFilter / IteratorFilterBase.hasNext(true) route it through handleCorruption - so it never reaches the mapper. The javadoc 'only the chain hop surfaces here' is accurate against the code.
  • Striped path is consistent. edgeIteratorForRemoval composing allChains(true) means an unloadable stripe head raises ConcurrentModificationException while BUILDING the iterator, which lands in the NeedRetryException catch arm in collectEdgesToDelete rather than at a hop - same handling, correctly documented.
  • Catch-arm ordering is correct. ConcurrentModificationException extends NeedRetryException, so it is caught by the retryable arm before the decode-family arm; the test asserting isInstanceOf(ConcurrentModificationException).isInstanceOf(NeedRetryException) pins that.
  • The closed decode catch is the right call. Replacing the old blanket catch (Exception) with an explicit list (decode family + ClassCastException / SchemaException / IllegalArgumentException) closes the exact 'tolerate and delete anyway' hole the issue exists to remove, and the reasoning tying it to LocalDatabase.deleteRecordNoLock proceeding with force==false on a corrupt buffer (issues 4420 / 4432) is sound - failing here would re-break undeletable-record deletion.
  • The 5660 route (reading heads off getMostUpdatedVertex(vertex)) and the force end-to-end gap (absorbing the NeedRetryException while disconnecting the far end of a collected edge) are both genuine and worth fixing alongside.
  • All new catch types (SerializationException, BufferUnderflowException, etc.) are imported and the code compiles cleanly.

Test coverage

Strong. 8 tests with 3 that fail on main, deterministic corruption fixtures that assert their own preconditions (degree, chunk count, promotion) so they cannot pass on an unrelated throw, plus the self-loop control and the @tag('slow') concurrent invariant test scoped to what this fix owns. The honest call-out that the super-node end-to-end test is vacuous (raises with or without the fix) and the switch to asserting the two walks side by side is exactly the right way to keep it meaningful.

Minor observations (non-blocking)

  1. Behaviour change for client-managed transactions. DELETE VERTEX / vertex.delete() can now raise ConcurrentModificationException where it previously succeeded. database.transaction(..) and the server auto-retry absorb it, but an explicit multi-request begin/commit client sees it and must retry. Documented in the release notes and mirrors the 5678 contract, so intentional - just flagging it as the one externally visible semantics shift for reviewers weighing the merge.

  2. Comment density. The inline commentary is exceptionally thorough (several multi-paragraph blocks). It genuinely aids future readers on this concurrency-sensitive path and fits the repo documented preference, but a couple of the longest blocks (the decode-catch rationale, the getMostUpdatedVertex note) restate the class/PR-level javadoc. Not a change request - just noting a reader could get the same signal from a fraction of the text.

  3. deleteRemainingChunks reload asymmetry. hadOutList / hadInList are computed from the strict getEdgeHeadChunkForWrite, while the drain reloads via best-effort getEdgeHeadChunk. Deliberate and correct (a head that goes unreadable between collection and drain costs only orphaned chunks, which CHECK DATABASE reclaims) and documented - calling it out only so it is not simplified into symmetry by a later change.

Overall this reads as a correct, well-tested fix that closes four distinct silent-loss routes and makes force do what it always claimed. Nice work.

Reviewed with Claude Code.

@lvca lvca self-assigned this Aug 1, 2026
@lvca lvca added this to the 26.8.1 milestone Aug 1, 2026
@lvca
lvca merged commit 324eca1 into main Aug 1, 2026
15 checks passed
@lvca
lvca deleted the issue-5680-deletevertex-strict-collection branch August 1, 2026 17:42
@codacy-production

Copy link
Copy Markdown

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

🟢 Metrics 0 complexity

Metric Results
Complexity 0

View in Codacy

🟢 Coverage 90.74% diff coverage · -6.28% coverage variation

Metric Results
Coverage variation ✅ -6.28% coverage variation
Diff coverage ✅ 90.74% diff coverage

View coverage diff in Codacy

Coverage variation details
Coverable lines Covered lines Coverage
Common ancestor commit (ffe7865) 150555 113861 75.63%
Head commit (07a769c) 175578 (+25023) 121764 (+7903) 69.35% (-6.28%)

Coverage variation is the difference between the coverage for the head and common ancestor commits of the pull request branch: <coverage of head commit> - <coverage of common ancestor commit>

Diff coverage details
Coverable lines Covered lines Diff coverage
Pull request (#5707) 54 49 90.74%

Diff coverage is the percentage of lines that are covered by tests out of the coverable lines that the pull request added or modified: <covered lines added or modified>/<coverable lines added or modified> * 100%

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

@codecov

codecov Bot commented Aug 1, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 90.74074% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 67.63%. Comparing base (ffe7865) to head (07a769c).
⚠️ Report is 6 commits behind head on main.

Files with missing lines Patch % Lines
.../src/main/java/com/arcadedb/graph/GraphEngine.java 89.79% 5 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               main    #5707      +/-   ##
============================================
+ Coverage     66.74%   67.63%   +0.89%     
============================================
  Files          1774     1729      -45     
  Lines        150555   143554    -7001     
  Branches      31938    30791    -1147     
============================================
- Hits         100487    97093    -3394     
+ Misses        36732    33745    -2987     
+ Partials      13336    12716     -620     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

lvca added a commit that referenced this pull request Aug 1, 2026
…med records only

CHECK DATABASE could be narrowed to a TYPE or a BUCKET, but not to a record. That is the sharp edge
of the strict vertex delete merged in #5707: a genuinely broken edge chain makes its vertex
undeletable until CHECK DATABASE ... FIX rebuilds the adjacency from the surviving edge records, and
the smallest way to ask for that repair was two full passes over the whole vertex type.

  CHECK DATABASE RECORD #12:3 FIX
  CHECK DATABASE RECORD #12:3, #12:9 FIX

The per-record work and the fix are identical to the type-wide run, including the edge-list rebuild;
only the enumeration changes. Combines with FIX and COMPRESS. Rejected when combined with TYPE or
BUCKET - RECORD is already the narrowest scope, so letting one silently win would run a check nobody
asked for and an intersection would be a third semantics nobody asked for. The database-wide passes
(buckets, external properties, indexes) and the orphaned-segment reclaim are skipped, since none can
be narrowed to a record.

Two costs it deliberately does not bound, documented rather than glossed: rebuilding an adjacency
needs every surviving edge that points at the vertex and no index maps endpoints back to edges, so
the scoped run saves the vertex passes but still scans the edge types; and a listed record that
turns out to be corrupted still has every index on its bucket rebuilt.

Also fixes a vacuously-passing assertion in Issue5670EdgeDeleteDanglingBackRefTest: it asserted
totalErrors == 0, and check database reports no such field, so the null-tolerant reader turned it
into 0 == 0 on every run whatever the check found. It now asserts on autoFix, invalidLinks,
totalWarnings and totalCorruptedRecords, and fails loudly on a field that does not exist.
lvca added a commit that referenced this pull request Aug 2, 2026
…med records only (#5710)

* feat(engine) #5680: CHECK DATABASE RECORD <rid> checks and repairs named records only

CHECK DATABASE could be narrowed to a TYPE or a BUCKET, but not to a record. That is the sharp edge
of the strict vertex delete merged in #5707: a genuinely broken edge chain makes its vertex
undeletable until CHECK DATABASE ... FIX rebuilds the adjacency from the surviving edge records, and
the smallest way to ask for that repair was two full passes over the whole vertex type.

  CHECK DATABASE RECORD #12:3 FIX
  CHECK DATABASE RECORD #12:3, #12:9 FIX

The per-record work and the fix are identical to the type-wide run, including the edge-list rebuild;
only the enumeration changes. Combines with FIX and COMPRESS. Rejected when combined with TYPE or
BUCKET - RECORD is already the narrowest scope, so letting one silently win would run a check nobody
asked for and an intersection would be a third semantics nobody asked for. The database-wide passes
(buckets, external properties, indexes) and the orphaned-segment reclaim are skipped, since none can
be narrowed to a record.

Two costs it deliberately does not bound, documented rather than glossed: rebuilding an adjacency
needs every surviving edge that points at the vertex and no index maps endpoints back to edges, so
the scoped run saves the vertex passes but still scans the edge types; and a listed record that
turns out to be corrupted still has every index on its bucket rebuilt.

Also fixes a vacuously-passing assertion in Issue5670EdgeDeleteDanglingBackRefTest: it asserted
totalErrors == 0, and check database reports no such field, so the null-tolerant reader turned it
into 0 == 0 on every run whatever the check found. It now asserts on autoFix, invalidLinks,
totalWarnings and totalCorruptedRecords, and fails loudly on a field that does not exist.

* style(engine) #5680: strip trailing whitespace flagged by pre-commit

The setup job failed on a trailing-whitespace line an earlier edit left after a comment in the
RECORD-scoped edge branch. Removed, along with the stray blank line beside it.

* fix(engine) #5680: a missing RID is not corruption, and the scoped progress step stops double-counting

Two independent review passes on the RECORD scope, four actionable points between them.

- A listed RID that simply is not there was recorded through addCorrupted, which puts its BUCKET
  into affectedBuckets - so CHECK DATABASE RECORD ... FIX dropped and rebuilt every index on that
  bucket, a full bucket scan, over what is usually a mistyped or already-deleted RID. That is
  exactly the cost the scope exists to avoid, and the scope is meant to be hand-typed after a failed
  delete. RecordNotFoundException is now reported and never flagged corrupted, in all three arms
  (vertex, edge, document). Mutation-checked: restoring addCorrupted fails the new test with
  totalCorruptedRecords=1, autoFix=1.
- checkScopedRecords bumped currentStep for every group, but the document arm reaches stepBegin,
  which bumps it again - so a scope containing a plain document reported "step 3 of 2" against a
  totalSteps budgeted at one step per group. Each arm now owns its bump; the graph arms need it
  because they pass a fixed value into the nested GraphDatabaseChecker instead.
- setRecords and the statement's stream now drop null RIDs. Rid.toRecordId can answer null for a
  non-literal form, and groupRecordsByType would meet it as an NPE on getBucketId(). Not reachable
  through the RECORD #n:n literal grammar, but setRecords(Set<RID>) is public API.
- The parser test's cross-reference named a test class that no longer exists after the rescope.

Tests: four added for the branches that were unexercised - an edge-typed RID, a document-typed RID
(the branch the progress bug lived in), several RIDs spanning different types in one command, and a
missing-but-valid RID under FIX, which pins the first item above.

* fix(engine) #5680: a RECORD scope that resolves to nothing is refused, not widened to the whole database

Review round on the rescoped PR: LGTM with three minor points, two taken.

- CHECK DATABASE RECORD whose RIDs all resolve to null left the checker with an empty scope, which
  reads as "no scope" - so a command that explicitly named records quietly ran a FULL database
  check. The review judged this unreachable from SQL; it is not. Only the `#n:n` literal always
  resolves, while the non-literal {"@Rid": <expression>} form goes through an expression that can
  answer null, so `CHECK DATABASE RECORD {"@Rid": null}` reached it. Refused now, with a test that
  fails when the guard is stubbed out.
- A warning dropped for exceeding maxWarnings was discarded silently in the two scoped arms, where
  GraphDatabaseChecker.addWarning logs it. Extracted addScopedWarning so both behave the same:
  always counted, retained under the cap, logged when dropped.

Not taken: extracting the duplicated RecordNotFoundException/Exception catch blocks in the scoped
vertex and edge arms. They would need six accumulators threaded through a shared helper, which reads
worse than the duplication, and the surrounding type-wide methods already parallel each other the
same way - the review noted as much in raising it.

* fix(engine) #5680: the empty-scope guard moves into setRecords, so it protects the public API too

Review round: LGTM once the setRecords widening note was considered. Went one step further than the
note it asked for.

The guard against "a RECORD scope that resolves to nothing silently becomes a full database check"
lived only in the SQL layer, so an embedded caller reaching DatabaseChecker.setRecords directly
still hit the widening. The review reasoned that setRecords cannot tell "named records, none
usable" from "named no records" - true only because the SQL layer filtered the nulls out BEFORE
calling it. It no longer does: the nulls are passed through, setRecords owns both the filtering and
the refusal, and one guard now covers both entry points. A genuinely empty (or null) set still means
"no scope" and is accepted. Both are asserted.

Also: the document arm said "cannot be loaded, removing it", but nothing removes the record there -
corruptedRecords only drives the index rebuild. The review called the wording inherited and asked
for no action; it is new code here, so it now says what is true, with a note on why it diverges from
the type-wide checkDocuments (which also calls a document a "vertex").

Adds the warning-cap test the review asked for: 40 bogus RIDs against maxWarnings=10 retains 10
messages and still counts 40.

Not added: an execution test for RECORD ... FIX COMPRESS. The compress pass is shared, unscoped code
already covered elsewhere, and routing it through the scoped enumeration exercises no new branch.

* docs(engine) #5680: state the COMPRESS caveat, and pin the corrupted-record fix cost with a test

Review approved with three non-blocking notes; two taken.

- COMPRESS is unaffected by the RECORD scope and still compresses the whole database. It is opt-in,
  but it is the one clause that breaks the "naming a record bounds the cost" promise, so it now sits
  in the release notes beside the other two costs the scope does not bound.
- Added the test the review asked for: a listed record that is GENUINELY corrupted - not merely
  missing, not merely chain-broken - is flagged, and under FIX every index on its bucket is dropped
  and rebuilt. That turns the "RECORD bounds the check, not necessarily the fix" paragraph into an
  executable assertion, so a future change cannot quietly make a scoped repair either cheaper or
  more destructive than documented. Mutation-checked: dropping addCorrupted from the scoped arm
  fails it, and the injected corruption is the intended shape (buffer truncated to 8 bytes against
  the 25-byte vertex prefix), distinct from the missing-RID path.

Not taken: moving the RECORD-plus-TYPE/BUCKET rejection from check() into the setters. setTypes and
setRecords can be called in either order, so a setter-level guard would fire or not depending on
call order; check() is the only place that sees the whole configuration. The review called the
current placement acceptable.

* fix(engine) #5680: a dropped RID is reported, the no-type step is named, and a comment stops overselling

Review round: approve, all notes minor. Three taken.

- The "Exception, not Throwable" comment claimed more protection than the code gives. It guards the
  LOOKUP, but the shared per-record consumer still catches Throwable around the connectivity check
  itself - matching the type-wide path, which is why it stays. The comment now says exactly what is
  and is not covered rather than implying an Error can never be recorded as corruption.
- setRecords refused an all-null scope but dropped a PARTIAL one silently, so a caller who mistyped
  one of several RIDs got a clean report for a check that skipped it. Narrowing is still allowed -
  unlike widening it cannot do more than asked - but it now surfaces a warning naming how many were
  dropped, with a test.
- The no-type group advanced currentStep without ever naming the step, so a progress poller watched
  it go by blank. Routed through stepBegin like the document arm, which also owns the bump.

Not taken:

- Rejecting RECORD together with COMPRESS for symmetry with the TYPE/BUCKET rejection. The two are
  not the same case: RECORD plus TYPE/BUCKET has no coherent meaning, whereas "check this record,
  then compress the database" is perfectly meaningful - it is just not cheap. That is a
  documentation problem, and the release note now states it.
- @tag("slow") on the 200-edge tests. Measured rather than assumed: the whole class runs in 0.53s,
  and CLAUDE.md's criteria are multi-second elapsed or large batches. 200 tiny transactions under a
  second is neither.

* fix(engine) #5680: diagnose the clause conflict first, warn that COMPRESS is unscoped

Review round: refinements, not blockers. Four taken.

- CHECK DATABASE TYPE Hub RECORD {"@Rid": null} reported "none of them resolves to a RID", because
  the RIDs are resolved before check() reaches the combination guard - pointing the operator at the
  inner failure while the actual mistake is the clause combination. The statement now diagnoses the
  combination before resolving anything; check() keeps its own guard for programmatic callers.
- setRecords left droppedRecords stale on its early-return path. Harmless today, since the warning
  is only emitted under a record scope, but the field is public-API-visible state and should not
  depend on call order.
- COMPRESS now warns that it is not limited by the RECORD scope. The review suggested this as a
  middle ground between refusing the combination and staying silent, and it is a better answer than
  the documentation-only one: the command still does what it was asked, but an operator who named a
  record and expected a bounded run is told otherwise. Tested both ways, so the warning tracks the
  clause rather than the scope.
- Recorded, not fixed: the type-wide checkDocuments adds to the warnings/corrupted sets without
  incrementing the corresponding totals, so a corrupt document reports different totals depending on
  which path found it. The scoped arm here is the correct one; aligning the type-wide path is a
  behaviour change to a command this issue does not otherwise touch.

Agreed with the review that the duplicated catch blocks read better than threading six accumulators
through a shared helper; no change.

* fix(engine) #5680: one owner for the conflict message, no counts a Set cannot know, bounded corrupted set

Review round: nothing blocking. All four points taken.

- The "cannot be combined with TYPE or BUCKET" text was written out in both the statement and the
  checker. The two-layer guard stays - the statement has to diagnose the clause conflict before the
  RIDs are resolved - but the wording now lives in one constant so the two cannot drift.
- Two messages quoted a record count the Set could not know: several RID expressions that all
  resolve to null collapse to a single null element, so "given 3 record(s)" was never reachable and
  "1 of the record(s) did not resolve" under-reported. Rather than plumb the original count through,
  both messages stop quoting a number, which is accurate whatever the input.
- checkScopedDocuments grew corruptedRecords without a bound while every other path honours
  maxCorrupted/maxWarnings. Bounded the same way, with the total still counting every occurrence.
- Documented that the edge sweep a rebuild needs runs once per distinct vertex TYPE named, not once
  per run: ten vertices of one type cost one sweep, one vertex of each of three types cost three.
lvca added a commit that referenced this pull request Aug 2, 2026
…airs it, and the conflict keeps its cause (#5767)

* fix(engine) #5764: a refused vertex delete names the command that repairs it, and the conflict keeps its cause

Follow-ups from #5680, whose two halves landed separately: #5707 made deleteVertex strict, #5710 added the
CHECK DATABASE RECORD scope. Neither is defective; the seams between them were never closed.

1. The repair advice now names the scoped command with the RID substituted. Split in two, which the report did
   not distinguish: the RECORD scope is the right answer only where the delete was REFUSED and the vertex is
   still there to repair. The force arms log AFTER the record is gone, so a scoped check would name a record
   that no longer exists - they keep the whole-database form and say what the scoped form buys when run first.
   And at the other-end conflict the broken list belongs to the NEIGHBOUR, so that is the RID named.
   The previously bare rethrow now carries the advice instead of only getEdgeHeadChunkForWrite's
   "concurrent commit in flight", which is the diagnosis for the transient case nobody ever sees.

2. ConcurrentModificationException gains the (message, cause) constructor NeedRetryException already declared.
   Eight sites across GraphEngine, EdgeLinkedList and StripedEdgeList kept the message and dropped the stack.
   A conflict is normally absorbed by the retry, so the run that surfaces one is the retry-exhausted run -
   exactly the one whose trace has to be diagnosable. The class on the wire is unchanged.

3. checkDocuments and checkScopedDocuments report a corrupt document identically. The type-wide arm added to
   the warning and corrupted-record SETS without touching totalWarnings/totalCorruptedRecords, called a
   document a "vertex" it was "removing" when nothing on that path removes anything, and honoured no
   maxWarnings cap at all. Both now go through shared addWarning/addCorrupted.

4. The scoped vertex check running one pass where the type-wide runs two is NOT a gap: lookupByRID IS the
   raw-view materialisation (same newImmutableRecord(type, rid, bucket.getRecord(rid).copyOfContent())) and
   checkConnectivity opens with the same asVertex(true) that forces the decode, while getRecord and scan
   resolve placeholders and multi-page chains identically. Documented and pinned by a test rather than
   "fixed" by adding a second pass, which would double the cost of the scope that exists to be cheap.

The advice tests execute what the message says: the command is parsed out of the raised exception and run
verbatim, then the delete must succeed - so a wrong RID or a shape the parser rejects fails here.

* review: state addCorrupted's dedup guarantee honestly, and drop the reflection in the advice test

Three points from the code review on #5767, all minor and none blocking.

1. addCorrupted's "a record flagged twice is one corrupted record" holds only while the retained set is under
   the cap. Behaviour kept (the contains() check past the cap is strictly better than an unconditional
   increment); the Javadoc now says exactly what survives the cap and why closing the gap is not worth it - the
   only exact fix is a second unbounded set of every RID ever counted, i.e. the memory the cap exists to refuse.
   Also recorded that neither caller reaches it: the bucket scan visits each RID once and the RECORD scope is a
   Set, so a duplicate cannot survive as far as groupRecordsByType.

2. The reviewer flagged that the retained set is capped with maxWarnings while the graph arms take a separate
   maxCorrupted. Documented rather than changed: DatabaseChecker has no maxCorrupted field, and checkScopedRecords
   already passes the remaining maxWarnings budget into the GraphDatabaseChecker parameter of that name. Two
   names, one setting.

3. The advice test reached the LogManager logger by reflecting on the private field. LogManager exposes a public
   getLogger() for precisely this case - a caller that swaps the logger and has to put the original back - so the
   reflection helper is gone along with its rename fragility.

* review: record the maxWarnings/maxCorrupted coupling at the field, and why the interpolated cause is not redundant

Three non-blocking observations from the second review on #5767. All three resolve to recording constraints a
future reader would otherwise have to rediscover - one of them by breaking a test.

1. maxWarnings caps the corrupted set while GraphDatabaseChecker takes a maxCorrupted for the SAME set. The
   method Javadoc said "two names, one setting"; the note now lives on the field, where a reader meets it, and
   adds the consequence the reviewer pointed out: retained corruptedRecords become affectedBuckets, so a record
   dropped past the cap does not get its bucket's indexes rebuilt under FIX.

2. The advice test swaps the process-wide LogManager logger. Restored in a finally and safe under sequential
   class execution, but it is shared state; documented as a latent constraint if class-level parallelism is ever
   enabled, along with why there is no per-invocation seam to capture through instead.

3. getEdgeHeadChunkForWrite and hasNextEdgeToDelete interpolate e.getMessage() AND keep e as the cause, which
   reads as redundant. It is not: the missing CHUNK's RID appears ONLY in the record-not-found message - the
   surrounding text names the vertex - and the top-level message is what a log line and an HTTP error body carry,
   while the cause chain surfaces only in a full trace. Removing it would drop the RID and fail
   Issue5670EdgeDeleteDanglingBackRefTest, which asserts on it. Commented at both sites.

* review: guard the advice against a message-less conflict, and record why otherEndOf can equal the vertex

Two of the three optional-polish points from the third review on #5767 (which was LGTM).

1. withRepairAdvice built the message as e.getMessage() + advice. Every conflict on this path carries a message
   today, but a message-less one would render "null. If it persists...", which reads as a bug in the advice rather
   than as a missing diagnosis. Falls back to toString(), i.e. the class name.

2. otherEndOf would answer vertexRID itself for a self-loop, which the method name reads as impossible. That is
   the correct answer - a self-loop has no other end and the list that failed to disconnect it IS the vertex's own
   - so the equality is documented rather than "simplified" away. Also recorded why there is no test: the caller
   runs only after both of the vertex's lists have been collected, and a self-loop is read out of those same two
   lists, so a list broken enough to fail the disconnection fails the collection first.

Not taken: the reviewer flagged that otherEndOf is computed before the force branch, so it runs on every retry of
a transient conflict. Both branches need the value, so moving it saves nothing - and the reviewer scoped it as
"for completeness" on an exception path where asEdge() is light.
robfrank pushed a commit that referenced this pull request Aug 14, 2026
…instead of losing its edges (#5707)

* fix(engine) #5680: an unreadable edge list retries the vertex delete instead of losing its edges

deleteVertex collected the edges to remove through the best-effort getEdgeHeadChunk with the whole
walk wrapped in catch (Exception), so any part of the list that could not be read at that instant
was taken for "nothing to remove here" and the vertex record was deleted on top of that empty or
partial view: the edges outlived their endpoint, with a back-reference left on an innocent
neighbour. Three routes reached that outcome - an unreadable head chunk, an unreadable hop mid-chain
(everything behind the hole dropped), and, on a promoted super-node, a stripe chain the read walk is
allowed to skip, which lost a whole stripe with no exception behind it at all. None of them is a
fact about the graph; they are the transient publication window #5670 was about, which deleteEdge
has answered with a retry since #5678.

The collection now reads the list the way a removal must: the head through getEdgeHeadChunkForWrite,
the walk through the new EdgeLinkedList.edgeIteratorForRemoval (strict over stripe chains), and the
chain hops mapped to a retryable ConcurrentModificationException. force is the single escape hatch
and now works end to end: it also absorbs the conflict raised while disconnecting a collected edge
from the vertex at its OTHER end, which previously let a broken neighbour block a forced delete.
Draining the leftover chunk records stays best-effort in both modes on purpose - by then a miss
loses orphaned chunks, never a reference. The heads are also read from the transaction's own vertex
instance (#5660), closing a fourth silent-loss route that needed no race.

Strictness costs no repairability: CHECK DATABASE ... FIX rebuilds an unloadable chain from the
surviving edge records and is never blocked by the delete being blocked.

* docs(engine) #5680: state why a chunk that cannot be DECODED stays tolerated, and where the striped conflict is raised

Review follow-ups on #5707, both documentation.

The blanket catch (Exception) left in collectEdgesToDelete is the residue once the transient window is
split off into the retryable branch: an undecodable buffer, which no retry can fix and which is
tolerated even when force is false. That reads as inconsistent with the strictness around it until you
follow where such a record comes from - LocalDatabase.deleteRecordNoLock catches the same exception
family around the index cleanup and proceeds WITHOUT raising its force flag, so a corrupt-buffer vertex
arrives here with force == false and failing would make it undeletable again (#4420, #4432). Its cost is
larger than a single tolerated dangling entry, so the warning now names it and points at CHECK DATABASE.

hasNextEdgeToDelete's javadoc claimed only a chain hop surfaces through it, which is true on the classic
layout but not the whole story on a promoted super-node: StripedEdgeList.edgeIteratorForRemoval resolves
every stripe head while BUILDING the iterator, so an unloadable stripe raises there instead. Same try,
same handling - the note keeps a future reader from reading the helper as the only strict point.

* test(engine) #5680: pin the self-loop delete, and say what the strict/tolerant split actually keys on

Second review follow-ups on #5707.

A self-loop is the one shape both collection walks yield, so it lands in the delete list twice and the
second removal runs against a chain the first already cleaned. That is unchanged by this fix and
resolves quietly, but the removal walk is stricter now and a self-loop is the shape most likely to
surprise a later change here, so it gets a test rather than an argument.

The two catch arms in collectEdgesToDelete are easy to read as transient-versus-permanent, which is
wrong and would make the next reader draw the line in the wrong place. They key on what a MISS COSTS: a
chunk that cannot be found retries because deleting the vertex on a short list loses references, and
that is applied to a genuinely lost chunk too - no retry recovers it, so the delete now fails once the
retries are spent instead of quietly completing. Stated explicitly, since it is a deliberate behaviour
change for the lost-chunk case and not only for the transient one.

* fix(engine) #5680: close the tolerant catch to named corruption shapes, and scope the concurrent test to what this fix owns

Third review follow-up on #5707, plus a flake the follow-up exposed.

The trailing catch (Exception) in collectEdgesToDelete would have absorbed an NPE or an
IllegalStateException from a future change as "corrupted chunk, delete anyway" - reintroducing exactly
the silent-edge-loss path this issue exists to remove. It now names the shapes it tolerates: the decode
family LocalDatabase already uses for the same purpose, plus ClassCastException and SchemaException,
which are the two further shapes a corrupt EDGE LIST adds (a head RID naming a record that is not an
edge segment, an edge bucket whose type is gone). Anything else propagates.

While re-running, concurrentVertexDeletionAndEdgeAppendingLeaveNoEdgeBehind proved flaky (~1 in 3), and
the cause is not this fix - it reproduces identically with the fix reverted. An edge an APPENDER commits
while its hub is being deleted can outlive that hub: the survivor has an existing out vertex and an in
naming the deleted hub, so it never existed when the deleter's collection walk ran and no vertex-side
strictness can prevent it. That is a separate defect on the append path, reported separately.

The test now asserts the invariant this fix actually owns - every edge that was IN the hub's list when
the deleter started must be gone with it, i.e. the fixture edges - instead of "no LINK edge survives at
all", which folded in the append-side defect. The round-start precondition moves from a global LINK
count to a per-hub degree for the same reason, so a leftover from an earlier round cannot make a healthy
round look wrong, and the end-of-run integrity check is dropped with the reason stated inline (the
non-racing fixtures still assert it). 10 consecutive green runs.

* docs(engine) #5680: record why IllegalArgumentException earns its place in the tolerated set

Fourth review follow-up on #5707, comment only.

IllegalArgumentException is the loosest member of the closed catch list and the one a future change is
most likely to throw for a non-corruption reason. It stays, because Binary raises it ("Invalid
position", Binary.java:179) for a content offset that decodes past the end of the buffer - a genuine
corruption shape, and the reason LocalDatabase carries it in the same family - and dropping it would
make a vertex with that damage undeletable again. Naming the origin means the periodic second look the
review asks for can be done without re-deriving it, and narrowing it later is a decision with the
evidence attached rather than a guess.

(cherry picked from commit 324eca1)
robfrank pushed a commit that referenced this pull request Aug 14, 2026
…med records only (#5710)

* feat(engine) #5680: CHECK DATABASE RECORD <rid> checks and repairs named records only

CHECK DATABASE could be narrowed to a TYPE or a BUCKET, but not to a record. That is the sharp edge
of the strict vertex delete merged in #5707: a genuinely broken edge chain makes its vertex
undeletable until CHECK DATABASE ... FIX rebuilds the adjacency from the surviving edge records, and
the smallest way to ask for that repair was two full passes over the whole vertex type.

  CHECK DATABASE RECORD #12:3 FIX
  CHECK DATABASE RECORD #12:3, #12:9 FIX

The per-record work and the fix are identical to the type-wide run, including the edge-list rebuild;
only the enumeration changes. Combines with FIX and COMPRESS. Rejected when combined with TYPE or
BUCKET - RECORD is already the narrowest scope, so letting one silently win would run a check nobody
asked for and an intersection would be a third semantics nobody asked for. The database-wide passes
(buckets, external properties, indexes) and the orphaned-segment reclaim are skipped, since none can
be narrowed to a record.

Two costs it deliberately does not bound, documented rather than glossed: rebuilding an adjacency
needs every surviving edge that points at the vertex and no index maps endpoints back to edges, so
the scoped run saves the vertex passes but still scans the edge types; and a listed record that
turns out to be corrupted still has every index on its bucket rebuilt.

Also fixes a vacuously-passing assertion in Issue5670EdgeDeleteDanglingBackRefTest: it asserted
totalErrors == 0, and check database reports no such field, so the null-tolerant reader turned it
into 0 == 0 on every run whatever the check found. It now asserts on autoFix, invalidLinks,
totalWarnings and totalCorruptedRecords, and fails loudly on a field that does not exist.

* style(engine) #5680: strip trailing whitespace flagged by pre-commit

The setup job failed on a trailing-whitespace line an earlier edit left after a comment in the
RECORD-scoped edge branch. Removed, along with the stray blank line beside it.

* fix(engine) #5680: a missing RID is not corruption, and the scoped progress step stops double-counting

Two independent review passes on the RECORD scope, four actionable points between them.

- A listed RID that simply is not there was recorded through addCorrupted, which puts its BUCKET
  into affectedBuckets - so CHECK DATABASE RECORD ... FIX dropped and rebuilt every index on that
  bucket, a full bucket scan, over what is usually a mistyped or already-deleted RID. That is
  exactly the cost the scope exists to avoid, and the scope is meant to be hand-typed after a failed
  delete. RecordNotFoundException is now reported and never flagged corrupted, in all three arms
  (vertex, edge, document). Mutation-checked: restoring addCorrupted fails the new test with
  totalCorruptedRecords=1, autoFix=1.
- checkScopedRecords bumped currentStep for every group, but the document arm reaches stepBegin,
  which bumps it again - so a scope containing a plain document reported "step 3 of 2" against a
  totalSteps budgeted at one step per group. Each arm now owns its bump; the graph arms need it
  because they pass a fixed value into the nested GraphDatabaseChecker instead.
- setRecords and the statement's stream now drop null RIDs. Rid.toRecordId can answer null for a
  non-literal form, and groupRecordsByType would meet it as an NPE on getBucketId(). Not reachable
  through the RECORD #n:n literal grammar, but setRecords(Set<RID>) is public API.
- The parser test's cross-reference named a test class that no longer exists after the rescope.

Tests: four added for the branches that were unexercised - an edge-typed RID, a document-typed RID
(the branch the progress bug lived in), several RIDs spanning different types in one command, and a
missing-but-valid RID under FIX, which pins the first item above.

* fix(engine) #5680: a RECORD scope that resolves to nothing is refused, not widened to the whole database

Review round on the rescoped PR: LGTM with three minor points, two taken.

- CHECK DATABASE RECORD whose RIDs all resolve to null left the checker with an empty scope, which
  reads as "no scope" - so a command that explicitly named records quietly ran a FULL database
  check. The review judged this unreachable from SQL; it is not. Only the `#n:n` literal always
  resolves, while the non-literal {"@Rid": <expression>} form goes through an expression that can
  answer null, so `CHECK DATABASE RECORD {"@Rid": null}` reached it. Refused now, with a test that
  fails when the guard is stubbed out.
- A warning dropped for exceeding maxWarnings was discarded silently in the two scoped arms, where
  GraphDatabaseChecker.addWarning logs it. Extracted addScopedWarning so both behave the same:
  always counted, retained under the cap, logged when dropped.

Not taken: extracting the duplicated RecordNotFoundException/Exception catch blocks in the scoped
vertex and edge arms. They would need six accumulators threaded through a shared helper, which reads
worse than the duplication, and the surrounding type-wide methods already parallel each other the
same way - the review noted as much in raising it.

* fix(engine) #5680: the empty-scope guard moves into setRecords, so it protects the public API too

Review round: LGTM once the setRecords widening note was considered. Went one step further than the
note it asked for.

The guard against "a RECORD scope that resolves to nothing silently becomes a full database check"
lived only in the SQL layer, so an embedded caller reaching DatabaseChecker.setRecords directly
still hit the widening. The review reasoned that setRecords cannot tell "named records, none
usable" from "named no records" - true only because the SQL layer filtered the nulls out BEFORE
calling it. It no longer does: the nulls are passed through, setRecords owns both the filtering and
the refusal, and one guard now covers both entry points. A genuinely empty (or null) set still means
"no scope" and is accepted. Both are asserted.

Also: the document arm said "cannot be loaded, removing it", but nothing removes the record there -
corruptedRecords only drives the index rebuild. The review called the wording inherited and asked
for no action; it is new code here, so it now says what is true, with a note on why it diverges from
the type-wide checkDocuments (which also calls a document a "vertex").

Adds the warning-cap test the review asked for: 40 bogus RIDs against maxWarnings=10 retains 10
messages and still counts 40.

Not added: an execution test for RECORD ... FIX COMPRESS. The compress pass is shared, unscoped code
already covered elsewhere, and routing it through the scoped enumeration exercises no new branch.

* docs(engine) #5680: state the COMPRESS caveat, and pin the corrupted-record fix cost with a test

Review approved with three non-blocking notes; two taken.

- COMPRESS is unaffected by the RECORD scope and still compresses the whole database. It is opt-in,
  but it is the one clause that breaks the "naming a record bounds the cost" promise, so it now sits
  in the release notes beside the other two costs the scope does not bound.
- Added the test the review asked for: a listed record that is GENUINELY corrupted - not merely
  missing, not merely chain-broken - is flagged, and under FIX every index on its bucket is dropped
  and rebuilt. That turns the "RECORD bounds the check, not necessarily the fix" paragraph into an
  executable assertion, so a future change cannot quietly make a scoped repair either cheaper or
  more destructive than documented. Mutation-checked: dropping addCorrupted from the scoped arm
  fails it, and the injected corruption is the intended shape (buffer truncated to 8 bytes against
  the 25-byte vertex prefix), distinct from the missing-RID path.

Not taken: moving the RECORD-plus-TYPE/BUCKET rejection from check() into the setters. setTypes and
setRecords can be called in either order, so a setter-level guard would fire or not depending on
call order; check() is the only place that sees the whole configuration. The review called the
current placement acceptable.

* fix(engine) #5680: a dropped RID is reported, the no-type step is named, and a comment stops overselling

Review round: approve, all notes minor. Three taken.

- The "Exception, not Throwable" comment claimed more protection than the code gives. It guards the
  LOOKUP, but the shared per-record consumer still catches Throwable around the connectivity check
  itself - matching the type-wide path, which is why it stays. The comment now says exactly what is
  and is not covered rather than implying an Error can never be recorded as corruption.
- setRecords refused an all-null scope but dropped a PARTIAL one silently, so a caller who mistyped
  one of several RIDs got a clean report for a check that skipped it. Narrowing is still allowed -
  unlike widening it cannot do more than asked - but it now surfaces a warning naming how many were
  dropped, with a test.
- The no-type group advanced currentStep without ever naming the step, so a progress poller watched
  it go by blank. Routed through stepBegin like the document arm, which also owns the bump.

Not taken:

- Rejecting RECORD together with COMPRESS for symmetry with the TYPE/BUCKET rejection. The two are
  not the same case: RECORD plus TYPE/BUCKET has no coherent meaning, whereas "check this record,
  then compress the database" is perfectly meaningful - it is just not cheap. That is a
  documentation problem, and the release note now states it.
- @tag("slow") on the 200-edge tests. Measured rather than assumed: the whole class runs in 0.53s,
  and CLAUDE.md's criteria are multi-second elapsed or large batches. 200 tiny transactions under a
  second is neither.

* fix(engine) #5680: diagnose the clause conflict first, warn that COMPRESS is unscoped

Review round: refinements, not blockers. Four taken.

- CHECK DATABASE TYPE Hub RECORD {"@Rid": null} reported "none of them resolves to a RID", because
  the RIDs are resolved before check() reaches the combination guard - pointing the operator at the
  inner failure while the actual mistake is the clause combination. The statement now diagnoses the
  combination before resolving anything; check() keeps its own guard for programmatic callers.
- setRecords left droppedRecords stale on its early-return path. Harmless today, since the warning
  is only emitted under a record scope, but the field is public-API-visible state and should not
  depend on call order.
- COMPRESS now warns that it is not limited by the RECORD scope. The review suggested this as a
  middle ground between refusing the combination and staying silent, and it is a better answer than
  the documentation-only one: the command still does what it was asked, but an operator who named a
  record and expected a bounded run is told otherwise. Tested both ways, so the warning tracks the
  clause rather than the scope.
- Recorded, not fixed: the type-wide checkDocuments adds to the warnings/corrupted sets without
  incrementing the corresponding totals, so a corrupt document reports different totals depending on
  which path found it. The scoped arm here is the correct one; aligning the type-wide path is a
  behaviour change to a command this issue does not otherwise touch.

Agreed with the review that the duplicated catch blocks read better than threading six accumulators
through a shared helper; no change.

* fix(engine) #5680: one owner for the conflict message, no counts a Set cannot know, bounded corrupted set

Review round: nothing blocking. All four points taken.

- The "cannot be combined with TYPE or BUCKET" text was written out in both the statement and the
  checker. The two-layer guard stays - the statement has to diagnose the clause conflict before the
  RIDs are resolved - but the wording now lives in one constant so the two cannot drift.
- Two messages quoted a record count the Set could not know: several RID expressions that all
  resolve to null collapse to a single null element, so "given 3 record(s)" was never reachable and
  "1 of the record(s) did not resolve" under-reported. Rather than plumb the original count through,
  both messages stop quoting a number, which is accurate whatever the input.
- checkScopedDocuments grew corruptedRecords without a bound while every other path honours
  maxCorrupted/maxWarnings. Bounded the same way, with the total still counting every occurrence.
- Documented that the edge sweep a rebuild needs runs once per distinct vertex TYPE named, not once
  per run: ten vertices of one type cost one sweep, one vertex of each of three types cost three.

(cherry picked from commit 56b47f4)
robfrank pushed a commit that referenced this pull request Aug 14, 2026
…airs it, and the conflict keeps its cause (#5767)

* fix(engine) #5764: a refused vertex delete names the command that repairs it, and the conflict keeps its cause

Follow-ups from #5680, whose two halves landed separately: #5707 made deleteVertex strict, #5710 added the
CHECK DATABASE RECORD scope. Neither is defective; the seams between them were never closed.

1. The repair advice now names the scoped command with the RID substituted. Split in two, which the report did
   not distinguish: the RECORD scope is the right answer only where the delete was REFUSED and the vertex is
   still there to repair. The force arms log AFTER the record is gone, so a scoped check would name a record
   that no longer exists - they keep the whole-database form and say what the scoped form buys when run first.
   And at the other-end conflict the broken list belongs to the NEIGHBOUR, so that is the RID named.
   The previously bare rethrow now carries the advice instead of only getEdgeHeadChunkForWrite's
   "concurrent commit in flight", which is the diagnosis for the transient case nobody ever sees.

2. ConcurrentModificationException gains the (message, cause) constructor NeedRetryException already declared.
   Eight sites across GraphEngine, EdgeLinkedList and StripedEdgeList kept the message and dropped the stack.
   A conflict is normally absorbed by the retry, so the run that surfaces one is the retry-exhausted run -
   exactly the one whose trace has to be diagnosable. The class on the wire is unchanged.

3. checkDocuments and checkScopedDocuments report a corrupt document identically. The type-wide arm added to
   the warning and corrupted-record SETS without touching totalWarnings/totalCorruptedRecords, called a
   document a "vertex" it was "removing" when nothing on that path removes anything, and honoured no
   maxWarnings cap at all. Both now go through shared addWarning/addCorrupted.

4. The scoped vertex check running one pass where the type-wide runs two is NOT a gap: lookupByRID IS the
   raw-view materialisation (same newImmutableRecord(type, rid, bucket.getRecord(rid).copyOfContent())) and
   checkConnectivity opens with the same asVertex(true) that forces the decode, while getRecord and scan
   resolve placeholders and multi-page chains identically. Documented and pinned by a test rather than
   "fixed" by adding a second pass, which would double the cost of the scope that exists to be cheap.

The advice tests execute what the message says: the command is parsed out of the raised exception and run
verbatim, then the delete must succeed - so a wrong RID or a shape the parser rejects fails here.

* review: state addCorrupted's dedup guarantee honestly, and drop the reflection in the advice test

Three points from the code review on #5767, all minor and none blocking.

1. addCorrupted's "a record flagged twice is one corrupted record" holds only while the retained set is under
   the cap. Behaviour kept (the contains() check past the cap is strictly better than an unconditional
   increment); the Javadoc now says exactly what survives the cap and why closing the gap is not worth it - the
   only exact fix is a second unbounded set of every RID ever counted, i.e. the memory the cap exists to refuse.
   Also recorded that neither caller reaches it: the bucket scan visits each RID once and the RECORD scope is a
   Set, so a duplicate cannot survive as far as groupRecordsByType.

2. The reviewer flagged that the retained set is capped with maxWarnings while the graph arms take a separate
   maxCorrupted. Documented rather than changed: DatabaseChecker has no maxCorrupted field, and checkScopedRecords
   already passes the remaining maxWarnings budget into the GraphDatabaseChecker parameter of that name. Two
   names, one setting.

3. The advice test reached the LogManager logger by reflecting on the private field. LogManager exposes a public
   getLogger() for precisely this case - a caller that swaps the logger and has to put the original back - so the
   reflection helper is gone along with its rename fragility.

* review: record the maxWarnings/maxCorrupted coupling at the field, and why the interpolated cause is not redundant

Three non-blocking observations from the second review on #5767. All three resolve to recording constraints a
future reader would otherwise have to rediscover - one of them by breaking a test.

1. maxWarnings caps the corrupted set while GraphDatabaseChecker takes a maxCorrupted for the SAME set. The
   method Javadoc said "two names, one setting"; the note now lives on the field, where a reader meets it, and
   adds the consequence the reviewer pointed out: retained corruptedRecords become affectedBuckets, so a record
   dropped past the cap does not get its bucket's indexes rebuilt under FIX.

2. The advice test swaps the process-wide LogManager logger. Restored in a finally and safe under sequential
   class execution, but it is shared state; documented as a latent constraint if class-level parallelism is ever
   enabled, along with why there is no per-invocation seam to capture through instead.

3. getEdgeHeadChunkForWrite and hasNextEdgeToDelete interpolate e.getMessage() AND keep e as the cause, which
   reads as redundant. It is not: the missing CHUNK's RID appears ONLY in the record-not-found message - the
   surrounding text names the vertex - and the top-level message is what a log line and an HTTP error body carry,
   while the cause chain surfaces only in a full trace. Removing it would drop the RID and fail
   Issue5670EdgeDeleteDanglingBackRefTest, which asserts on it. Commented at both sites.

* review: guard the advice against a message-less conflict, and record why otherEndOf can equal the vertex

Two of the three optional-polish points from the third review on #5767 (which was LGTM).

1. withRepairAdvice built the message as e.getMessage() + advice. Every conflict on this path carries a message
   today, but a message-less one would render "null. If it persists...", which reads as a bug in the advice rather
   than as a missing diagnosis. Falls back to toString(), i.e. the class name.

2. otherEndOf would answer vertexRID itself for a self-loop, which the method name reads as impossible. That is
   the correct answer - a self-loop has no other end and the list that failed to disconnect it IS the vertex's own
   - so the equality is documented rather than "simplified" away. Also recorded why there is no test: the caller
   runs only after both of the vertex's lists have been collected, and a self-loop is read out of those same two
   lists, so a list broken enough to fail the disconnection fails the collection first.

Not taken: the reviewer flagged that otherEndOf is computed before the force branch, so it runs on every retry of
a transient conflict. Both branches need the value, so moving it saves nothing - and the reviewer scoped it as
"for completeness" on an exception path where asEdge() is light.

(cherry picked from commit c7349f8)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

deleteVertex can delete a vertex without collecting its edges when a chunk is transiently unreadable (the #5670 window, vertex side)

1 participant