[Bug] ShortestPathTraverser does not close edge iterators on early return

[contrueCT]

Bug Type (问题类型)

logic / resource lifecycle

Before submit

• I searched the existing issues, pull requests, and FAQ and did not find a report for this exact ShortestPathTraverser iterator leak.

Environment (环境信息)

• Server source: current apache/hugegraph master (431f6e6b431014feb5b50d27ea67180b4610f10b)
• Backend: HStore, one PD + one Store + one Server on one machine
• OS: Ubuntu 24.04, 16 vCPU, about 78 GiB RAM, swap disabled during the reproduction
• Data size: 41,652,230 vertices and 1,468,365,182 directed edges (SNAP Twitter-2010)
• Workload: REST shortestpath, concurrency 12, 180-second warmup + 600-second measurement

Expected & Actual behavior (期望与实际表现)

Every edge iterator returned by edgesOfVertex() should be closed on normal completion, early return, and exception paths. For HStore, closing the iterator is also the signal that allows the remote scanner stream to be cancelled and cleaned up.

ShortestPathTraverser.Traverser does not currently guarantee that ownership contract:

1. forward() returns immediately when it finds the requested path without closing the current edge iterator.
2. backward() has the equivalent early-return path.
3. superNode() passes an edge iterator to IteratorUtils.count() without closing it explicitly.

In the HStore reproduction, successful requests left Store threads waiting in the scanner delivery path (HgChannel.send). The blocked frames accumulated from 11 shortly after launch to 38, 75, 108, and finally 113. The measured phase completed 5,817 of 5,818 requests; one request returned HTTP 500 after approximately 61 seconds. Swap remained zero, so the failure was not caused by swapping.

As a diagnostic control, reducing the Store client-take wait caused abandoned scanner senders to be reaped instead of accumulating, but it also produced 107 timeout warnings and reduced throughput. This confirms the scanner lifecycle path; increasing or reducing timeouts is not a fix.

A focused regression test uses close-tracking edge iterators while keeping the path assertions unchanged. On unpatched master, all three path results are correct and all three close assertions fail:

Tests run: 3, Failures: 3

The proposed fix retains the original iterator returned by edgesOfVertex() and closes it with CloseableIterator.closeIterator() in finally blocks. The original iterator must be retained because skipSuperNodeIfNeeded() may consume and replace it. superNode() receives the same explicit finally cleanup. Query semantics, limits, and timeouts do not change.

With the patch, the focused test passes 3/3. A full-graph HStore validation then completed all 1,516 warmup and 5,791 measured requests with zero errors, zero swap, stable service PIDs, and zero HgChannel.send frames at every sampled point from 0 through 600 seconds.

Vertex/Edge example (问题点 / 边数据举例)

The problem is independent of a particular vertex or edge value. It is triggered when bounded bidirectional shortest-path traversal finds a path before the current backend iterator has been exhausted.

Schema [VertexLabel, EdgeLabel, IndexLabel] (元数据结构)

schema.vertexLabel("user")
      .useCustomizeNumberId()
      .ifNotExist()
      .create()
schema.edgeLabel("link")
      .sourceLabel("user")
      .targetLabel("user")
      .ifNotExist()
      .create()

[dosubot]

Good catch — the bug is confirmed in the current master source. In ShortestPathTraverser.Traverser:

1. forward() (lines 209–210): when !all and a path is found, it returns immediately without closing the edges iterator obtained from edgesOfVertex() [1].

2. backward() (lines 255–256): same early-return pattern, leaking the iterator [2].

3. superNode() (lines 271–273): passes an iterator directly to IteratorUtils.count(edges), which exhausts it but never closes it [3].

In both forward() and backward(), skipSuperNodeIfNeeded() may consume and replace the original iterator, so the original reference must be retained for closing [4].

The fix pattern is well-established in the codebase — CountTraverser wraps its edge loop in try/finally { CloseableIterator.closeIterator(edges); } [5], and PR #1643 applied the same idiom to QueryResults.toList. The same approach applies here:

// forward() / backward()
Iterator<Edge> edgesOrig = edgesOfVertex(source, this.direction, this.labels, degree);
Iterator<Edge> edges = skipSuperNodeIfNeeded(edgesOrig, this.degree, this.skipDegree);
try {
    while (edges.hasNext()) {
        ...
        if (!all) {
            return paths;   // early return; finally still runs
        }
    }
} finally {
    CloseableIterator.closeIterator(edgesOrig);
}

// superNode()
Iterator<Edge> edges = edgesOfVertex(vertex, direction, this.labels, this.skipDegree);
try {
    return IteratorUtils.count(edges) >= this.skipDegree;
} finally {
    CloseableIterator.closeIterator(edges);
}

Note that edgesOrig (not edges) must be closed because after skipSuperNodeIfNeeded() the variable edges may point to a new wrapping iterator, while the underlying backend iterator is still rooted in edgesOrig.

Your reproduction evidence — accumulating HgChannel.send frames correlating with un-exhausted iterators, cleared by the patch — is very convincing. Would you like to open a PR with the fix?

To reply, just mention @dosu.

---

Docs are dead. Just use Dosu.