Trace grace subtrees on the legacy heap, and gate heap integrity in CI

vm/benchmarks/README.md

@@ -120,7 +120,22 @@ runs, then goes quiet across the next cycle. Gate:
 The fresh-page-stack grace scheme this audit was written against reported
 100-370 missed slots and 100-250 doomed children per cycle; the
 `gcAllocedSinceSweep`-pruned registry walk reports zero doomed across the
-suite. `StormAB` (sustained single-thread storm) and `LoadLoop` (repeated
+suite.
+
+`LegacyGrace` is the same hazard on the OTHER heap: objects above
+`CN1_BIBOP_MAX_OBJECT` never reach the page heap, so the page walk cannot see
+them, yet the legacy sweep grants them the identical one-cycle grace. It
+reports through `[GRACE-AUDIT-LEGACY]` lines with the same contract. Two things
+in that driver are load-bearing and easy to get wrong when writing a new one:
+the hazard must be built in a window with **no mark in flight** (during a mark
+the SATB barriers cover the very reference move being tested, so a driver that
+keeps the collector busy tests nothing), and the driver must **scrub its own
+native stack** afterwards, because the conservative root scan marks whatever a
+returned frame's leftover words still point at -- an un-scrubbed driver pins
+the hazard it just built and comes back green. `CN1_GC_VERIFY_CENSUS` (below)
+is how you confirm the hazard set actually ages instead of being retained.
+
+`StormAB` (sustained single-thread storm) and `LoadLoop` (repeated
 dictionary build/drop) are the matching wall-time/RSS A/B drivers.
 `TimerLatency` is the sleep/timer fidelity probe for the Thread.sleep
 signal-truncation defect (a single usleep was EINTR'd by the collector's
@@ -143,6 +158,81 @@ host JVM by design — ParparVM's initialization-failure semantics differ): a
 throwing `<clinit>` must release the class-init monitor so other threads
 don't deadlock. Build and run it directly with `translate-and-build.sh`.
 
+## Heap-integrity gate (`-DCN1_GC_VERIFY`, `run-gc-verify.sh`)
+
+The gauntlet proves the VM computes the right answer. It cannot prove the
+collector left the heap in a legal state, and that is the failure mode this
+subsystem actually has: a dangling reference reads whatever object recycled the
+slot, so nothing diverges at the point of the bug -- the damage surfaces later,
+somewhere else, as corrupted data (issue 5425's "non word" dictionary entries
+and its impossible NPE). Checksums are structurally blind to it.
+
+`-DCN1_GC_VERIFY` compiles in a QA-only mode that makes the invariant directly
+observable, by destroying the plausible replacement object:
+
+- **Poison.** Every reclaimed page slot and every legacy block the sweep frees
+  is stamped with a poison header and payload. This includes the O(1) all-dead
+  page reclaim, which is where nearly all page memory is actually reclaimed and
+  which normally drops a page without writing a single slot -- leaving every
+  dead object with an intact-looking header. That is exactly why a dangling
+  reference in this VM reads plausible data rather than crashing.
+- **Quarantine.** Freed legacy blocks go to a ring (`CN1_GC_VERIFY_QUARANTINE`,
+  default 65536) instead of back to the C allocator, so a poisoned block stays
+  mapped and recognizable rather than being reused or unmapped underneath a
+  dangling reference.
+- **Verify.** After every sweep, each surviving object is walked through its
+  own generated mark function with the collector in verify mode: instead of
+  marking, every reference field is classified against the page registry, the
+  live-extent index and the quarantine set. A field pointing into a freed slot,
+  a recycled page, or a quarantined block is reported with the holder's class,
+  the victim's class, the mark call site, and then aborted -- at the cycle that
+  created it.
+
+```bash
+./run-gc-verify.sh            # tortures + drivers + the self-test
+./run-gc-verify.sh GraceAudit # one driver
+```
+
+The gate holds CURRENT-EPOCH survivors to the invariant: the sweep either
+marked them reachable (marking traces children, so a dangling field is a
+mark-completeness bug) or promoted them by the grace rule (tracing the subtree
+was the grace pass's job). Both readings make a dangling field unambiguous, so
+the gate has no judgement calls in it.
+
+**The self-test is the point of the script.** A gate nobody has watched fail is
+not a gate, so the run finishes by re-injecting the exact defect #5442 fixed
+(`CN1_GC_FAULT=nograce` disables the grace-subtree pass, reproducing #5436) and
+requires the verifier to catch it. It does, immediately:
+
+```
+[GC-VERIFY] DANGLING REFERENCE after sweep at epoch 15
+            holder  = 0xd38070050 class=com.bench.GraceAudit.Node mark=15 (epoch+0) heapPos=-3
+            field   -> 0xd3806cfe0 class=com.bench.GraceAudit.Node mark=-7 heapPos=-3
+            victim  = RECYCLED page slot (above bump cursor) (page-resident)
+```
+
+### Diagnostics
+
+| variable | effect |
+|---|---|
+| `CN1_GC_VERIFY_SOFT=1` | report every violating cycle instead of aborting on the first |
+| `CN1_GC_VERIFY_LOG=1` | one line per cycle even when clean (holders, refs checked, reclaim counts, referenced-child age histogram) |
+| `CN1_GC_VERIFY_AGING=1` | ALSO hold previous-epoch survivors to the invariant. Not a gate -- unreachable objects are entitled to dangle -- but a census of landmines, because this collector resurrects unreachable objects routinely (see `CN1_GC_TRACE_MARK`) |
+| `CN1_GC_VERIFY_ALL=1` | walk every object, including ones already given up on. Investigation only |
+| `CN1_GC_VERIFY_CENSUS=<class>` | per-cycle age histogram of every resident object of a class. Use it to confirm a driver's hazard set actually ages out instead of being pinned |
+| `CN1_GC_VERIFY_DUMP=<class>` | print holder/child marks for holders of a class |
+| `CN1_GC_TRACE_MARK=<class>` | name the mark pass that re-marks a class each cycle -- the answer to "what is still keeping this alive?", most often `conservative-native-stack`. Add `-DCN1_BIBOP_VALIDATE` to also get the drain parent |
+| `CN1_GC_FAULT=nograce` | fault injection: disable the grace-subtree pass |
+
+The mode is deliberately asymmetric about uncertainty: a reference it cannot
+place (allocated after the snapshot, unmapped, mid-construction body) is
+skipped. It reports only what it can prove, so a violation is never a false
+alarm, at the cost of catching some real defects a cycle later than it could.
+
+Cost is a full extra heap walk per cycle plus poison writes, so it is a
+correctness gate, never a perf configuration -- run it alongside the gauntlet,
+not with it.
+
 ## Mandatory compiler flags
 
 Generated C **must** be compiled with

vm/benchmarks/run-gc-verify.sh

@@ -0,0 +1,86 @@
+#!/bin/bash
+# ParparVM heap-integrity gate (-DCN1_GC_VERIFY).
+#
+# The gauntlet proves the VM COMPUTES the right answer. This proves the
+# collector leaves the heap in a legal state: after every sweep, no object the
+# sweep kept may reference memory the sweep reclaimed. Checksums cannot see that
+# invariant break -- a dangling reference reads whatever object recycled the
+# slot, so the damage surfaces later, somewhere else, as corrupted data rather
+# than as a wrong answer (issue 5425). Here it aborts at the cycle that caused
+# it, naming the holder class, the victim class and the field's mark call site.
+#
+#   run-gc-verify.sh              # full gate: tortures + drivers + self-test
+#   run-gc-verify.sh GraceAudit   # one driver
+#
+# Requirements: JDK_8_HOME, Maven, clang.
+set -e
+cd "$(dirname "$0")"
+J8="${JDK_8_HOME:?set JDK_8_HOME}"
+mkdir -p target/bin
+
+# This script's result is decided by these variables, and anyone debugging the
+# collector has them exported. An inherited CN1_GC_FAULT would fail every
+# driver below; an inherited CN1_GC_VERIFY_SOFT would stop the self-test from
+# aborting. Either inverts a result instead of failing loudly.
+unset CN1_GC_FAULT CN1_GC_VERIFY_SOFT CN1_GC_VERIFY_AGING CN1_GC_VERIFY_ALL \
+      CN1_GC_VERIFY_LOG CN1_GC_VERIFY_CENSUS CN1_GC_VERIFY_DUMP CN1_GC_TRACE_MARK
+
+# Every workload that allocates enough to drive real collection cycles. The
+# point is coverage of ALLOCATION SHAPES, not of answers: page-heap churn,
+# monitors, finalizers, threads, oversized/legacy objects, adopted survivors.
+DRIVERS="${*:-GraceAudit LegacyGrace GcStress MtStress MapTorture SbTorture FusedTest ThreadChurn LargeArrayLoad}"
+
+fail=0
+for d in $DRIVERS; do
+    printf '%-16s ' "$d"
+    if ! ./translate-and-build.sh "$d" "target/bin/$d-verify" -DCN1_GC_VERIFY > target/bin/$d-build.log 2>&1; then
+        echo "BUILD FAILED"
+        tail -25 "target/bin/$d-build.log"
+        fail=1
+        continue
+    fi
+    if out="$(./target/bin/$d-verify 2>&1)"; then
+        # passes=0 means the workload never finished a collection cycle, so the
+        # verifier never ran and "no violations" would mean only that nothing
+        # was ever checked. Treat a vacuous pass as a failure -- that is the
+        # same hollow-gate problem the self-test below exists to prevent.
+        passes="$(printf '%s' "$out" | sed -n 's/.*SUMMARY passes=\([0-9]*\).*/\1/p' | tail -1)"
+        if printf '%s' "$out" | grep -q 'GC-VERIFY. DANGLING'; then
+            echo "FAILED (verifier reported a dangling reference)"
+            printf '%s\n' "$out" | grep -A 4 'DANGLING' | head -20
+            fail=1
+        elif [ -z "$passes" ] || [ "$passes" -eq 0 ]; then
+            echo "FAILED (vacuous: 0 verify passes -- the workload never completed a GC cycle)"
+            fail=1
+        else
+            echo "clean ($passes verify passes)"
+        fi
+    else
+        echo "FAILED (exit $?)"
+        printf '%s\n' "$out" | tail -25
+        fail=1
+    fi
+done
+
+# SELF-TEST. A gate nobody has watched fail is not a gate: re-inject the defect
+# #5442 fixed (grace-subtree pass disabled, the #5436 behavior) and require the
+# verifier to catch it. If this run comes back clean the gate above is inert and
+# a green result from it means nothing.
+printf '%-16s ' "self-test"
+# The faulted run is EXPECTED to abort. Capture it in a command substitution so
+# the shell does not print its own job-control notice for the SIGABRT -- a line
+# that reads like a failure sitting next to a passing gate is how people learn
+# to ignore the gate.
+faultOut="$(CN1_GC_FAULT=nograce ./target/bin/GraceAudit-verify 2>&1)" && faultExit=0 || faultExit=$?
+if [ "$faultExit" -eq 0 ]; then
+    echo "BROKEN -- injected grace-pass fault was NOT detected"
+    fail=1
+elif printf '%s' "$faultOut" | grep -q 'DANGLING REFERENCE'; then
+    echo "detected the injected grace-pass fault ($(printf '%s' "$faultOut" | grep -c 'DANGLING REFERENCE') reports)"
+else
+    echo "BROKEN -- faulted run died (exit $faultExit) without a verifier report"
+    printf '%s\n' "$faultOut" | tail -20
+    fail=1
+fi
+
+[ "$fail" -eq 0 ] && echo "GC-VERIFY GREEN" || { echo "GC-VERIFY FAILED"; exit 1; }

vm/benchmarks/src/com/bench/GcStress.java

@@ -1,3 +1,26 @@
+/*
+ * Copyright (c) 2012, Codename One and/or its affiliates. All rights reserved.
+ * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
+ *
+ * This code is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 only, as
+ * published by the Free Software Foundation.  Codename One designates this
+ * particular file as subject to the "Classpath" exception as provided
+ * by Oracle in the LICENSE file that accompanied this code.
+ *
+ * This code is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
+ * version 2 for more details (a copy is included in the LICENSE file that
+ * accompanied this code).
+ *
+ * You should have received a copy of the GNU General Public License version
+ * 2 along with this work; if not, write to the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
+ *
+ * Please contact Codename One through http://www.codenameone.com/ if you
+ * need additional information or have any questions.
+ */
 package com.bench;
 
 import java.util.HashMap;
@@ -64,5 +87,18 @@ public static void main(String[] args) {
             System.out.println("ROUND " + round + " checksum=" + c);
         }
         System.out.println("DONE " + c);
+
+        // Give the collector one complete cycle over the heap this workload
+        // built. This driver's own churn triggers collections, but whether the
+        // last one reaches its sweep before the process exits is a race -- so a
+        // -DCN1_GC_VERIFY build verified 1 cycle or 0 depending on the run.
+        // Prints nothing, so the byte-identical comparison against the host JVM
+        // is unaffected.
+        System.gc();
+        try {
+            Thread.sleep(250);
+        } catch (InterruptedException ignored) {
+            Thread.currentThread().interrupt();
+        }
     }
 }