Add exitAndClearTask(), backed by Android's finishAndRemoveTask() - #5746
Merged
Conversation
Exiting a Codename One application has only ever meant exitApplication(), which on Android kills the process and leaves the task sitting in the recents list -- the user can pick the app back out of the task switcher, and on a relaunch from there the system restores a task the application believed it had ended. Applications that need a real sign-out or kiosk style exit had no way to ask for that. Display.exitAndClearTask() (and the CN static mirror) adds it. The shape follows the existing exit()/exitApplication() pair exactly: the new implementation entry point exitAndClearTask() runs the setOnExit callback and then calls the port hook exitApplicationAndClearTask(), whose base implementation is a plain exitApplication(). The fallback is therefore structural rather than conditional -- every port that does not override the hook keeps today's behaviour, and none of them needed touching. isExitAndClearTaskSupported() lets an application branch when the difference matters; it answers false everywhere but Android. The Android override calls Activity.finishAndRemoveTask() on the UI thread and then kills the process the way exitApplication() does. It degrades to a plain exit when there is no activity -- a push or background service process owns no task of its own -- and below API 21, where the platform method does not exist. Killing the process immediately after finishAndRemoveTask() looks like it could race the removal, so it was measured rather than assumed, with a throwaway app running that exact sequence on an API 36 emulator: the task was gone from "dumpsys activity recents" in all 29 trials where the trigger arrived, while the control that only killed the process left it there every time. The finding is recorded beside the call. ExitAndClearTaskTest pins the three contract points -- an unsupported port degrades to exitApplication(), a supporting port does not also run it, and setOnExit fires on both paths. Removing the fallback body makes the first test fail, so it is not vacuous. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 722eb63040
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| Runnable finishAndKill = new Runnable() { | ||
| public void run() { | ||
| try { | ||
| a.finishAndRemoveTask(); |
There was a problem hiding this comment.
When the exported CN1 activity is opened by another activity without FLAG_ACTIVITY_NEW_TASK—for example through an android.xintent_filter deep link—the generated singleTop activity can be placed on the caller/browser's task. Calling finishAndRemoveTask() here then finishes and removes that entire foreign task, destroying the caller's back stack rather than merely clearing this application; handle the non-task-root case by finishing only the CN1 activity instead.
Useful? React with 👍 / 👎.
Contributor
|
Developer Guide build artifacts are available for download from this workflow run:
Developer Guide quality checks: |
Contributor
Cloudflare Preview
|
Contributor
✅ Continuous Quality ReportTest & Coverage
Static Analysis
Generated automatically by the PR CI workflow. |
Collaborator
Author
|
Compared 151 screenshots: 151 matched. Native Android coverage
✅ Native Android screenshot tests passed. Native Android coverage
Benchmark ResultsDetailed Performance Metrics
|
Collaborator
Author
|
Compared 181 screenshots: 181 matched. |
Collaborator
Author
|
Compared 217 screenshots: 217 matched. |
Collaborator
Author
|
Compared 148 screenshots: 148 matched. Benchmark Results
Detailed Performance Metrics
|
Collaborator
Author
|
Compared 160 screenshots: 160 matched. Benchmark Results
Detailed Performance Metrics
|
Collaborator
Author
|
Compared 149 screenshots: 149 matched. Benchmark Results
Build and Run Timing
Detailed Performance Metrics
|
shai-almog
added a commit
that referenced
this pull request
Sep 8, 2026
cn1GcDiscoverReference now defers to gcMarkObject while a thread is inside its own nursery minor collection. cn1PromoteDrain runs the generated mark functions with nurseryPromoting raised, and gcMarkObject's promotion branch is what moves a referenced object out of the block being recycled -- routing the referent past it meant a surviving WeakReference could be promoted alone and left pointing into a reclaimed block. Nothing clears on that path, so the edge costs a promotion and nothing else. The nursery arm did not compile at all, here or on master: nativeMethods' bulk barrier calls cn1SatbBulkBegin unconditionally while the prototype sat in the #else of the CN1_NURSERY split, which clang rejects as an implicit declaration under C99. The load barrier added a second instance. Declared beside the deletion barrier instead, where the callers are. Two gates that could not fail: - RefPolicy printed WEAK_LIVE_KEPT and exited 0 whatever it said. A referent cleared while still strongly reachable is heap-SAFE -- null dangles nothing -- so run-gc-verify cannot see it either, and the checksum reads live[] directly. Both advertised validation paths could stay green while WeakReference silently emptied every cache built on it. It is an assertion now. - ab-refs.sh ignored the return code. Metrics print before exit, so a VM that corrupted its heap and died in an atexit handler still emitted RESULT and the whole table, and the harness published checksum-matched medians from a crashed run. The cast-semantics baseline names anonymous classes as Outer$N, so #5746 adding one to AndroidImplementation renumbered onReceive from $46 to $47 and left the entry stale. master is red on the gate today; this is the same cast, not a new one. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Collaborator
Author
|
Compared 144 screenshots: 144 matched. |
Collaborator
Author
|
Compared 143 screenshots: 143 matched. Benchmark Results
Build and Run Timing
Detailed Performance Metrics
|
shai-almog
added a commit
that referenced
this pull request
Sep 9, 2026
* Give ParparVM real weak and soft references
The collector had no notion of a weak root. java.lang.ref.WeakReference held
its referent in an ordinary field, so the translator emitted a gcMarkObject for
it and the referent was STRONG -- a "weak" reference pinned its referent for the
life of the process, and every cache built on
CodenameOneImplementation.createSoftWeakRef (the EncodedImage decode cache,
Image's scale and RGB caches, Border's round-rect cache, rasterised gradients)
was unbounded. There was no SoftReference at all.
The referent now lives in Reference and the translator does not trace it
(ByteCodeClass.isReferenceReferent): it emits cn1GcDiscoverReference, which
hands the collector the field addresses and decides soft retention on the spot.
Clearing happens in cn1GcProcessReferences, using the sweep's own liveness test
-- both halves of the sweep free on `mark != -1 && mark < currentGcMarkValue - 1`
and nothing else may be cleared, because a dangling read on this VM is a native
crash no Java catch can see.
Three things about the placement, none of them optional:
- The clear pass runs INSIDE the SATB termination loop, barrier still armed. A
thread scanned and released early can pull a referent out through get() and
hold it in a local the collector has walked past, and that referent is then
neither marked nor fresh -- the one case the sweep's "already marked or FRESH"
invariant does not cover. get() carries a load barrier, so a racing read makes
the trial clear of gcSatbActive find a non-empty log, which re-arms and re-runs
the fixpoint and this pass with it.
- That barrier is FILTERED (CN1_SATB_REF_LOAD). Logging every referent read is
not a cost but a failure: cn1SatbEnqueue takes a mutex per accepted reference
and get() on a hot cache is called far more often than any store barrier sees.
Unfiltered it put over 10,000 entries in the log per cycle and reached
CN1_SATB_MAX_REOPENS on EVERY cycle. Skipping referents already marked this
epoch or fresh -- exactly the ones the clear pass would refuse to clear -- took
passes 32 -> 1 and refMs 0.06 -> 0.005.
- Soft retention is ranked by age since the last get(), decided when the mark
first reaches the reference so the mark stays single pass. Deciding afterwards
would mean a second reachability closure over the retained set, on a mark that
already spends most of its time in the grace pass.
Measured (RefPolicy + ab-refs.sh, five interleaved reps, checksums identical
across every arm). References themselves are unambiguous: 255/256 unreachable
referents reclaimed against 0/256, for 1.8-4% of mark time, with a vm/benchmarks
geomean of 1.011 over 12 interleaved reps against master. The RANKING is a cheap
rider rather than the payoff -- against never-clearing it is the same hit rate
for about a megabyte. What it does beat decisively is the pressure-triggered
alternative, which gave up 12 points of hit rate for zero footprint saving and
was worse on both axes at a 160MB ceiling; that arm is the model of the iOS
port's didReceiveMemoryWarning -> flushSoftRefMap.
vm/CLAUDE.md carries the table and the three conclusions that were drawn from
single runs and turned out to be wrong, including the one that generalises: a
pressure-triggered cache policy cannot work on this collector, because the pacing
loop defends the reserve by throttling the mutator, so headroom converges on any
threshold placed there and never crosses it.
This change is confined to the VM. The porting layer, the iOS softReferenceMap
override and the core call sites are deliberately untouched and follow separately,
since they change behaviour in every iOS app and want their own bisect point.
Gates: run-gc-verify.sh green (RefPolicy clean over 47 verify passes, both
fault-injection self-tests firing), run-gauntlet.sh bit-identical to the host
JVM, vm/tests 552 tests / 0 failures, GC integration leg 10/10 including
GcSteadyStateIntegrationTest and ProcessBudgetPacingIntegrationTest.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Make reference field access atomic, and decide aliases together
Both from review of the previous commit, and both premises hold.
ATOMICITY. The collector cleared the referent with a plain store while the
generated accessor read it with a plain load, on threads that run concurrently
by design -- a data race, undefined in C however benign the emitted instruction
is on the targets built here. It was also an inconsistency within the same
function: the line below the clear already read the MARK WORD with an acquire
atomic, because cn1_globals.m converted its hand-written stores for exactly this
reason. Relaxed atomics now at every site that touches the referent: the
generated getter and setter (so Reference.clear() and the constructor are
covered), CN1_SATB_REF_LOAD, and the three collector paths. cn1TouchAge gets the
same treatment -- the mutator stamps it from the accessor while the collector
reads and ages it, which is the identical defect one field over.
ALIASES. Two references to one referent were decided at different instants, so a
get() landing mid-pass could stamp the second as recently read after the first
had been cleared -- one alias answering null while another answers the object.
The contract says all references to a weakly reachable object are cleared
atomically. For a cache a split is a spurious miss; for the callers that use a
reference as a LIFETIME ORACLE, reading a null get() as proof the referent died,
it is a false death report on one alias while the object is alive through
another. That is the failure mode that makes the iOS soft-reference table
dangerous today.
The clear pass is now two sub-passes: A marks every referent read this cycle and
drains, B then clears on the referent's mark word alone. Liveness is a property
of the referent, so every alias reads the same answer and they are cleared
together or kept together. This removes the possibility rather than narrowing the
window -- a get() racing sub-pass B still gets a non-null referent and still
enqueues it, so the object survives and every alias is cleared, which is a legal
spurious clear and is what "atomically" asks for.
RefPolicy grows an alias phase with a concurrent reader. Read the comment on it
before citing it: it is NOT a self-test. Built with the new
-DCN1_REF_NO_ALIAS_ATOMICITY arm, which restores the single-loop form that has
the bug, it still reports ALIAS_SPLIT=0/256 -- the window is microseconds and
could not be opened from a driver. Two earlier versions of that phase were worse
and are documented so they are not rebuilt: one read the aliases only after
quiescing, where every alias carries the same stamp and nothing can fail; the
other kept every referent marked, so ALIAS_CLEARED_GROUPS was 0/256 and the thing
being tested never happened. The fix ships because the two-sub-pass form is
unconditionally correct and simpler, not because a test proved the old one broken
-- the same footing as CN1_NO_BULK_INSERTION_BARRIER.
Gates: run-gc-verify.sh green (RefPolicy clean over 71 verify passes, both
fault-injection self-tests firing), run-gauntlet.sh green with all eleven
tortures bit-identical and both GC stress modes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Release soft referents on allocation failure, and stop losing touches
Both from review, and both premises hold.
ALLOCATION FAILURE. The retention ladder reads cn1ProcessHeadroom(), which
answers -1 wherever there is no per-process budget to probe -- every desktop
build and the simulator. There it could only ever say "plenty", so a soft
referent read at least once every CN1_REF_SOFT_RETAIN_MAX cycles was never
dropped however tight memory actually was, while codenameOneGcMalloc's failure
path only asked for a collection and retried. That breaks the one guarantee
SoftReference makes -- every soft reference cleared before the VM gives up -- and
turns recoverable pressure into a retry loop that collects nothing.
cn1RefDropAllSoftReferents() raises a latch on the failure path which the next
cycle consumes, overriding every policy including "never clear". A latch rather
than a direct write, because cn1RefBeginCycle recomputes the budget at the top of
each cycle and would erase one.
Proven, not argued: with failures injected while soft referents are live, the
emergency cycles report retained=0 and cleared=1097 against retained>0 on the
same workload uninjected. Getting that proof needed CN1_SIMULATE_ALLOC_FAILURES
to grow a "<n>:<skip>" form -- it could only fail the FIRST n allocations, which
is startup, and a state that exists only at startup cannot exercise anything the
program builds later. Every attempt without it produced real emergency cycles
that all reported discovered=0.
LOST TOUCHES. Ageing was a load, a compute and a store, so a get() landing
between the load and the store was erased outright -- the referent looked cold
with no record it had been read at all, and the age was not even reset. Worse, it
falsified a claim: consuming CN1_REF_TOUCHED at DISCOVERY meant the clear pass's
"was it read?" fallback only ever covered reads landing after discovery, not the
whole cycle as its comment implied.
The ageing therefore moves out of discovery to the end of the clear pass, and is
a compare-exchange. Any read anywhere in the cycle is now still visible to
sub-pass A, so the fallback covers what it says it does, and a racing touch can
no longer be lost -- a failed exchange reloads, sees CN1_REF_TOUCHED and resets
the age to 0, which is what the touch means.
Two consequences worth naming. The CN1_REF_NO_ALIAS_ATOMICITY arm had to start
ageing inline, or with discovery no longer doing it the arm would silently have
become "retain everything" rather than the shape it exists to reproduce. And
Reference.cn1TouchAge now starts at 0 rather than TOUCHED: TOUCHED means "read
since the collector last aged this", which is false for a reference nobody has
called get() on, and starting there made the clear pass mark the referent of
every newly discovered reference for a cycle, weak ones included.
Gates: run-gc-verify.sh green (RefPolicy clean over 71 verify passes, both
fault-injection self-tests firing), run-gauntlet.sh green with all eleven
tortures bit-identical and both GC stop modes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Retain a referent discovery could not record, and load it once
Two accepted review findings and one declined, plus a driver that had gone
vacuous again.
A DROPPED DISCOVERY MUST RETAIN. When the discovery list could not grow -- the
realloc failed, or growth was declined because a thread is signal-frozen -- the
entry was dropped AND the referent left unmarked. The comment above it claimed
that was safe because "the referent stays reachable through a field nothing
cleared", which is exactly backwards and is the kind of sentence that reads as
obviously true: nothing else marks a weak referent, that being the point of a
weak edge, so the sweep frees it and the uncleared field becomes a dangling
pointer inside a perfectly reachable Reference, handed to the next get(). On this
VM that is a native crash no Java catch can see, on a path only ever taken when
memory is already short. The drop path now marks the referent, which costs one
deferred reclaim.
ONE LOAD IN THE ACCESSOR. CN1_SATB_REF_LOAD only read the field INSIDE its
gcSatbActive branch, so with the barrier down the accessor loaded again
afterwards and the gap between the two was a hole: a thread that read the flag as
0, was SIGUSR2-frozen with the referent not yet in any register, scanned,
released, and only then loaded, came away holding an unmarked referent nothing
had enqueued. The accessor now loads once into a local and the barrier
(CN1_SATB_REF_KEEP) acts on that same value, so what get() returns is what the
barrier saw -- and with the load first the value is in a register before any
freeze, where the conservative root scan finds it.
ATOMIC PUBLICATION OF ALIAS CLEARING IS DECLINED, and the reasoning is in the
code at the store rather than only here. Making N stores visible as one step
needs a lock that Reference.get() also takes, and get() is the single hot path
this design exists to keep free of one; HotSpot does not do it either, clearing
referents one at a time without synchronising against get(). The contract's
requirement is that the DECISION covers every alias together, which the two
sub-passes provide and which was the part genuinely broken before them. The
residual window is transient and self-healing -- a get() inside it arms the
barrier and resurrects the object.
The alias phase had gone hollow again after ageing moved to the clear pass: with
the stamp surviving the whole cycle, a reader touching one alias of EVERY group
kept every group alive and ALIAS_CLEARED_GROUPS read 0/256, which is the vacuum
that phase exists to detect. It now reads one group in four, and collection is
back to 191/256.
Gates: run-gc-verify.sh green (RefPolicy clean over 71 verify passes, both
fault-injection self-tests firing), run-gauntlet.sh green with all eleven
tortures bit-identical and both GC stop modes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Process references to a fixpoint, and register the get() barrier
Two accepted review findings, both real, both about the same thing: the
reference pass assumed it could see the whole picture at one instant.
DISCOVERY IS NOT DONE WHEN THE PASS STARTS. cn1GcProcessReferences snapshotted
cn1RefDiscoveredTop once and cleared against that snapshot -- but sub-pass A
DRAINS, and a drain discovers references. Marking a touched referent traces it,
and an object kept alive only by that reference can itself hold weak references
whose mark functions call cn1GcDiscoverReference; those landed past the snapshot
and sub-pass B never looked at them. The result is the failure this whole design
exists to prevent: a reachable Reference nothing cleared, holding a referent the
sweep freed. Sub-pass A now repeats until a drain adds nothing new, which
terminates because the set only grows and is bounded by the live set.
THE BARRIER HAD TO JOIN THE TERMINATION HANDSHAKE. Checking gcSatbActive and then
calling cn1SatbEnqueue is not enough here: the enqueue takes a mutex, so a thread
can pass the check, be delayed acquiring it, and land its entry in a log the
collector has already stopped draining -- after which the referent it is about to
return gets swept. The per-store barrier accepts precisely that window, and the
argument it accepts it on does NOT extend to this path: "a reference stored after
the fixpoint is already marked or FRESH" is true of a store and false of a weak
referent handed out by get(), which is neither. cn1SatbBulkBegin already registers
before it answers, and gcSatbTerminating stays raised across the whole termination
loop including reference processing, so reusing that handshake closes it with the
machinery already present. Off-GC the fast path is unchanged: two
predicted-not-taken flag loads.
Also restores the CN1_REF_NO_ALIAS_ATOMICITY arm, which the sub-pass rewrite had
silently taken with it -- RefPolicy's alias phase documents that flag, so losing
it would have left a comment describing a build that no longer existed.
Gates: run-gc-verify.sh green (RefPolicy clean over 71 verify passes, both
fault-injection self-tests firing), run-gauntlet.sh green with all eleven
tortures bit-identical and both GC stop modes. The emergency soft-drop still
fires under injected allocation failure (retained=0 at softBudget=-1).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Let the emergency clear soft referents without the discovery list
Accepted review finding, and it is two of this branch's own fixes colliding at
the one moment both were written for.
The emergency retention budget is raised by an allocation FAILURE. An allocation
failure is also the likeliest reason the discovery list's realloc cannot grow. So
the conservative fallback added for weak-reference safety -- mark anything that
could not be recorded -- was marking exactly the soft referents the emergency had
just condemned, the collection freed nothing, and codenameOneGcMalloc's retry
loop had nothing to make progress against. Each fix is right on its own; together
they livelock.
A soft reference the emergency has condemned needs no list entry, because the
decision is already final rather than deferred. Its field is cleared in place,
which is the allocation-free path this situation calls for, and it is safe for
the same reason the ordinary clear is: a get() that already loaded the referent
enqueued it through the armed barrier and keeps it alive for the cycle, and a
get() after the store reads null.
Weak references, and anything read since the last ageing, still take the marking
branch -- that is where a dangling pointer would actually come from, since a
mutator may be holding the referent in a local the collector has walked past.
The accepted cost is stated at the code: two aliases of one SOFT referent can
disagree when only some of them were recorded. That is confined to soft
references, which are caches by definition, and the alternative is an allocator
that cannot make progress. The callers that read a null get() as proof of death
use weak references, which mark.
Gates: run-gc-verify.sh green (RefPolicy clean, both fault-injection self-tests
firing), run-gauntlet.sh green with all eleven tortures bit-identical and both GC
stop modes, and the emergency drop still fires under injected allocation failure.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Hold the SATB registration across the referent load
Two accepted review findings. The P1 is this branch's own optimisation undoing
its own fix, which is worth naming plainly.
REGISTRATION MUST SPAN THE LOAD. The previous commit made the get() barrier
register through cn1SatbBulkBegin, whose entire value is that it registers FIRST
and reports whether logging is needed afterwards -- and then gated entry to it on
an outer read of the same two flags, as a fast path. That puts the race back one
level out: a thread can be descheduled between the outer read and the
registration, and the collector can clear the field, finish its empty final take,
lower gcSatbTerminating and quiesce with the in-flight count still zero. The
enqueue is then declined and the sweep frees the referent get() is about to
return. Nothing may be sampled before the registration, so the accessor now
brackets the whole load: CN1_REF_LOAD_BEGIN, load, CN1_SATB_REF_KEEP,
CN1_REF_LOAD_END.
The cost is two seq_cst read-modify-writes on every Reference.get(), paid
unconditionally, and there is no sound way to skip them: any flag consulted
before registering can go stale in exactly the window the registration exists to
close. With it held across the load a false answer is safe rather than merely
unlikely -- the collector cannot be mid-termination because its quiesce waits for
this registration, so either no mark is running and a later one scans this thread
with the value already in a register, or reference processing is complete and a
field still holding a pointer was not condemned.
THE OOM FALLBACK MUST NOT CLEAR A STRONGLY REACHABLE REFERENT. The emergency
clear added last commit did not consult the mark word, so an application holding
both an ordinary field and a SoftReference to one object could watch get() answer
null under allocation pressure for an object that was never softly reachable. It
now skips anything already marked. Partial by construction, and said so at the
code: the mark is still running there, so a referent a strong edge reaches LATER
in the cycle is not yet marked and can still be cleared. Being certain would mean
deferring to the clear pass, which is precisely what that path exists because it
cannot do.
Gates: run-gc-verify.sh green (RefPolicy clean over 69 verify passes, both
fault-injection self-tests firing), run-gauntlet.sh green with all eleven
tortures bit-identical and both GC stop modes. Both the -DCN1_DISABLE_SATB and
default arms build and run; an earlier revision of this commit left the header's
conditional nesting unbalanced, which is why that arm is now checked explicitly.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Finish the referent's atomic accesses, and stop clearing after a dropped log entry
Two accepted review findings, both leftovers of earlier fixes on this branch
rather than new ground.
HALF A RACE WAS FIXED. Making the referent's accesses atomic converted the getter
and the setter's own store but left CN1_SATB_DELETE, emitted immediately before
that store, reading the same field through a plain JAVA_OBJECT volatile*. The
collector writes JAVA_NULL there atomically from cn1GcProcessReferences, so the
pair stayed a mixed atomic/non-atomic access -- the exact defect the earlier
change set out to remove. CN1_SATB_DELETE_REF is the deletion barrier with an
atomic load, used for this field only; every other field keeps the generic macro,
which is correct for them because nothing else writes them concurrently.
A DROPPED LOG ENTRY VOIDS THE CLEAR PASS'S EVIDENCE. cn1SatbEnqueue discards a
reference when its stack cannot grow. The comment there argues that is
survivable, and for an ordinary STORE it is -- "only re-opens the original race".
It is not survivable for a referent Reference.get() has already handed to a
mutator: the enqueue was the only record that it escaped, sub-pass B decides
purely on mark state, and the final take stays empty so nothing re-opens, leaving
the sweep free to reclaim an object a thread is holding. Registering the load in
the termination handshake does not help here; it delays the take, it does not
stop a drop.
Nothing on that path can allocate its way out, so the pass now snapshots the drop
count at cycle start and declines to clear at all if it moved. That costs one
cycle of reclaim in a process that is already out of memory, and it deliberately
does not disable the emergency soft-drop, which clears at DISCOVERY and never
touches the log.
Gates: run-gc-verify.sh green (RefPolicy clean over 71 verify passes, both
fault-injection self-tests firing), run-gauntlet.sh green with all eleven
tortures bit-identical and both GC stop modes; the emergency drop still fires
under injected allocation failure and the driver's weak, alias and cache
assertions are unchanged.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Count take-side SATB losses, and stop clearing when SATB is compiled out
Two accepted review findings, and one pre-existing bug found next to the first.
THE DROP COUNTER GUARDED HALF THE LOSS PATH. The previous commit invalidated
reference clearing when cn1SatbEnqueue discarded an entry, but cn1SatbTake
discards too: it resets gcSatbTop unconditionally and, when its scratch buffer
cannot be grown, reports the batch as EMPTY. Entries that were logged
successfully are then thrown away and the collector reads "nothing slipped in",
which is exactly the signal that lets termination finish. To the referent that
gets swept a lost enqueue and a lost batch are the same event, so the take now
counts as a drop as well.
PRE-EXISTING, in the same function and worth its own paragraph: realloc's result
was assigned straight back over scratch, so a failure lost the buffer that was
already there, and scratchCap was advanced whether or not the growth succeeded.
After a single failure every later take saw n <= scratchCap, skipped the realloc,
found scratch NULL and returned 0 -- the barrier logging into a stack nothing
would ever drain again, permanently and silently. It now grows through a
temporary and advances the cap only on success.
-DCN1_DISABLE_SATB WAS UNSOUND, NOT MERELY SLOWER. That arm compiles out the load
barrier that makes handing a weak referent to a mutator safe, but reference
processing is not part of SATB and kept running: a thread released after its stack
scan could load a referent, have the field cleared underneath it and the object
swept before it could use the pointer. Reference processing now switches off with
the barrier, degrading to the behaviour that preceded this feature -- referents
traced strongly and never cleared -- which is the right fallback for an escape
hatch and keeps the arm measuring the barrier's cost rather than a different
collector. Verified rather than assumed: WEAK_DEAD_CLEARED reads 0/256 in that
arm against 255/256 by default.
Gates: run-gc-verify.sh green (RefPolicy clean over 70 verify passes, both
fault-injection self-tests firing), run-gauntlet.sh green with all eleven
tortures bit-identical and both GC stop modes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Stop the verifier being blind to referents, and close a heap overflow
Two accepted review findings. One is a heap overflow this branch introduced an
hour ago; the other says the gate that was supposed to catch such things could
not see the field in question at all.
THE OVERFLOW WAS MINE. Growing cn1SatbTake's scratch buffer through a temporary
keeps the old allocation when realloc fails -- which was the point, the previous
shape leaked it -- and therefore leaves scratch NON-NULL and SMALLER than n. The
guard still tested only scratch != 0, so the memcpy wrote n entries into an
allocation sized for fewer: heap corruption written by the collector under memory
pressure, strictly worse than the leak it replaced. The buffer is usable only if
it exists AND scratchCap >= n.
THE VERIFIER COULD NOT SEE THE REFERENT. cn1GcVerifyHeap walks survivors through
the generated mark functions and relies on every reference field reaching
gcMarkObject, whose verify branch classifies it. Suppressing that call for the
referent -- the very thing that makes the edge weak -- also took the referent out
of the verifier's reach, so a live Reference holding a pointer into reclaimed
memory passed with violations=0. Every "clean over N verify passes" recorded on
this branch before now was silent about the referent specifically, which is the
one thing this work risks. cn1GcDiscoverReference now routes it to
cn1GcVerifyChild, ahead of the ageing and the dedupe: a verify walk is not a
collection cycle, and letting it age references or consume the dedupe stamp would
corrupt what the next real cycle reads and drop exactly the repeat visits a
whole-heap walk produces.
PROVEN, not assumed. CN1_GC_FAULT=refnoclear leaves a dead referent in its field
instead of clearing it, and the verifier reports DANGLING REFERENCE; clean it
reports none. run-gc-verify.sh gains self-test3 so this cannot go quietly blind
again. Note the obvious-looking fault is the wrong one and was tried first:
clearing MORE references than liveness warrants only produces extra nulls, which
are safe, and it reported violations=0 -- stopping there would have "confirmed"
the hook while proving nothing. The dangling direction is clearing LESS.
The fault's use is inside CN1_GC_VERIFY because the cn1GcFault* family is
declared there. Unguarded it broke every ordinary build while the verifier build
-- the one configuration in which the symbol exists -- kept passing, so all three
shapes are now built explicitly: plain, CN1_GC_CONFORM and CN1_GC_VERIFY.
Gates: run-gc-verify.sh green with all three self-tests firing, run-gauntlet.sh
green with all eleven tortures bit-identical and both GC stop modes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Retain discovered referents when a dropped log abandons the clear pass
Accepted review finding, and a repeat of a mistake this branch already corrected
once. The drop-counter fallback returned without clearing, which leaves every
discovered referent unmarked AND its field non-null -- so the sweep frees objects
that live References still point at. That is the dangling read the pass exists to
prevent, produced by the code meant to prevent it.
"Skip the clear" is not "keep the referent alive". Nothing else marks a weak
referent; that is what makes the edge weak. The note on the unrecorded-discovery
path says exactly this, and the same confusion reappeared here.
The fallback now marks every discovered referent and drains before returning.
Gates: run-gc-verify.sh green with all three self-tests firing, run-gauntlet.sh
green with all eleven tortures bit-identical and both GC stop modes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Measure what the Reference.get() load barrier actually costs
The barrier registers in the SATB termination handshake, which is two seq_cst
read-modify-writes on every Reference.get() and cannot be skipped soundly -- any
flag sampled before registering can go stale in the window the registration
exists to close. Whether that price is worth paying is a question for a number,
not an intuition, so this adds the means to get one.
-DCN1_REF_NO_LOAD_BARRIER compiles the registration and the enqueue out, leaving
the load and the touch stamp. It is UNSOUND and is not a shipping configuration;
it is the arm the cost is measured against.
cn1RefGets counts get() calls, and [GCREF-TOTAL] prints the run total at exit.
The per-cycle [GCREF] line cannot answer "how often does this workload call
get()", because it only prints when a collection happens -- so a get()-heavy but
allocation-light phase, which is exactly the shape that costs the barrier most,
leaves the last line stranded early in the run. Costing the barrier off that
number understates the calls and overstates the nanoseconds: it read 153,408 for
a run that made 6,155,728.
Measured on this host, RefPolicy, nine interleaved reps, both arms
-DCN1_GC_CONFORM -flto=thin:
get-dominated 64 keys x 64B, 6M reads, no churn 6,155,728 gets
+0.07% median, +0.35% floor -> +0.45 ns median, +2.14 ns floor per get()
image-cache 512 keys x 32KB, 400k reads, churn 556,416 gets
+0.10% median, -0.30% floor -> unmeasurable at this call volume
So the barrier costs roughly half a nanosecond to two nanoseconds per call, and
0.35% of wall time only at 1.6 MILLION get() calls per second. An image-heavy
screen redrawing fifty encoded images at 60fps calls it three thousand times a
second, some three orders of magnitude below the rate at which it becomes
visible.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Close the remaining reference-clearing races, and fix the no-BiBOP build
Five accepted review findings.
-DCN1_DISABLE_BIBOP DID NOT LINK. bibopGcEpoch is defined inside cn1_globals.m's
#ifndef CN1_DISABLE_BIBOP block, and the load barrier named it unguarded, so a
supported A/B and fallback configuration failed with an undefined _bibopGcEpoch.
Confirmed by building it rather than by reading guards. The epoch is only a
filter -- dropping it enqueues referents that would have been skipped, which is
more work and never less safety -- so the disabled build compares against a value
the mark word cannot equal and the barrier keeps its single comparison.
DROP RECOVERY DID NOT REACH A FIXPOINT. The same defect already fixed in
sub-pass A, reappearing in the recovery loop added a commit earlier: marking a
retained referent traces it, and an object kept alive only that way can itself
hold references whose mark functions register after the loop has passed them.
THE CAPPED SATB EXIT SKIPPED REFERENCES ENTIRELY. The CN1_SATB_MAX_REOPENS branch
drains twice on its way out, and those drains can newly mark an object whose graph
contains a Reference. It was the one exit that left without a reference pass, so a
reachable reference kept an unmarked referent the sweep then freed.
THE DROP CHECK COULD NOT SEE A DROP THAT HAD NOT HAPPENED YET. A get() starting
after the pre-loop check loads an unmarked referent, and a failed enqueue moves
the counter only once clearing has already decided it was safe. The pass now
records what each entry cleared, quiesces -- every getter registers for the
duration of its load, so an in-flight count of zero means every getter that
overlapped has finished and published its drop -- and re-reads the counter,
marking what it cleared if it moved. The stores are not undone and need not be: a
cleared reference answering null is legal, the object being freed under a mutator
is not.
THE EMERGENCY PATH HAD NOWHERE TO RECORD. It clears at discovery precisely
because the list could not grow, so the recovery above had nothing to consult and
a racing get() with a failed enqueue would have been handed a freed pointer. It
now remembers what it cleared in a fixed preallocated array -- allocation being
the one thing unavailable there -- and REFUSES TO CLEAR when that is full,
marking instead: memory the emergency wanted back is retained, which is worse
than clearing and far better than a dangling read.
Rebased onto master, which landed the parallel-mark collector work (#5717) in the
same two files; the conflict was additive on both sides and the resolved tree was
compiled before the rebase continued.
Gates: run-gc-verify.sh green with all three self-tests firing, run-gauntlet.sh
green with all eleven tortures bit-identical and both GC stop modes, and all five
build shapes compile -- plain, CN1_GC_CONFORM, CN1_DISABLE_BIBOP,
CN1_DISABLE_SATB and CN1_GC_VERIFY.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Define the GC epoch mirror unconditionally instead of working around it
The load barrier needed bibopGcEpoch, which was defined inside cn1_globals.m's
#ifndef CN1_DISABLE_BIBOP block, so that configuration failed to link. The first
fix taught the barrier to compare against a sentinel when BiBOP was compiled out
-- a configuration branch on a hot path to accommodate a symbol that had no
reason to be conditional.
The epoch is a plain mirror of currentGcMarkValue for mutator-side reads. Nothing
about it belongs to the page heap; it sat inside that guard by accident of
placement. Hoisting it out removes the macro entirely and returns the barrier to
one comparison with no configuration in it.
Keeping the arm alive rather than dropping it costs nothing now and keeps
vm/CLAUDE.md honest: -DCN1_DISABLE_BIBOP is listed there as an ablation, and an
ablation that does not link is a trap for whoever reaches for it. With BiBOP off
nothing advances the epoch, so the barrier enqueues referents it would otherwise
have skipped -- the epoch is a filter, and a stale one only ever declines to
skip.
Six build shapes verified: plain, CN1_GC_CONFORM, CN1_DISABLE_BIBOP,
CN1_DISABLE_SATB, CN1_GC_VERIFY, and CN1_DISABLE_BIBOP with CN1_GC_CONFORM
together.
Gates: run-gc-verify.sh green with all three self-tests firing, run-gauntlet.sh
green with all eleven tortures bit-identical and both GC stop modes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Serialize emergency recovery slots, and stop the epoch mirror going stale
Four accepted review findings; three are defects in the recovery paths added a
commit earlier.
EMERGENCY SLOT RESERVATION WAS RACY. cn1RefEmergencyTop++ runs after cn1RefMutex
has been released, and mark functions run on however many workers
gcMarkDrainParallel is using -- so increments were lost and two workers could
publish into one slot. An emergency clear that goes unrecorded is exactly the
case the drop recovery cannot repair, leaving the sweep free to take the referent
under a racing get(). The index is now an atomic reserve-then-publish, and a
reservation that cannot be satisfied means the clear does not happen at all.
THE EARLY DROP FALLBACK DID NOT RETAIN EMERGENCY CLEARS. It returns before the
post-clear recovery, and a referent cleared by the emergency path is gone from
its field -- it exists only in the recovery array. Retaining cn1RefDiscovered
alone therefore left those objects to be swept under a getter that had already
been handed one.
THE POST-CLEAR RECOVERY DRAINED ONCE. The same fixpoint defect as the other two
recovery loops: a referent brought back can hold a further Reference whose mark
function registers during the drain, after the loops have run. It now iterates,
and retains anything the drain discovered rather than clearing it -- by that
point the pass is past where clearing is safe.
THE STALE EPOCH WAS NOT HARMLESS. The previous commit hoisted bibopGcEpoch out of
the BiBOP guard so -DCN1_DISABLE_BIBOP would link, and its comment claimed a
frozen mirror cost only some extra enqueues. It costs convergence:
CN1_SATB_REF_KEEP skips a referent whose mark equals the epoch, and against a
mirror frozen at 1 that matches nothing from the second collection onward -- the
unfiltered shape already measured on this branch to put over 10,000 entries a
cycle into the log and reach CN1_SATB_MAX_REOPENS every cycle. A filter that
silently stops filtering is a cliff, not a rounding error.
The epoch is now published by codenameOneGCMark, which every cycle passes through
whether or not the page heap is compiled in, leaving one writer instead of two.
Verified rather than argued: with -DCN1_DISABLE_BIBOP -DCN1_GC_CONFORM the
termination loop reports passes=1 on all 32 cycles.
Gates: run-gc-verify.sh green with all three self-tests firing, run-gauntlet.sh
green with all eleven tortures bit-identical and both GC stop modes; plain,
CN1_GC_CONFORM, CN1_DISABLE_BIBOP, CN1_DISABLE_SATB and CN1_GC_VERIFY all build.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Retain rather than clear on the capped SATB termination path
Accepted review finding, and the defect was in a comment before it was in the
code.
The capped path -- reached when a mutator storms the barrier past
CN1_SATB_MAX_REOPENS -- called cn1GcProcessReferences on the way out, justified
by "the barrier is already down on this path, so a get() racing this pass cannot
log". That sentence is false. cn1SatbBulkBegin answers gcSatbActive OR
gcSatbTerminating, and gcSatbTerminating stays raised until after the loop, so a
getter there registers, enqueues successfully, and lands its entry in a log this
path never takes again. Clearing on that basis can free an object a getter is in
the middle of being handed.
Rather than correct the reasoning and keep clearing, the path now RETAINS.
Nothing is cleared, so nothing can dangle however the race falls, and no argument
about flag ordering is load-bearing. The cost is one cycle of reclaim on a path
whose own comment records reaching it 0-4 times against a cap of 32.
cn1GcRetainAllReferences factors out the retain-to-fixpoint walk that the drop
fallback and this path both need -- written three times by hand across this
branch, and the fixpoint was missing from two of them. It covers discovered
referents, referents already cleared this cycle, and the emergency array, and
iterates because marking a retained referent can reach a Reference whose mark
function registers during the drain.
Gates: run-gc-verify.sh green with all three self-tests firing, run-gauntlet.sh
green with all eleven tortures bit-identical and both GC stop modes; plain,
CN1_GC_CONFORM, CN1_DISABLE_BIBOP, CN1_DISABLE_SATB and CN1_GC_VERIFY all build.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Recover references when the final SATB take loses its batch
Accepted review finding, and the last unguarded corner of the same take-side
loss.
cn1SatbTake reports an empty batch for two different reasons: the log really was
empty, or its scratch buffer could not grow and the entries were thrown away. It
records the second case in cn1SatbDrops -- but the final catch in
codenameOneGCMark runs AFTER cn1GcProcessReferences made its last drop check, and
`if(n == 0) break` reads an empty batch as "the mark is closed". Nothing looked at
the new counter value, so a Reference.get() that logged its referent
successfully, and then had that batch discarded here, kept a pointer the
following sweep freed.
The break now retains when the counter has moved since the cycle began. Retaining
rather than another clear pass, because the barrier is coming down at that point
and there is no sound basis left for calling anything dead.
Gates: run-gc-verify.sh green with all three self-tests firing, run-gauntlet.sh
green with all eleven tortures bit-identical and both GC stop modes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Re-arm the SATB barrier before tracing recovered referents
Accepted review finding, and it catches this change breaking a rule stated in
capitals a few lines above it.
The drop recovery added by the previous commit ran cn1GcRetainAllReferences after
gcSatbActive had already been cleared. Retaining MARKS referents that were white
and gcMarkDrain then scans them, so those objects are grey at a moment when no
barrier is watching -- and a mutator moving an old child out of one of them in
that window logs nothing on either side, leaving the child unmarked, not fresh,
and reachable only from a grey object the sweep will not protect. That is exactly
the hazard the trial-clear comment above describes, and the reason the clear is a
TRIAL rather than the end of the mark.
Recovery is one more thing that can turn out to mark something new, so it now
behaves like the catch it sits next to: re-arm gcSatbActive, retain, and go round
the fixpoint again rather than draining underneath a lowered barrier. At the
reopen cap it falls through to the same weaker invariant the cap already
documents.
recoveredDrops is what makes that terminate. Comparing against
cn1RefDropsAtCycleStart would re-trigger the recovery on every pass, because that
baseline never moves once a drop has happened -- the loop would then re-arm and
retain until it hit CN1_SATB_MAX_REOPENS every time a single drop occurred.
Recording the count each recovery consumed means another pass happens only if a
NEW drop has since been recorded.
Gates: run-gc-verify.sh green with all three self-tests firing, run-gauntlet.sh
green with all eleven tortures bit-identical and both GC stop modes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Track emergency slots when deciding the retain fixpoint
Accepted review finding. cn1GcRetainAllReferences decided it had converged by
comparing cn1RefDiscoveredTop alone, but discovery has two ways to make progress
and that counter only sees one of them.
During an emergency cycle the drain can reach another soft reference whose
referent cn1GcDiscoverReference clears on the spot -- and that path deliberately
does NOT append to the discovery list, because it exists precisely for the case
where the list could not grow. It fills a cn1RefEmergencyCleared slot instead. So
the length could be unchanged while a fresh recovery slot had just been written,
the loop would read that as "nothing new" and return, and the referent recorded
in that slot would never be marked. Reached from the CN1_SATB_MAX_REOPENS
fallback, that leaves the sweep free to take a referent a concurrent
Reference.get() is being handed.
The loop now watches cn1RefEmergencyTop as well and continues while either
counter moves.
Gates: run-gc-verify.sh green with all three self-tests firing, run-gauntlet.sh
green with all eleven tortures bit-identical and both GC stop modes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Consume the touch stamp when the emergency path retains an unrecorded reference
Accepted review finding, and it is a livelock inside the path that exists to
prevent one.
The emergency clear refuses to act on a reference whose stamp reads
CN1_REF_TOUCHED and marks the referent instead, which is right for this cycle: a
mutator may be holding it. But the ageing loop walks cn1RefDiscovered, and a
reference reaching that branch is there precisely BECAUSE it could not be
recorded in that list -- so nothing ever resets the stamp. It stays TOUCHED for
the life of the process, every later emergency cycle refuses to clear the same
referent however long ago it was last read, and if that retained memory is what
is blocking the allocation then codenameOneGcMalloc's retry loop never makes
progress. The emergency was raised by an allocation failure; this is the failure
mode it was added to break.
Consuming the stamp after marking closes it. The referent has just been retained,
which is the whole of what the stamp was protecting, so the next cycle is free to
clear it if nothing reads it again -- and if something does, that get() stamps it
afresh. Compare-exchange rather than a plain store so a get() landing between the
read and the reset is not silently discarded.
Gates: run-gc-verify.sh green with all three self-tests firing, run-gauntlet.sh
green with all eleven tortures bit-identical and both GC stop modes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Export SoftReference to the supported API, and retain under the barrier at the cap
Two accepted review findings. The first is a hole in this branch's own plan that
every gate here was structurally unable to see.
SOFTREFERENCE WAS IMPLEMENTED AND UNUSABLE. It existed only in vm/JavaAPI, but
maven/java-runtime builds the SUPPORTED API SURFACE from Ports/CLDC11/src and
BytecodeComplianceMojo indexes that artifact as the set of types an application
may touch. Application code would therefore resolve SoftReference against the
host JDK and then be rejected as forbidden API -- a confusing way to discover
that a shipped feature was never exported. The plan for this work said to add the
CLDC11 stub and it was never done; nothing caught it because every gate here
compiles vm/JavaAPI directly, which is exactly the path that bypasses the
compliance surface.
RETENTION TRACED WITH THE BARRIER DOWN AT THE REOPEN CAP. The drop-recovery path
was corrected for this a commit ago and the identical defect was left at
CN1_SATB_MAX_REOPENS: cn1GcRetainAllReferences marks referents that were white
and gcMarkDrain then traces them, so with gcSatbActive already lowered those
objects are grey and unwatched, and a mutator moving an old child out of one logs
nothing on either side. Retention now runs BEFORE the barrier is lowered, so
everything known at that point is traced under it.
What remains after the final take -- retaining references that last drain
discovered -- does trace with the barrier down, and that is said plainly at the
call rather than glossed: it is the same weaker invariant the cap already relies
on for the gcMarkDrain immediately above it, not a new exposure, on a path whose
own comment measures 0-4 reopens against a cap of 32.
Gates: run-gc-verify.sh green with all three self-tests firing, run-gauntlet.sh
green with all eleven tortures bit-identical and both GC stop modes; the CLDC11
java.lang.ref package compiles.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Resolve unrecorded referents before reading them, and stop self-test3 going stale
Two accepted review findings, both introduced by this branch in the last few
commits.
A NATIVE CRASH ON THE EMERGENCY PATH. Deciding whether an unrecorded referent was
already live read its mark word directly, without the cn1ConservativeResolve
guard the clear pass carries. Under conservative roots a dead Reference kept alive
by a stale native-stack word can hold a referent swept in an earlier cycle, whose
memory is now unmapped -- and gcMarkObject's own comment says reading even the
mark word of such a pointer faults. So a collection under memory pressure, which
is the only situation that reaches this path, could take the process down. The
referent is now resolved before any dereference, and an unresolvable one is
treated as neither live nor clearable: nothing can validate it, and it is either
garbage or an object allocated after this cycle's extent snapshot that the grace
rule keeps anyway.
SELF-TEST3 COULD PASS ON A STALE BINARY. It built RefPolicy-verify only when the
file was absent, while every driver above it rebuilds unconditionally -- so a
regression in the reference-verifier hook could still produce a green self-test by
running an old binary. That is the same "a gate that cannot fail" problem this
self-test was added to solve, reintroduced in the way the self-test itself is
built. It now rebuilds every invocation.
Gates: run-gc-verify.sh green with all three self-tests firing, run-gauntlet.sh
green with all eleven tortures bit-identical and both GC stop modes;
CN1_GC_CONFORM and CN1_DISABLE_BIBOP both build.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Delete the duplicated retain walk, and make the self-test rebuild fail loudly
Two accepted review findings, and both are the same mistake: a fix applied to one
copy of something and not to its duplicate.
THE EARLY DROP RECOVERY WAS A SECOND COPY of the retain-to-fixpoint walk. The
shared helper was taught that discovery advances TWO counters -- the emergency
path clears a referent into cn1RefEmergencyCleared without touching
cn1RefDiscoveredTop, because it exists precisely for when that list cannot grow
-- and this copy was left comparing the list length alone. It therefore read
"nothing new" over a freshly written recovery slot and returned without marking
what that slot held, so a racing get() whose enqueue had also failed could be
handed an object the sweep then freed. The copy is deleted and the helper called;
patching it would have left a third place to drift.
THE SELF-TEST REBUILD STILL ACCEPTED STALE CODE. Rebuilding unconditionally was
one of three things needed and the only one done. translate-and-build.sh replaces
its output only after the final compiler run succeeds, so a FAILED rebuild leaves
the previous binary in place; and `|| true` discarded the status, so the -x test
below accepted that binary. The self-test could then run old code and report
green -- the "gate that cannot fail" problem it exists to prevent, for the second
time in how it is built. It now removes the output first, checks the build
status, and fails the gate.
Gates: run-gc-verify.sh green with all three self-tests firing, run-gauntlet.sh
green with all eleven tortures bit-identical and both GC stop modes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Let drop recovery reclaim what the emergency condemned, and delete the last copy
Two accepted review findings.
DROP RECOVERY DEADLOCKED THE ALLOCATOR. Retaining everything on an SATB loss is
the safe reflex and it is wrong under the emergency budget: that budget is raised
by an allocation FAILURE, sustained exhaustion is exactly what keeps the SATB
stack from growing, and if soft-referenced data is what exhausted memory then
every retry cycle loses a batch, retains the same data, and codenameOneGcMalloc
spins on collections that free nothing. The emergency exists to break that
deadlock and this recovery was reinstating it.
The distinction that resolves it: the emergency decision never depended on the
log. "Drop every soft referent" is a policy choice taken from the memory budget at
cycle start, not an inference from liveness, so a lost log entry does not
invalidate it. What the log would have protected is a referent a mutator is
mid-read of -- and the touch stamp records that independently and
allocation-free, which is the signal sub-pass A already trusts. So
cn1GcRecoverAfterDrop retains touched referents, still clears condemned soft
ones, and retains everything else. The residual, stated at the code, is a get()
that loaded a soft referent and was descheduled before stamping: the window the
emergency path already accepts, against an allocator that otherwise cannot
progress.
A THIRD COPY OF THE FIXPOINT. The previous commit deleted one hand-written copy
of the retain walk and said a third would drift again; there already was one, in
the post-clear recovery, and it was not looked for. It had drifted the same way
-- watching cn1RefDiscoveredTop while the emergency path advances
cn1RefEmergencyTop -- so it read "nothing new" over a freshly written recovery
slot. There is now one implementation and no copies.
Gates: run-gc-verify.sh green with all three self-tests firing, run-gauntlet.sh
green with all eleven tortures bit-identical and both GC stop modes; the
emergency still reclaims under injected allocation failure (retained=0 at
softBudget=-1).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Quiesce before reading touch stamps in drop recovery, and spare live referents
Two accepted review findings, both on the recovery helper added one commit ago.
THE TOUCH STAMP WAS READ TOO EARLY. cn1SatbDrops becomes visible when an enqueue
FAILS, and that happens before the accessor reaches its stamp -- so entering
recovery on that signal and reading touchAgeField immediately can see "not
touched" for a getter that is mid-load and about to stamp. Clearing on that
reading hands the sweep an object the getter is being given, and this path
deliberately does not record clearedReferent, so nothing downstream could recover
it.
The helper now quiesces first. Every getter registers across its whole load, the
stamp included, so an in-flight count of zero means every getter that overlapped
has finished and published -- which is what makes the stamp evidence rather than
a race.
A STRONGLY REACHABLE REFERENT COULD BE CLEARED. The other emergency path consults
the mark word before clearing, because a SoftReference may only be cleared when
its referent is SOFTLY reachable; this helper omitted that test and would clear an
application's cache entry for an object it also holds in an ordinary field. It now
applies the same current-epoch-or-fresh check.
Both are the same shape: new code on one path missing a guard its sibling already
carries.
Gates: run-gc-verify.sh green with all three self-tests firing, run-gauntlet.sh
green with all eleven tortures bit-identical and both GC stop modes; the emergency
still reclaims under injected allocation failure (retained=0 at softBudget=-1).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Guard the recovery's header read, and record what it clears
Two accepted review findings, both on the helper this branch added three commits
ago, and the first is the previous fix creating the next defect.
THE MARK-WORD READ HAD NO RESOLVE GUARD. It was added one commit back to stop the
recovery clearing strongly reachable referents -- and it dereferences the referent
without cn1ConservativeResolve, which the clear pass and the unrecorded emergency
path both apply before the same access. A Reference kept alive by a stale
native-stack word can hold a referent swept in an earlier cycle whose memory is
unmapped, and reading even its mark word faults. So a fix for a correctness gap
introduced a native crash, in code whose two siblings show the right pattern.
THE QUIESCE CANNOT HOLD. It drains the getters in flight when recovery starts and
cannot stop a new one registering immediately afterwards, while gcSatbActive and
gcSatbTerminating are both still raised. That getter can load the referent, have
its enqueue fail, and be descheduled before stamping, so this loop reads the old
stamp and clears a field whose referent is being handed out -- with nothing saved,
no later pass could mark it.
The clear is now recorded. The post-clear recovery therefore marks it whenever a
drop is visible, which gives the emergency's reclaim back in exactly the case
where safety is uncertain, and keeps it in the common case where no further drop
occurs. That is the right way round: reclaim is the goal, not being right.
This is the fourth consecutive round on this one helper, each fix producing the
next finding. The oscillation is between two poles this path genuinely sits
between -- reclaiming under memory pressure, and staying safe against an
unreliable log -- which is an argument about the shape of the design rather than
about any of the individual fixes.
Gates: run-gc-verify.sh green with all three self-tests firing, run-gauntlet.sh
green with all eleven tortures bit-identical and both GC stop modes. The first
gauntlet run was killed at IbpTest and re-run from the start; a partial gauntlet
is not a result.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Do no reference work in a program that has no references
Localises and fixes an intermittent CI regression, and removes a cost every
application was paying for a feature most of them never use.
BibopPageFloorIntegrationTest began failing on the 4-marker arm64 job -- five
times against master's five clean samples -- reporting its pages as released
while phys_footprint stayed at ~231MB. The workload contains no Reference at all,
which is why it took so long to believe: nothing in this feature can run there.
Except one thing did. cn1GcProcessReferences ran on every outer termination pass
regardless, and once the drop re-check landed it called cn1SatbBulkQuiesce()
unconditionally -- which is not free. It spins in usleep(50) while any BULK ARRAY
COPY is in flight, and that app copies object arrays, so the collector could stall
inside the termination loop on behalf of a feature the program does not use. It
fits the shape that never made sense otherwise: intermittent, only under the
configuration where four markers and bulk copies actually overlap, and reporting
release while the footprint does not move.
Dispatching the workflow on wip/refbisect-base -- master plus only the first
commit, weak references with none of this machinery -- passes the same job, which
is what localised it to the later commits rather than to the feature.
The fix is what should have been there from the start: with nothing discovered and
nothing in the emergency array, the pass returns immediately and touches nothing.
An earlier hypothesis for this failure was WRONG and is recorded so it is not
retried: cn1RefBeginCycle's headroom probe was suspected of adding a per-cycle
footprint syscall, but the test sets only CN1_LOG_PAGE_RELEASE, never
CN1_SIMULATE_PROC_MEMORY_LIMIT, and cn1SimulatedProcLimitBytes caches -- so on
Linux that path returns -1 from a cached atomic read.
Gates: run-gc-verify.sh green with all three self-tests firing, run-gauntlet.sh
green with all eleven tortures bit-identical and both GC stop modes; RefPolicy's
weak, alias and cache assertions unchanged.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Nursery promotion, two silent gates, and a baseline the rename broke
cn1GcDiscoverReference now defers to gcMarkObject while a thread is inside
its own nursery minor collection. cn1PromoteDrain runs the generated mark
functions with nurseryPromoting raised, and gcMarkObject's promotion branch
is what moves a referenced object out of the block being recycled -- routing
the referent past it meant a surviving WeakReference could be promoted alone
and left pointing into a reclaimed block. Nothing clears on that path, so the
edge costs a promotion and nothing else.
The nursery arm did not compile at all, here or on master: nativeMethods'
bulk barrier calls cn1SatbBulkBegin unconditionally while the prototype sat
in the #else of the CN1_NURSERY split, which clang rejects as an implicit
declaration under C99. The load barrier added a second instance. Declared
beside the deletion barrier instead, where the callers are.
Two gates that could not fail:
- RefPolicy printed WEAK_LIVE_KEPT and exited 0 whatever it said. A referent
cleared while still strongly reachable is heap-SAFE -- null dangles nothing
-- so run-gc-verify cannot see it either, and the checksum reads live[]
directly. Both advertised validation paths could stay green while
WeakReference silently emptied every cache built on it. It is an assertion
now.
- ab-refs.sh ignored the return code. Metrics print before exit, so a VM that
corrupted its heap and died in an atexit handler still emitted RESULT and
the whole table, and the harness published checksum-matched medians from a
crashed run.
The cast-semantics baseline names anonymous classes as Outer$N, so #5746
adding one to AndroidImplementation renumbered onReceive from $46 to $47 and
left the entry stale. master is red on the gate today; this is the same cast,
not a new one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* The emergency reclaim retained everything it condemned
cn1GcRecoverAfterDrop is reached precisely when soft-referenced data is what
is exhausting memory, and blanket retention there is what lets the allocator
spin -- which is why it clears selectively instead of calling
cn1GcRetainAllReferences. It then recorded each condemned referent in
clearedReferent and marked that same referent one statement later in the same
iteration, so the sweep kept every one of them and the reclaim never happened.
Its own comment described the behaviour it did not have: "in the common case,
where no further drop occurs, the clear stands and the reclaim happens".
The recovery is owed to clears made by an EARLIER pass, which predate the drop
that brought this one here; it is not owed to the clears this pass has just
decided for itself, after its own quiesce. Entries therefore record which pass
cleared them.
A drop count cannot make that distinction, and the first version of this fix
used one. The getter the record defends against fails its enqueue BEFORE the
clear stamps anything, so the count at the clear already includes it, and
"the count moved since" is false in exactly the case where the mark is owed --
it would have reintroduced the dangling read the record exists to prevent.
The drop count is now read once per pass, after the quiesce that bounds the
window, and a drop past it retains through the shared walk.
Two harness gates that could not fail:
- RefPolicy exits nonzero on ALIAS_SPLIT. This does NOT make the phase a
detector for the violation, and the code says so: with
-DCN1_REF_NO_ALIAS_ATOMICITY putting the single-loop bug back, three runs
still reported 0/256, because catching it needs a get() inside a window
microseconds wide. The ablation is what tests that path.
- ab-refs.sh requires the weak phase to have cleared something -- and for
noweak, nothing. The checksums are independent of retention policy by
design, so they agree just as well when an arm keeps every referent strong,
which is the regression that matters.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Record why the porting layer still hands back a WeakReference
A review read the unmigrated createSoftWeakRef as the feature being undelivered:
the base implementation still returns a WeakReference and the iOS override still
pins every entry in a Hashtable until a memory warning replaces the map, so no
framework cache constructs the new class.
The facts are right and the conclusion is not. Migrating them changes the
lifetime of every decoded image, gradient and resource cache in every app on
every platform, which wants its own change and its own bisect point -- and each
call site needs deciding rather than sweeping, because a lifetime tracker like
JavascriptContext reads a null extract as proof of collection and breaks under a
reference that outlives its referent. This change is the mechanism and the
measurement that justifies it.
Noted at both places a reader arrives from, since a PR thread is not somewhere
anyone looks later.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The gap
Exiting a Codename One application has only ever meant
exitApplication(). On Android that kills the process and leaves the task sitting in the recents list, so the user can pick the app back out of the task switcher, and a relaunch from there restores a task the application believed it had ended. Applications that need a real sign-out or kiosk-style exit had no way to ask for it.The API
Display.exitAndClearTask()andCN.exitAndClearTask(), plusisExitAndClearTaskSupported()for applications that want to branch on the difference.The shape follows the existing
exit()/exitApplication()pair exactly:CodenameOneImplementation.exitAndClearTask()runs thesetOnExitcallback, then calls the new port hookCodenameOneImplementation.exitApplicationAndClearTask()is that hook, and its base implementation is a plainexitApplication()So the fallback is structural, not conditional: every port that does not override the hook keeps today's behaviour, and none of them needed touching.
isExitAndClearTaskSupported()answers false everywhere but Android.Android
AndroidImplementation.exitApplicationAndClearTask()callsActivity.finishAndRemoveTask()on the UI thread (inline when already there) and then kills the process the wayexitApplication()does. It degrades to a plain exit whengetActivity()is null -- a push or background service process owns no task of its own -- and below API 21, where the platform method does not exist.Killing right after finishAndRemoveTask() was measured, not assumed
That sequence looks like it could race the removal, so it was checked with a throwaway Android app running the exact sequence on an API 36 emulator:
killProcessonly (whatexitApplication()does) -- controlfinishAndRemoveTask()onlyfinishAndRemoveTask()+ kill after the looper drainsfinishAndRemoveTask()+ immediate kill (what ships here)The single apparent failure across 30 trials had the driving broadcast dropped before the handler ran, not a race. The control is what makes the result mean anything. The finding is recorded in a comment beside the call.
Test
ExitAndClearTaskTestpins the three contract points: an unsupported port degrades toexitApplication(), a supporting port does not also run it, andsetOnExitfires on both paths. Removing the fallback body makes the first test fail, so the test is not vacuous.Checks run locally
core,androidandcore-unittestsbuild clean; SpotBugs reports zero findings in both🤖 Generated with Claude Code