Skip to content

Full mark-sweep marks the remembered set as roots, retaining young objects through DEAD old-gen pages #9629

Closed

Description

@proggeramlug

What

A full mark-sweep runs RootScanSubphase::RememberedSet, which marks every dirty old-gen page's slots as roots. In a full trace the old generation is already traced from real roots, so the only thing this adds is retention: it marks through old objects that are themselves dead.

Evidence

Fixture (compiled, Linux x86_64): build 10,000 strings into an array large enough to be promoted, drop the array, call gc().

PERRY_GC_TRACE=1 for that collection:

remembered_set {"dirty_pages_scanned": 20, "dirty_slots_scanned": 10000,
                "newly_marked": 10000, "old_objects_considered": 1, "valid_roots": 1}

newly_marked: 10000 is the entire dropped population, and old_objects_considered: 1 is the dead backing array. A heap census taken inside the same cycle confirms the 10,000 strings are classified live, not dead, with the remembered-set count accounting for all of them.

The dead array survives because of #9628, so its pages never leave the dirty set, and every later full collection re-marks the same 10,000 strings. The two interact: #9628 keeps the owner alive, this keeps everything the owner points at alive.

Why it looks redundant for a full cycle

mark_remembered_set_roots validates that the dirty page's object is a plausible pointer (valid_ptrs.contains), not that it is live. That is the right check for a minor, where the old generation is deliberately not traced and old→young edges genuinely are roots. A full trace visits the old generation from the real root set, so a live old object's young children are reached anyway, and the only objects this phase adds are those reachable exclusively from dead old objects.

Impact

Any old-gen object that dies while holding young references keeps them alive until a collection that does not consult the remembered set, which currently is no collection at all. On the compiled claude-code TUI this is part of why an idle process holds a large dead set that only allocation pressure clears.

Not proposing the fix here

Gating the subphase on self.minor.is_some() is the obvious shape, but the remembered set is also the input to evacuation and to the rebuild that runs in AtomicFinalize, so whether a full cycle can simply skip it is a question for whoever owns the generational invariants. I have not tested that change.

Found while validating a heap-census instrument (PERRY_GC_CENSUS, branch feat/gc-census).

Activity

  1. proggeramlug commented on Sep 3, 2026

    @proggeramlug
    ContributorAuthor

    Fixed on main (PR #9651, re-landed through a merge train so this did not auto-close).

    A full trace now takes the remembered-set snapshot but marks nothing from it. Keeping the snapshot is load-bearing and was the trap in the change: remembered_dirty_snapshot() is what lazily arms the write barrier and reconstructs the log from the heap, so skipping it as well would leave old-to-young stores unlogged, which fails in the opposite and far worse direction. Minors are unchanged, since there old-to-young edges genuinely are roots.

    Two existing tests moved from a full cycle to a minor as part of this. Both drove a full cycle with an unrooted alloc_old_test_object owner and asserted its children stayed marked, which is this bug asserted as the contract; their real subject is budget slicing and that is untouched.

    Measured on the compiled claude-code TUI at idle: the three full collections marked 845, 800 and 800 young objects from the set. The retention actually removed is the subset of those reachable only from dead old objects, which the existing telemetry does not separate — everything genuinely reachable is still reached from the real roots.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions