Skip to content

PR attribution: reconciliation pass for unattributed sessions #990

Open

Description

@iamtoruk

Problem

PR attribution only ever runs forward. A turn is attributed to the PR active when it happened, and a spawned subagent inherits the PR set live at the spawning turn (buildSpawnPrSets). Work done before a PR reference exists can never be attributed to it, so it lands in the unattributed pool and stays there permanently.

That pool is not noise. For orchestrated workflows it is often most of the money: a task fans out to five or ten subagents that explore, generate, and get discarded, and only a final cheap synthesis step produces the PR. None of the workers individually "did" the PR, and at the time they ran there was no PR to attribute them to. The grouping exists only in the orchestrator's head, and the logs never see it.

Refusing to smear that spend onto whatever session happened to touch the PR is the right call and should stay (see the black hole below for what happens when we guess). But leaving it orphaned forever is not the only alternative.

Proposal

A reconciliation pass: when a PR reference appears, offer previously-unattributed sessions in the same repository and a nearby window as suggestions for that PR, which a human confirms or rejects. Not automatic, not confident, just a queue.

Credit: proposed by a user in a discussion thread about how CodeBurn handles subagent fan-out.

Why this fits the existing design

  • Provenance already exists. SessionSummary.prAttributionSource is an enum (transcript | explicit-reference | working-directory | launcher-prompt, src/types.ts:206). A confirmed match becomes a fifth variant (user-confirmed), so every surface that already renders provenance handles it without special-casing, and confirmed attributions stay visibly distinct from derived ones.
  • Confirm-and-record has a precedent. codeburn act applies changes through a journal with list and undo (src/act/). A confirmation queue should ride that pattern rather than invent a parallel one, so a wrong confirmation is reversible.

Two things that have to be designed around

1. The unattributed pool is a scalar, not a set

attributeSessionPrSpend returns unattributed: { cost, calls, savingsUSD } and buildPrAttribution accumulates it into a single unattributedCost number (src/sessions-report.ts). There is no retained list of which sessions contributed. The candidate set has to be rebuilt from the session cache, which does retain workingDirectory, firstTimestamp / lastTimestamp, parentSessionId, agentId, and spawnPrSets.

2. Source retention bounds how far back this can reach

Claude Code deletes transcripts after ~30 days. Verified on a real machine while investigating a separate report: 20 of 30 June days had no session files left at all, and no cache file ever held them.

This makes the naive design (wait for the PR, then walk backward through sessions) fail silently past the retention window. Worse, it degrades invisibly: fewer suggestions appear, with no signal distinguishing "nothing to suggest" from "the candidates were deleted."

So invert it. Snapshot candidate metadata eagerly at parse time (session id, provider, canonical repo root, activity window, cost, calls, spawn parentage) into the durable daily cache, then reconcile lazily against that snapshot when a PR shows up. Cheap to store, and it survives source deletion. The daily cache already does exactly this trick for history (#755), so the mechanism exists.

The risk: this relaxes a bound that exists for cause

Unbounded working-directory correlation is what produced the eywa#160 black hole: a repo whose only captured PR link was pasted once absorbed 129 of 131 sessions, ~$7.4K, including a month of unrelated work. The fix was CWD_WINDOW_PAD_MS = 6h (src/parser.ts), bounding cwd evidence to the same working stretch.

A human gate is genuinely safer than a heuristic window, but the failure mode relocates rather than disappearing: if the queue is usually right, people learn to click yes, and the black hole returns through the confirm button. Mitigations that should be part of the design, not bolted on later:

  • Keep the queue small and high-signal. Three suggestions a week that make someone think beat thirty that train reflexive approval.
  • Show the evidence, not just the claim: which repo, which window, why this session was proposed, what it costs.
  • Never bulk-confirm. No "accept all".
  • Cap suggestion reach explicitly, and state the cap in the UI so a bounded search is not mistaken for an exhaustive one.
  • Confirmations are journaled and undoable, and stay tagged user-confirmed forever.

Sketch

  1. Capture (parse time): persist per-session candidate metadata for sessions ending unattributed, in the durable cache alongside daily history.
  2. Match (when a PR reference appears): propose candidates sharing the canonical repo root within a bounded window, ranked; subagent children of a proposed parent travel with it as one unit, since splitting a fan-out across PRs is meaningless.
  3. Confirm (explicit): codeburn reconcile lists suggestions with evidence; confirming writes prAttributionSource: 'user-confirmed' through the act journal. Also surfaceable in the desktop PR view.
  4. Report: confirmed spend counts as attributed and is visually distinguishable from derived attribution. undo restores unattributed.

Open questions

  • Bound for the candidate window: reuse the 6h pad, or something wider given a human gates it?
  • Should confirmations generalize (learn "this repo's sessions usually belong to its open PR"), or stay strictly one-off? Generalizing is where reflexive approval turns back into a black hole.
  • Rejections are also signal. Persist them so the same suggestion is not offered twice.

Activity

  1. ozymandiashh commented on Aug 18, 2026

    @ozymandiashh
    Collaborator

    Two notes on the capture step, both from reading the code the proposal leans on, and then a go at the open questions.

    The daily cache will silently eat the candidate metadata.

    sanitizeProviders (src/daily-cache.ts:254) is a strict allowlist, not a filter. It constructs a fresh slice from calls/cost/savingsUSD, copies only the seven keys in OPTIONAL_SLICE_NUMERICS, then models, categories and projects, and drops everything else on the floor. It runs on read and on merge, so a candidate-metadata field written by step 1 survives exactly until the next cache generation and then disappears.

    That is worse than the usual "remember to update the allowlist" chore, because the failure is invisible in the specific way this design is trying to avoid: nothing errors, the queue just quietly holds fewer suggestions, and there is still no signal separating "nothing to suggest" from "the metadata was dropped on the last merge". If capture lands, it needs its own declaration in that allowlist plus a test that goes red when a field is added to the slice type without being declared — the same shape as tests/provider-env-declarations.test.ts, which exists for exactly this failure mode on a different map.

    The retention argument holds, but it is worth naming which cache.

    The two caches have opposite retention, and the proposal's "cheap to store, survives source deletion" claim only holds in one of them. The daily cache keeps ten years (src/daily-cache.ts:21). The 90-day age-out is in the session cache, on durable provider entries (src/parser.ts:3274-3277) — that is the one #987 is about. So capture has to go in the daily cache specifically; putting it in the session cache would reintroduce a cliff at 90 days, and per #987 that cliff already drops history whose sources are still present.

    On the open questions:

    Window bound. I would not reuse the 6h pad. CWD_WINDOW_PAD_MS bounds evidence that gets applied automatically, so it is sized to keep a wrong guess cheap. A human-gated queue has the opposite constraint: the cost of a wide window is reviewer fatigue, not a bad write. Sizing by time looks like the wrong axis to me — a fan-out that ran over two days and one that ran over two hours are equally reviewable if the queue presents each as a single unit. Bounding the number of suggestions and letting the ranking decide what fills it gets the "keep the queue small and high-signal" property directly, instead of hoping a time constant produces it.

    Generalizing confirmations. No, and I think the risk section already answers it. "This repo's sessions usually belong to its open PR" is the eywa#160 rule with a friendlier name: the same unbounded working-directory correlation, learned rather than hardcoded, and it re-earns the black hole the moment a repo has two PRs open at once. One-off only.

    Rejections. Persist them, and treat them as more than suppression. A rejection is the only thing that distinguishes "we never proposed this" from "we proposed it and a human said no". Without that, there is no way to measure whether the queue is actually high-signal or whether people have started clicking yes — which is the failure mode the design is most worried about, and the one it otherwise has no instrument for.

  2. added
    enhancementNew feature or request
    area: cliThe codeburn CLI and its core parsing/reporting engine
    on Sep 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area: cliThe codeburn CLI and its core parsing/reporting engineenhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions