Repository navigation
feat(cli): receipt the absorbed update under the job token, and skip the queued run - #7456
waynesun09 wants to merge 2 commits into
Conversation
PR Summary by QodoAuthenticate steer receipts and skip absorbed queued runs
AI Description
Diagram
High-Level Assessment
Files changed (14)
|
|
🤖 Review · Commit: |
Code Review by Qodo
1.
|
3b1b6bc to
bf569fe
Compare
|
🤖 Finished Review · ✅ Success · Started 3:32 PM UTC · Completed 4:12 PM UTC Commit: Runtime: pi · Model: sonnet → claude-sonnet-5 · Effort: high · Cost: $20.62 |
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
|
Risk Assessment: moderate (2/5) DetailsLarge blast radius (23 files, 3341 lines) and hotspot churn on run.go and sandbox.go are offset by a solid 0.30 test ratio, zero protected-path hits, no CI or dependency changes, opt-in feature gating, and established contributor status, preserving the moderate (2/5) assessment from prior review rounds. Previous runRisk Assessment: moderate (2/5) DetailsLarge blast radius (23 files, 3341 lines) and hotspot churn on run.go and sandbox.go are offset by a solid 0.30 test ratio, zero protected-path hits, no CI or dependency changes, opt-in feature gating, and established contributor status, preserving the moderate (2/5) assessment from prior review rounds. Previous run (2)Risk Assessment: moderate (2/5) DetailsLarge blast radius (23 files, 3344 lines) and hotspot churn on run.go and sandbox.go are offset by a solid 0.30 test ratio, zero protected-path hits, no CI or dependency changes, opt-in feature gating, and established contributor status, preserving the moderate (2/5) assessment from prior review rounds. Previous run (3)Risk Assessment: moderate (2/5) DetailsLarge blast radius (21 files, 3173 lines) offset by a healthy ~0.29 test ratio, zero protected-path hits, no CI/dependency changes, and an established contributor; two security-sensitive files (steer.go, review_skip.go touching job-token/credential-matching logic) and hotspot churn on run.go/steer.go are balanced by issue-scope alignment and no risk-elevating labels. Composite continues to round to 2 (moderate), matching prior rounds of this same PR. Previous run (4)Risk Assessment: moderate (2/5) DetailsUnchanged metadata signals (20 files, 3,167 lines, zero protected or security-sensitive paths, no CI or dependency changes, 0.30 test ratio) combined with opt-in default-disabled behavior preserve the composite score of 2 (moderate), balancing elevated churn and coupling on core runner hotspot files. Previous run (5)Risk Assessment: moderate (2/5) DetailsUnchanged metadata signals (20 files, 3,167 lines, zero protected or security-sensitive paths, no CI or dependency changes, 0.30 test ratio) combined with opt-in default-disabled behavior preserve the composite score of 2 (moderate), balancing elevated churn and coupling on core runner hotspot files. Previous run (6)Risk Assessment: moderate (2/5) DetailsLarge diff across 20 files with high churn and coupling on runner hotspots is counterbalanced by zero protected or security-sensitive path modifications, no CI or dependency changes, a strong 0.30 test ratio, and default-disabled gating, anchored to the prior rounds moderate score. Previous run (7)Risk Assessment: moderate (2/5) DetailsLarge diff across 20 files with high churn and coupling on existing runner hotspots is counterbalanced by zero protected or security-sensitive path changes, no CI or dependency modifications, a solid 0.30 test ratio, and default-disabled gating, anchored to the prior rounds moderate score. Previous run (8)Risk Assessment: moderate (2/5) DetailsLarge diff across 20 files with high churn on existing runner and forge hotspots is offset by zero protected or security-sensitive path changes, no CI or dependency modifications, solid test ratio (0.30), and an established contributor, anchored to the prior rounds moderate score. Previous run (9)Risk Assessment: moderate (2/5) DetailsLarge diff across 20 files with high churn on existing runner and forge hotspots is offset by zero protected or security-sensitive path changes, no CI or dependency modifications, solid test ratio (0.30), and an established contributor, anchored to the prior rounds moderate score. Previous run (10)Risk Assessment: moderate (2/5) DetailsLarge diff across 20 files with high churn on existing runner and forge hotspots is offset by zero protected or security-sensitive path changes, no CI or dependency modifications, solid test ratio (0.30), and an established contributor, anchored to the prior round's moderate score. Previous run (11)Risk Assessment: moderate (2/5) DetailsLarge diff across 20 files with high churn on existing runner and forge hotspots is offset by zero protected or security-sensitive path changes, no CI or dependency modifications, solid test ratio (0.30), and an established contributor, anchored to prior rounds moderate score. Previous run (12)Risk Assessment: moderate (2/5) DetailsLarge diff across 20 files with high churn on existing runner and forge hotspots is offset by zero protected or security-sensitive path changes, no CI or dependency modifications, solid test ratio (0.30), and an established contributor, anchored to prior rounds moderate score. Previous run (13)Risk Assessment: moderate (2/5) DetailsLarge diff (14 files, 2,340 lines) and elevated churn on existing core files (run.go, architecture.md) are counterbalanced by zero protected or security-sensitive path modifications, no CI or dependency changes, a 0.29 test ratio, and an established contributor, preserving the anchored score of 2 (moderate) under 62/38 Tier 1/Tier 2 weighting. This round is a pure rebase (all 14 files byte-identical to the prior anchored round) with no signal change. Previous run (14)Risk Assessment: moderate (2/5) DetailsLarge diff (14 files, 2,340 lines) and elevated churn on existing core files (run.go, architecture.md) are counterbalanced by zero protected or security-sensitive path modifications, no CI or dependency changes, a 0.29 test ratio, and an established contributor, preserving the anchored score of 2 (moderate) under 62/38 Tier 1/Tier 2 weighting. Previous run (15)Risk Assessment: moderate (2/5) DetailsLarge diff (14 files, 2,325 lines) and high churn on existing core files (run.go, architecture.md) elevate risk, but zero protected or security-sensitive path modifications, no CI or dependency changes, a 0.29 test ratio, and an established contributor preserve the anchored score of 2 (moderate) under 62/38 Tier 1/Tier 2 weighting, with this round's only delta being an isolated doc-comment update to gitlabMergeRequestHeadSHA. Previous run (16)Risk Assessment: moderate (2/5) DetailsLarge diff (14 files, 2,325 lines) and high churn on existing core files (run.go, architecture.md) elevate risk, but zero protected or security-sensitive path modifications, no CI or dependency changes, a 0.29 test ratio, and an established contributor preserve the anchored score of 2 (moderate) under 62/38 Tier 1/Tier 2 weighting. This round closed a 12-round-standing low-severity credential-isolation gap (stripOIDCEnv now strips swapped-away tokens by value too), a risk-reducing change too small to move the rounded composite. Previous run (17)Risk Assessment: moderate (2/5) DetailsNo linked issue (62/38 Tier1/Tier2 weighting): Tier1 (~1.88) reflects a large diff fully offset by zero protected/security-sensitive path hits, no CI/dependency changes, a 0.29 test ratio, and a known non-first-time author; Tier2 (~2.6) is pulled up by very high churn on run.go/architecture.md/security-threat-model.md but pulled down by brand-new files with no history. Composite rounds to 2 (moderate), preserving the prior anchored score; the newly-landed job-token receipt/credential-stripping logic is captured by run.go's already-elevated Tier2 churn score rather than by the path-pattern security heuristic, so no upward adjustment is warranted. Previous run (18)Risk Assessment: moderate (2/5) DetailsTier 1 re-verified this round: identical bucket outcomes to the prior assessment (14 files, large blast radius, 0 protected/security paths, no CI/dependency changes, 0.29 test ratio, known non-first-time member author). Tier 2 churn on the two files with a genuine behavior delta since the prior review (statuscomment.go new settledSHA field/rendering change, paired with a new steermarker_test.go regression test) shows moderate per-file history, not elevated. With no linked issue (62/38 Tier1/Tier2 weighting), the composite rounds to 2, preserving the prior anchored score of 2 (moderate). Previous run (19)Risk Assessment: moderate (2/5) DetailsTier 1 re-verified identical to the prior assessment (14 files, 2088 lines, large blast radius, 0 protected/security paths, no CI/dependency changes, 0.29 test ratio, known non-first-time member author); the only delta since the prior review is a comment-only reordering in run.go with no line-count or logic change. No linked issue, so weight remains 62/38 Tier1/Tier2. Score preserved at 2 (moderate) per re-review anchoring. Previous run (20)Risk Assessment: moderate (2/5) DetailsComposite = 0.62xTier1(1.88) + 0.38xTier2(2.29) = 2.03 -> 2 (moderate). Large diff (2088 lines/14 files) offset by zero protected-path and zero security-pattern-path hits, no CI/dependency changes, ~29% test ratio, and a known non-first-time maintainer; two long-lived hotspot files (run.go, docs/architecture.md) elevate churn/coupling signals. No issue is authoritatively linked, so weight was redistributed 62/38 to Tier1/Tier2. Previous run (21)Risk Assessment: elevated (3/5) DetailsRe-review anchoring holds: Tier 1 metadata is unchanged from the prior round (14 files, 2088 lines, large blast radius, no protected/dependency/CI changes, non-bot non-first-time author), and Tier 2 again shows run.go as an extremely hot, multi-author, fix/revert-heavy file while the new job-token trust-boundary code (steer.go, steermarker.go, ADR 0120) still has zero base-branch commit history since it was added within this same long-running PR. No signal has changed since the prior review that would justify moving off the anchored elevated score of 3. Previous run (22)Risk Assessment: elevated (3/5) DetailsRe-review anchoring: Tier 1 metadata is unchanged from the prior round (14 files, ~2085 lines, large blast radius, no protected/dependency-file changes, no CI changes, non-bot non-first-time author), and Tier 2 independently reproduces the same elevated picture -- run.go remains an extremely hot, multi-author, fix/revert-heavy file, and the new job-token trust-boundary code (steer.go, steermarker.go) still has zero base-branch commit history. No signal has changed since the prior review that would justify moving off the anchored elevated score of 3. Previous run (23)Risk Assessment: elevated (3/5) DetailsTier 1 metadata is essentially unchanged from the prior review (14 files, ~2078 lines, large blast radius, no protected/dependency-file changes), and Tier 2 independently reproduces the prior round's elevated picture -- run.go remains an extremely hot file (102 commits/30d, 17 authors, 164 fix/revert commits/90d) and the new job-token trust-boundary code (steer.go, steermarker.go) still has zero base-branch commit history -- so per re-review anchoring the prior elevated score is preserved. Review found the medium nested-mint credential-isolation gap from the prior round fixed, but a low-severity variant of the same class of gap (validation-script environment composition) remains open. Previous run (24)Risk Assessment: elevated (3/5) DetailsTier 1 metadata is essentially unchanged from the prior review (14 files/~1957 lines, large blast radius, no protected/dependency changes) and trends moderate alone, but Tier 2 independently reproduces the prior round's elevated picture: high churn/fix-revert density on core touched files, broad change-coupling to untouched files, and the new job-token trust boundary code remains entirely without base-branch history. This round is again a fix-up attempt, and review found the nested-mint credential-isolation gap still present, so the elevated score is preserved. Previous run (25)Risk Assessment: elevated (3/5) DetailsTier 1 metadata is essentially unchanged from the prior review (14 files / ~1947 lines, large blast radius, no protected/dependency changes) and raw weighted composite alone would round to moderate, but Tier 2 shows elevated churn and fix/revert history on the core files this PR touches, and the security-critical steer/steermarker files carrying the new job-token trust boundary remain entirely absent from base-branch git history -- the same condition that drove the prior elevated score. This round is a fix-up addressing two prior High findings, but review found the fix incomplete (a nested-mint edge case reopens the credential-isolation gap), so the qualitative risk of this unverified trust boundary is preserved and the score is kept at elevated rather than lowered. Previous run (26)Risk Assessment: elevated (3/5) DetailsLarge-blast-radius change (1815 lines/14 files, weighted composite ~2.2 rounds to moderate) is bumped to elevated because its highest-risk component is an entirely new job-token-authenticated trust boundary with no git history to assess, and review found a real fail-open defect (--no-post-script) plus a credential-isolation gap (GITHUB_TOKEN) in that exact boundary. |
ReviewFindingsHigh
Notes
Next steps:
Previous runLooks good to me Previous run (2)Looks good to me Previous run (3)Looks good to me Previous run (4)ReviewFindingsLow
Notes
Previous run (5)ReviewFindingsLow
Notes
Previous run (6)Looks good to me Previous run (7)ReviewFindingsLow
Previous run (8)ReviewFindingsLow
Notes
Previous run (9)ReviewFindingsLow
Previous run (10)ReviewFindingsLow
Notes
Next steps:
Previous run (11)Looks good to me Previous run (12)Looks good to me Previous run (13)Looks good to me Previous run (14)ReviewFindingsMedium
Low
Notes
Next steps:
Previous run (15)Looks good to me Previous run (16)ReviewFindingsLow
Notes
Next steps:
Previous run (17)Looks good to me Previous run (18)Looks good to me Previous run (19)ReviewFindingsLow
Next steps:
Previous run (20)Looks good to me Previous run (21)Looks good to me Previous run (22)ReviewFindingsMedium
Low
Notes
Next steps:
|
bf569fe to
f28d843
Compare
|
🤖 Review · ❌ Terminated · Started 6:07 PM UTC · Ended 6:36 PM UTC Commit: |
Site previewPreview: https://a69822c6-site.fullsend-ai.workers.dev Commit: |
|
🤖 Finished Review · ✅ Success · Started 9:44 PM UTC · Completed 10:21 PM UTC Commit: Runtime: pi · Model: sonnet → claude-sonnet-5 · Effort: high · Cost: $27.62 |
|
🤖 Finished Review · ✅ Success · Started 4:02 AM UTC · Completed 4:26 AM UTC Commit: Runtime: pi · Model: sonnet → claude-sonnet-5 · Effort: high · Cost: $12.01 |
|
🤖 Finished Review · ✅ Success · Started 2:31 PM UTC · Completed 3:08 PM UTC Commit: Runtime: pi · Model: sonnet → claude-sonnet-5 · Effort: high · Cost: $18.73 |
|
🤖 Finished Review · ✅ Success · Started 8:22 PM UTC · Completed 8:46 PM UTC Commit: Runtime: pi · Model: sonnet → claude-sonnet-5 · Effort: high · Cost: $13.38 |
|
🤖 Finished Review · ✅ Success · Started 10:17 PM UTC · Completed 10:38 PM UTC Commit: Runtime: pi · Model: sonnet → claude-sonnet-5 · Effort: high · Cost: $11.49 |
|
🤖 Finished Review · ✅ Success · Started 11:28 PM UTC · Completed 12:01 AM UTC Commit: Runtime: pi · Model: sonnet → claude-sonnet-5 · Effort: high · Cost: $14.61 |
|
🤖 Finished Review · ✅ Success · Started 12:33 AM UTC · Completed 12:57 AM UTC Commit: Runtime: pi · Model: sonnet → claude-sonnet-5 · Effort: high · Cost: $12.69 |
|
🤖 Finished Review · ✅ Success · Started 4:16 PM UTC · Completed 4:44 PM UTC Commit: Runtime: pi · Model: sonnet → claude-sonnet-5 · Effort: high · Cost: $13.80 |
|
🤖 Finished Review · ✅ Success · Started 7:26 PM UTC · Completed 7:41 PM UTC Commit: Runtime: pi · Model: sonnet → claude-sonnet-5 · Effort: high · Cost: $5.81 |
|
🤖 Finished Review · ✅ Success · Started 8:38 PM UTC · Completed 8:56 PM UTC Commit: Runtime: pi · Model: sonnet → claude-sonnet-5 · Effort: high · Cost: $9.58 |
A run in flight absorbs an update and the run queued behind it does the work again, so steering costs more than preserving alone: two reviews of one head where preserving alone produces one. After a successful run that absorbed at least one steer, the runner posts a receipt naming the follow-up runs it consumed. A queued run reads the latest receipt in its pre-flight, before the start comment and before the pre-script, and exits 0 without starting the agent when it finds its own run id. The receipt's author is the whole of the authentication, and it is the GitHub Actions job token's. Two conditions are enforced rather than assumed: minting must actually have swapped the credential, since otherwise it stays in the environment a post-script inherits; and the receipt login must differ from the role's, since the action's github_token is a caller-supplied input that can resolve to the identity the agent posts under. No receipt is honoured while any provider definition in .fullsend/providers/ expands GH_WORKFLOW_TOKEN, whichever harness uses it: that hands the job token to a sandbox, where the agent can recover it and forge one (ADR 0114). A queued review also skips when the review App already reviewed the pull request's current head after the queued run was created. Two events on one head route two reviews, and the run in flight writes no receipt for a run it rejects as not fresh. A human /fs-review, a manual re-run, an unreadable event and every error run the review, steering or not. Only an outright success writes a receipt, once per run, naming what the iteration whose output shipped absorbed. Everything fails open: a false "already handled" drops work, a false "not handled" costs one run. The terminal status comment keeps an informational copy of the marker that the skip check does not honour, and agent-authored text is neutralized before it reaches that comment. Assisted-by: Claude Signed-off-by: Wayne Sun <gsun@redhat.com>
steering.md gains the receipt's own sections: what the runner posts, who may write one and what that author proves, and which direction each failure takes. architecture.md records the decision beside the rest of the steering chain, and the threat model's coalescing mitigation names the receipt the queued run exits on. Assisted-by: Claude Signed-off-by: Wayne Sun <gsun@redhat.com>
|
🤖 Finished Review · ✅ Success · Started 9:20 PM UTC · Completed 9:36 PM UTC Commit: Runtime: pi · Model: sonnet → claude-sonnet-5 · Effort: high · Cost: $4.38 |
There was a problem hiding this comment.
Note: The following review comments could not be posted on the diff (GitHub returned 422) and are included here instead:
docs/contributing/steering.md(file-level): Line 100 · [high] incorrect-doc
Lines 100–101 of the new head-move paragraph claim that when a move is "cut off" by "the steer budget," it is not taken as told: lastHead stays put and the next steer reports the same move again. That is not what the watcher does, and the sentence sits immediately after the inner head-move comparison cuts ("a file list longer than the head-move budget"; patches bounded with the cut stated), so a reader applies the pinned-head rule to the wrong budget. Independently re-checked against the PR-base internal/steerwatch sources (not part of this PR's diff, but the behavior this new prose describes): maxHeadMoveBytes (6 KiB, delta.go) is the inner compare cap; maxDeltaBytes (16 KiB) is the outer envelope shared with amendments/comments. headMoveContext treats a listing that exceeds maxHeadMoveBytes the same as a force-push, a 300-file cap, or a forge error: it returns a complete "could not be read / read the pull request yourself" item (the listing-overflow reason even says "longer than the steer budget"). Patch overflow still delivers the full file listing plus a stated cut. In both inner-budget cases, buildText leaves headClipped=false, and markSteered runs if d.headMoved && !w.headClipped { w.lastHead = d.newHead } — the head still advances. TestHeadBaselineAdvancesOnDelivery uses the same unreadable/no-compare fixture as the force-push case and asserts the baseline advances, so the next delta does not repeat the move. lastHead is held only when the outer envelope budget (maxDeltaBytes) clips the entire head-move item out of the message (headClipped=true), which is what TestHeadBaselineHoldsWhenTheListingWasClipped covers — that scenario is driven by amendments exhausting the envelope, not by the file list exceeding maxHeadMoveBytes. Believing the current sentence would lead an engineer or on-call reader to expect a retry of an unreadable/truncated compare that the watcher will never send. This finding was independently identified by both the correctness and docs-currency review dimensions and confirmed by an adversarial re-check against the cited tests.
Suggested fix: Split the two budgets in the prose and do not call maxHeadMoveBytes "the steer budget" in the same sentence as the pinned-head rule. (1) An unreadable compare — force-push, listing at the 300-file cap, file list longer than maxHeadMoveBytes, or a forge error — is delivered as "could not be read"; lastHead advances and the next steer does not repeat the move. A listing or patches that exceed the head-move budget is still delivered (unreadable notice, or partial patches with the cut stated) and advances the head baseline. (2) Only when maxDeltaBytes clips the head-move item out of the envelope — typically because amendments exhausted the room — is the move not taken as told, lastHead stays, and the next steer reports it again.
|
Superseded by #9031; the steer design was reworked on top of ADRs 0133/0141/0144/0151. |
Implements the design recorded in #6957 (closed as completed; this stack ships it).
A run in flight absorbs an update to its work item, and the run queued behind it then does the
same work again — so steering currently costs more than preserving alone: two reviews of one
head where preserving alone produces one. This PR adds the record that closes that, and the check
that reads it.
After a successful run that absorbed at least one update, the runner posts a receipt naming the
follow-up runs it consumed. A queued run reads the latest receipt in its pre-flight — before the
start comment and before the pre-script, whose side effects are not free — and exits 0 without
starting the agent when it finds its own run id listed.
The receipt's author is the whole of the authentication, and it is the GitHub Actions job token's.
The runner captures that token before minting swaps in the role token, so nothing inside the
sandbox holds it. Two conditions are enforced in code rather than assumed:
swapped, the same token stays in the environment a post-script inherits, and both the writer and
the skip check turn off.
github_tokenis acaller-supplied input that can resolve to the identity the agent posts under; the check refuses
to skip when they match, and warns.
No receipt is honoured while any definition in
.fullsend/providers/expandsGH_WORKFLOW_TOKEN.That hands the job token to a sandbox, where the agent can recover it (ADR 0114).
Scope and failure direction: one receipt per run rather than per steer, naming only what the
iteration whose output shipped absorbed, and only when that output was published — a run whose
agent reported an error has its post-script withheld and writes none, as do failed, cancelled,
skipped and unvalidated runs, and runs that absorbed nothing. Everything fails open: a false
"already handled" silently drops work, while a false "not handled" costs one run, which for review is a full review.
The terminal status comment keeps an informational copy of the marker so a reader can see what a
run absorbed. The skip check does not honour it, and agent-authored text is neutralized before it
reaches that comment, so no App-authored marker of any shape is a receipt.
A second skip covers review duplicates no receipt can. When the coder App opens a pull
request and labels it
ready-for-reviewa second later, both events route a review. The run inflight rejects the other as not fresh, absorbs nothing and writes no receipt, so both ran a full
review. A queued review now exits 0 when the review App reviewed the current head after the
queued run was created. It is matched on the App's exact logins and the forge's App verdict. Only
opened, reopened, synchronize, ready-for-review and label events can skip. A human
/fs-review,a manual re-run, an unreadable event and every error run the review. It works with steering on or
off.
The decision is recorded in ADR 0120. Steering stays off by default on this PR.
What to look at
steerReceiptToken— the two conditions above, and why reading the Actions API is deliberatelynot held to the same bar (those reads prove no authorship).
shouldPostSteerReceipt— the post-script gate. It is stricter than the terminal statuscomment on purpose: a receipt instructs another run to do nothing, so it has to mean the output
was published.
steerAlreadyHandled— every path returns "not handled" on error.headAlreadyReviewedandrunPreflightSkips— the second skip, and the single call site thatruns both before the start comment. A
go/asttest pins that call.queued pipeline does the work exactly as it does today.