Skip to content

fix(#7387): surface silent risk-assessment omissions - #7388

Open
fullsend-ai-coder[bot] wants to merge 6 commits into
mainfrom
agent/7387-risk-assessment-observability
Open

fullsend-ai-coder[bot] wants to merge 6 commits into
mainfrom
agent/7387-risk-assessment-observability

Conversation

@fullsend-ai-coder

Copy link
Copy Markdown
Contributor

Summary

Risk-assessment comments were missing on ~15% of review runs (e.g. PR #7385) with no log marker and no PR-visible diagnostic. This change makes those misses observable in workflow logs and on the PR.

Related Issue

#7387

Changes

  • Pi Agent tool emits grep-stable fullsend:persona:{invoke,complete,error,reject,skip} lines for every named persona, including risk-assessment. Session shutdown logs skip reason=never-dispatched for registered personas the orchestrator never called.
  • Bootstrap logs the same skip marker when a persona file fails to register.
  • fullsend post-review parses optional risk_assessment. When REVIEW_RISK_ASSESSMENT_ENABLED is on (ADR 0089 default) and the field is absent, it posts a sticky diagnostic: Risk assessment unavailable this run. Failures posting the diagnostic do not block the review verdict.
  • Docs: runtime persona log examples, review agent post-script behavior, CLI internals, and a short ADR 0089 annotation.

The scored Risk Assessment: N/5 comment is still produced by post-review.sh in fullsend-ai/agents when the field is present. This PR covers the missing-field case in the CLI that script already invokes.

Testing

  • node --test internal/runtime/pi_extension/fullsend-agent.test.mjs — 56 pass, 1 skip (persona lifecycle invoke/complete/error/reject/skip).
  • Targeted Go tests for post-review risk-assessment parsing/diagnostics and persona skip markers — pass. riskAssessmentEnabled 100%, postMissingRiskAssessment 93.8%, parseReviewResult/sanitizeReviewResult 100%, resolvePersonaModels 98.2%.
  • go vet ./internal/cli/ ./internal/runtime/ — pass.
  • Local hook stand-ins (pre-commit could not fetch remote repos): gofmt, ADR lints, lychee, gitleaks, trailing-whitespace/EOF.

Checklist

  • PR title follows Conventional Commits
  • Commits are not DCO-signed (autonomous agent)
  • Changes are limited to the issue scope

Closes #7387

Post-script verification

  • Branch is not main/master (agent/7387-risk-assessment-observability)
  • Secret scan passed (gitleaks — 200a184607d39eeb6b2e8158f8ff698d19337ccb..HEAD)
  • PR body secret scan passed (gitleaks — no-git)

Risk-assessment comments were missing on ~15% of review runs with no
log marker and no PR-visible diagnostic, so a miss could not be told
apart from a skip.

The pi Agent tool now emits grep-stable fullsend:persona:{invoke,
complete,error,reject,skip} lines for every named persona. Bootstrap
also logs skip when a persona file fails to register. When
REVIEW_RISK_ASSESSMENT_ENABLED is on (ADR 0089 default) and the review
result omits risk_assessment, fullsend post-review posts a sticky
"Risk assessment unavailable this run" comment instead of staying
silent.

The agents-repo post-review.sh still posts the scored comment when
the field is present; this CLI path covers the missing-field case
without a companion change there.

Note: pre-commit could not fetch remote hook repos (HTTP 403). Local
equivalents were run: gofmt, go vet, ADR lints, lychee, gitleaks,
trailing-whitespace/EOF.

Closes #7387
@fullsend-ai-coder
fullsend-ai-coder Bot requested a review from a team September 16, 2026 22:01
@fullsend-ai-coder fullsend-ai-coder Bot added the ready-for-review Triggers review agent dispatch label Sep 16, 2026
@fullsend-ai-review

Copy link
Copy Markdown

🤖 Review · ⚠️ Cancelled · Ended 10:03 PM UTC

Commit: 3ea7745 · View workflow run →

@fullsend-ai-review

fullsend-ai-review Bot commented Sep 16, 2026 •

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 10:05 PM UTC · Completed 10:26 PM UTC

Commit: 3ea7745 · View workflow run →

Runtime: pi · Model: sonnet → claude-sonnet-5 · Effort: high · Cost: $6.64

@github-actions

github-actions Bot commented Sep 16, 2026 •

Copy link
Copy Markdown

Site preview

Preview: https://92583653-site.fullsend-ai.workers.dev

Commit: 9e44f387b1c4eb816357766b258ec76ab1df7f8c

@codecov

codecov Bot commented Sep 16, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 82.22222% with 16 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
internal/cli/postreview.go 80.48% 11 Missing and 5 partials ⚠️

📢 Thoughts on this report? Let us know!

@fullsend-ai-review fullsend-ai-review Bot added the risk/moderate PR risk: moderate label Sep 16, 2026
@fullsend-ai-review

fullsend-ai-review Bot commented Sep 16, 2026 •

Copy link
Copy Markdown

Risk Assessment: elevated (3/5)

Details

Re-review anchoring applies: Tier 1 signals (12 files, large blast radius, zero protected/dependency paths, bot author) are unchanged from the prior assessment characteristics, and the incremental delta since the prior SHA is small and additive (two files, mostly test coverage) rather than new functional surface, so the prior score of 3/elevated is preserved.

Previous run

Risk Assessment: elevated (3/5)

Details

Tier 1 signals are not truly unchanged from the prior anchor: this round added new files (sticky.go/sticky_test.go) making a genuine functional change to FindMarkedComment, the single shared matching primitive underlying sticky-comment upsert used across postreview/post-comment flows, alongside a 33% growth in file/line count beyond the previously-assessed additive doc rewording. Blast radius remains large but is now docs-plus-shared-utility rather than mostly docs, warranting elevation to 3/elevated, tempered by strong accompanying test coverage, zero protected-path or dependency changes, and no reverts in history.

Previous run (2)

Risk Assessment: moderate (2/5)

Details

Re-review anchoring applies: Tier 1 signals (10 files, large blast radius via 820 lines, 0 protected paths, no dependency changes, bot author) are unchanged in kind from the prior assessment, and Tier 2 git history reconfirms the same known high-churn/multi-author docs hotspots already factored into the prior moderate score, with no reverts or hack/workaround sentiment found; the incremental diff since the last review (a new marker constant, a collision test, and doc rewording) is additive and low-risk, so the composite score is preserved at 2/moderate rather than adjusted.

Previous run (3)

Risk Assessment: moderate (2/5)

Details

Tier 1 signals (file count, protected-path count, dependency changes, bot authorship) are unchanged from the prior assessment one commit ago, and git history confirms the same two docs files remain high-churn hotspots; this is a fix-agent iteration addressing prior review feedback with no new risk-relevant scope change, so the score is anchored to the prior moderate composite of 2.

Previous run (4)

Risk Assessment: moderate (2/5)

Details

Tier 1 signals are unchanged from the prior review (10 files, no protected/security paths, no dependency changes, bot author); Tier 2 churn/fix-rate hotspots in the touched docs files and a well-matched, low-mismatch linked issue reproduce the same low-moderate composite, so the prior score of 2 is preserved per re-review anchoring.

Previous run (5)

Risk Assessment: moderate (2/5)

Details

Well-scoped, bot-authored observability fix directly implementing a linked, well-matched issue, with no protected or security-sensitive paths and decent test coverage, but two touched docs files are high-churn/high-fix-rate hotspots which lifts the git-history tier to a low-moderate composite score of 2.

@fullsend-ai-review

fullsend-ai-review Bot commented Sep 16, 2026 •

Copy link
Copy Markdown

Review

Findings

Low

  • [scope-creep] internal/cli/postreview.go:394 — The issue (Risk-assessment review sub-agent silently omits its comment on ~15% of PRs (e.g. #7385) #7387) and ADR 0089's "Silent-miss observability" section describe structured lifecycle logging and posting a missing-assessment diagnostic when the field is absent. This PR also introduces a clear/supersede lifecycle (clearMissingRiskAssessmentDiagnostic / clearedRiskAssessmentBody) that overwrites a prior diagnostic sticky with "Risk assessment now present" once a later run succeeds. This is same-domain completeness of a sticky comment (leaving a stale "unavailable" comment would itself be misleading), not a new user-facing feature, and it is covered by TestPostMissingRiskAssessment_SupersedesStaleDiagnosticWhenPresent. Not blocking.
    Remediation: Optional: mention the supersede path explicitly in the PR description or ADR annotation. Not required to land the core fix.
Previous run

Review

This is a re-review. All three findings from the prior round are resolved: sticky.FindMarkedComment now matches by body prefix instead of unanchored substring — verified as a pure narrowing with zero regression risk, since every legitimate sticky comment created via sticky.Post already writes marker + "\n" + body, so no existing caller's (review comment, failure notice) matching behavior changes; parseReviewResult now requires a usable score/level via the new hasUsableRiskAssessment helper before treating a decoded risk_assessment as present, closing the {}/zero-value gap; and the persona skip-marker log fields (name/reason on the Go side, name/reason/model/stopReason on the JS side) are now routed through the same substitution rules as runtime.sanitizeOutput. One new finding surfaced in this round.

Findings

Low

  • [injection-pattern] internal/cli/postreview.go:1001 — parseReviewResult's unusable-risk_assessment path (the branch that fires when the decoded level is not in the ADR 0089 enum — the malicious/garbage-input case this PR is specifically hardening against) writes ra.Level to a GHA-visible stderr WARNING line with only %q, bypassing sanitizeRiskLevel, which this same PR added to allowlist Level before other output paths (e.g. printer.StepInfo). The WARNING: line prefix and %q's newline-escaping rule out a ::error::/::add-mask:: line-start injection, but this is still an incomplete application of the PR's own new allowlist to agent-authored JSON in the one code path designed to see malformed values.
    Remediation: Interpolate sanitizeRiskLevel(ra.Level) into the WARNING instead of raw ra.Level. Extend the parse tests so a level of ::error::pwned appears in stderr only as unknown (or otherwise broken/quoted).

Next steps:

  • /fs-fix — agent addresses review findings automatically
  • /fs-fix <your instruction> — agent fixes with your specific guidance
  • Push commits directly — review re-runs automatically on push
  • /fs-fix-stop — disable automatic fix runs for this PR
Previous run (2)

Review

This is a re-review. The two findings carried from the prior round are both resolved: the diagnostic marker no longer collides with the ADR 0089 score-card marker (verified via the new missingRiskAssessmentMarker constant and its non-containment test), and docs/agents/review.md now explains that REVIEW_RISK_ASSESSMENT_ENABLED is read independently in env.sandbox (pre-pass dispatch) and env.runner (diagnostic posting), and that both must be set together. Two findings surfaced in this round, both should be addressed before merge.

Findings

Medium

  • [injection-pattern] internal/cli/postreview.go:368 — postMissingRiskAssessment/clearMissingRiskAssessmentDiagnostic locate the diagnostic sticky via sticky.FindMarkedComment, which matches via unanchored strings.Contains on bot-authored comment bodies. sanitizeReviewResult's output pipeline only redacts secrets/Unicode obfuscation — it does not strip <!-- fullsend: sequences from agent-authored Body/Findings/Rationale. If a bot-authored review sticky (marker <!-- fullsend:review-agent -->) ever quotes the literal string <!-- fullsend:risk-assessment-missing --> in its body — plausible, since ADR 0089 and this PR now document that exact string and a future review finding could cite it — FindMarkedComment can select that review comment instead of (or as well as) any real diagnostic comment, and sticky.Post would then overwrite it with diagnostic text. The blast radius is narrower than a first read suggests: comments are listed oldest-first, so an existing real diagnostic (older) is preferred over a coincidental later match, and with the default keepHistory=true a subsequent review Post usually retargets/restores the same comment. The damaging window is mainly the keepHistory=false case or when no real diagnostic sticky exists yet. The PR's new test only asserts the two marker constants are not substrings of each other; it does not test marker-vs-agent-authored-comment-body collision.
    Remediation: Select the diagnostic comment by requiring the marker as a body prefix (mirroring how BuildUpdatedBody already strips markers by prefix, and how sticky.Post already writes marker + "\n" + body) instead of unanchored Contains, for both postMissingRiskAssessment and clearMissingRiskAssessmentDiagnostic. Add a regression test: a prior bot review comment whose body contains the marker substring but does not start with it must not be selected or overwritten.

Low

  • [logic-error] internal/cli/postreview.go:349 — parseReviewResult treats any successfully-decoded JSON object as a present risk_assessment, including {} or an object missing ADR 0089's required score/level fields — these decode into a zero-value struct rather than leaving the pointer nil. postMissingRiskAssessment then logs "present: unknown (0/5)" and calls clearMissingRiskAssessmentDiagnostic instead of posting the "unavailable" diagnostic, which is inconsistent with how null and wrong-typed values are handled (both correctly leave the pointer nil). Existing tests cover a fully-populated object, a missing key, and a wrong-typed value, but not {} or a zero-value object.
    Remediation: If the diagnostic is meant to catch "no usable ADR 0089 assessment" rather than only "key omitted," leave the pointer nil unless score is in 1–5 and level is one of the allowlisted values. Add parse + postMissingRiskAssessment tests for null, {}, and an object missing score/level.

  • [injection-pattern] internal/runtime/pi_personas.go:185 — The new fullsend:persona:skip lifecycle lines written to runner/GHA-visible stderr use formatPersonaLogName, which only collapses [\r\n\t]+. This repo has an established, more complete convention for sanitizing exactly this kind of untrusted-output-to-log-sink text — runtime.sanitizeOutput (internal/runtime/sanitize.go) — which also breaks :: workflow-command sequences, replaces %0A/%0a/%0D/%0d, and strips ANSI/OSC escapes. The discovery-skip site interpolates sk.Name (a skipped persona file's basename, which by definition has not passed ValidSubagentKey and can contain :: or ANSI/URL-encoded sequences) through formatPersonaLogName only, and sk.Reason through %q (escapes quotes/newlines but not ::). The mirrored JS logPersona helper in fullsend-agent.js applies the same newline-only filter to agent-controlled fields (subagent_type, reject reasons, model, stopReason), and those lines are copied to os.Stderr unsanitized. Impact is workflow-command injection into GHA logs (spurious ::error::/::add-mask:: annotations), not RCE, since the deprecated ::set-env::/::set-output:: commands require ACTIONS_ALLOW_UNSECURE_COMMANDS=true.
    Remediation: Route the interpolated fields in both Go skip-marker Fprintf calls and the JS logPersona helper through the same substitution rules as runtime.sanitizeOutput before writing to stderr. Extend the existing tests (which only cover embedded newlines) to also cover ::error::-style and %0A-encoded payloads in a skipped persona's file-derived name/reason.


Next steps:

  • /fs-fix — agent addresses review findings automatically
  • /fs-fix <your instruction> — agent fixes with your specific guidance
  • Push commits directly — review re-runs automatically on push
  • /fs-fix-stop — disable automatic fix runs for this PR
Previous run (3)

Review

This is a re-review. Six of the seven findings from the prior round are confirmed fixed: the shutdown() never-dispatched mislabeling, parseReviewResult discarding a valid result on a malformed nested risk_assessment, the Level allowlisting and discovery-skip quoting hygiene fixes, the stale-diagnostic-supersede fix, and the three-site fullsend:persona:skip quoting unification (verified against formatPersonaLogName and the new shared personaSkipNameRe test on both the Go and JS sides). Two new findings surfaced in this round.

Findings

Medium

  • [cross-repo-architecture-boundary] internal/cli/postreview.go:342 — This CLI now posts and supersedes a sticky comment under the marker <!-- fullsend:risk-assessment -->, which is the same marker the review pipeline's own orchestrator treats as the ADR 0089 score card (fetched via a contains("<!-- fullsend:risk-assessment -->") lookup for the scored **Risk Assessment: <level> (<score>/5)** comment). Both clearMissingRiskAssessmentDiagnostic (the present-value path, the common case) and postMissingRiskAssessment (the absent-value path) call sticky.Post with that marker. sticky.FindMarkedComment matches via strings.Contains on any bot-authored comment body — it does not require the marker as a prefix — so if the actual score sticky already exists, it is the "existing" match these code paths select and update. On the present-value path this would overwrite the score card with the clearedRiskAssessmentBody ("Risk assessment now present") text, dropping the score/level/rationale, unless fullsend-ai/agents' post-review.sh rewrites the same comment afterward (order-dependent, and post-review.sh is not checked out in this sandbox to verify). Separately, since BuildUpdatedBody only strips markers by prefix, a Contains match could also attach this logic to a review-agent sticky whose body happens to echo the marker substring (the output security pipeline does not strip embedded HTML-comment markers) — the two writers (this CLI and the agents-repo script) currently disagree about what this one marker identifies.
    Remediation: Use a distinct diagnostic marker (e.g. <!-- fullsend:risk-assessment-missing -->) so this code path cannot select or update the score sticky. Consider tightening FindMarkedComment (or a local wrapper) to require the marker as a body prefix, matching how BuildUpdatedBody already treats it. Confirm with a fullsend-ai/agents companion change that post-review.sh still owns the score comment after this change, and add a short ADR 0089 note documenting the split.

  • [stale-doc] docs/agents/review.md:94 — The new guidance correctly states that fullsend post-review reads REVIEW_RISK_ASSESSMENT_ENABLED from the runner's own environment (env.runner), but it doesn't mention that the ADR 0089 risk-assessment pre-pass itself — whether the persona is dispatched at all — is gated by the same variable name read from env.sandbox by the review orchestrator. Setting the variable only in env.runner suppresses the diagnostic but still dispatches the persona (spending tokens on a disabled feature); setting it only in env.sandbox stops dispatch but leaves the runner-side default of true in effect, so the diagnostic sticky still fires on every run. Fully enabling or disabling the feature requires setting the variable consistently in both places, which this doc doesn't state.
    Remediation: Note that REVIEW_RISK_ASSESSMENT_ENABLED is read in two separate execution contexts — the sandbox (gating pre-pass dispatch) and the runner (gating the post-script diagnostic) — and that both must be set together via harness base: composition to fully enable or disable the feature.


Next steps:

  • /fs-fix — agent addresses review findings automatically
  • /fs-fix <your instruction> — agent fixes with your specific guidance
  • Push commits directly — review re-runs automatically on push
  • /fs-fix-stop — disable automatic fix runs for this PR
Previous run (4)

Review

This is a re-review. Six of the seven prior findings are confirmed fixed (persona shutdown mislabeling, parseReviewResult discarding valid results on a malformed nested risk_assessment, the two log/printer injection-hygiene nits, and both doc-wording fixes). The remaining prior finding — cross-process propagation of REVIEW_RISK_ASSESSMENT_ENABLED — was left unaddressed by design and is carried forward below, now with a concrete in-repo instance of the risk it describes.

Findings

Medium

  • [logic-error] internal/cli/postreview.go:333 — postMissingRiskAssessment never clears or updates a previously-posted "Risk assessment unavailable this run" sticky comment once a later run on the same PR does produce a risk_assessment. The present-value path only logs and returns, so a stale "unavailable this run" comment is left on the PR describing a run that in fact succeeded.
    Remediation: On the present-value path, update or clear the existing diagnostic sticky (e.g. sticky.Post a superseded/cleared body under the same marker, or find-and-minimize/delete it). Add a test asserting the stale comment is superseded once risk_assessment becomes present.

  • [api-contract] internal/cli/postreview.go:307 — riskAssessmentEnabled() reads REVIEW_RISK_ASSESSMENT_ENABLED from the post-script's own environment (RunnerEnv/env.runner), not from env.sandbox. This PR's own new documentation row in docs/agents/review.md sits directly above pre-existing guidance to "set this in the harness's env.sandbox" and states "the post-script reads the value from the runner environment directly — no separate configuration is needed" — which is not true for this specific flag. An operator following this PR's own docs to disable the diagnostic via env.sandbox will still get it posted on every review. See also: the cross-repo-contract finding below at the same location.
    Remediation: Correct the Variables table/guidance so REVIEW_RISK_ASSESSMENT_ENABLED is documented as requiring env.runner (or wherever the post-script actually reads it), and drop the "no separate configuration is needed" claim for this flag.

  • [cross-repo-contract] internal/runtime/pi_personas.go:297 — The PR's core purpose is a "grep-stable" fullsend:persona:<action> marker contract, but the three emission sites use three different quoting rules for the same skip marker's name field: discovery-skip always double-quotes (%q), registered-persona skip leaves it bare (%s), and the JS emitter quotes only when whitespace is present. The documented example (docs/runtimes/pi.md) and the diagnostic's own grep hint only show the unquoted JS form, so a scraper written against the documented format will silently miss the Go discovery-skip form for the identical marker/action.
    Remediation: Pick one quoting convention and apply it at all three emission sites; add a test that greps both Go- and JS-emitted lines with one shared regex.

  • [cross-repo-contract] internal/cli/postreview.go:307 — Carried forward from the prior review (code here is unchanged; the fix agent explicitly declined this one as out of scope). Whether the risk-assessment persona was expected to run is decided by one process (the sandbox orchestrator, via env.sandbox); whether the diagnostic gets posted is decided by a different process (fullsend post-review) independently re-reading the same-named env var, defaulting to true when unset. An install that disables the flag only for the sandbox step will still receive an unrequested sticky comment on every review — this PR's own new documentation (see the api-contract finding above) now demonstrates this exact gap concretely rather than only theoretically.
    Remediation: File and link an explicit tracked follow-up against fullsend-ai/agents to confirm REVIEW_RISK_ASSESSMENT_ENABLED propagation into env.runner for every harness-composition path. Consider carrying an explicit "risk assessment expected" bit in the review-result JSON instead of an independently-read, same-named env var.


Next steps:

  • /fs-fix — agent addresses review findings automatically
  • /fs-fix <your instruction> — agent fixes with your specific guidance
  • Push commits directly — review re-runs automatically on push
  • /fs-fix-stop — disable automatic fix runs for this PR
Previous run (5)

Review

Findings

Medium

  • [logic-error] internal/runtime/pi_extension/fullsend-agent.js:890 — shutdown() labels every registered persona not yet in dispatchedPersonas as skip reason=never-dispatched, but dispatchedPersonas is only populated at invoke time (after acquire() succeeds), in rejectNamed, or in skipNamed. A persona whose dispatch is still queued behind maxConcurrent (default 4) when shutdown() fires is mislabeled never-dispatched — inverting the exact invoke/skip/error distinction Risk-assessment review sub-agent silently omits its comment on ~15% of PRs (e.g. #7385) #7387 asks this PR to add. skipNamed then no-ops because the name is already recorded, so the abort is never logged with an accurate reason either. The existing test only covers a persona that was never invoked at all, not one queued behind the concurrency limit.
    Remediation: Mark a persona as dispatched at the start of run() (once lookupPersona matches), before await acquire(), so never-dispatched is only emitted for personas whose run() was never entered. Have skipNamed log a distinct reason (e.g. aborted/shutdown) for in-flight/queued calls. Add a test that queues a persona behind maxConcurrent=1 and calls shutdown() while it is still queued, asserting a skip reason other than never-dispatched.

  • [api-contract] internal/cli/postreview.go:885 — parseReviewResult falls back to Action:"comment" with the raw input as Body on any json.Unmarshal error. Adding the optional risk_assessment field means a type mismatch on that field alone (e.g. "risk_assessment": "oops", or "score": "2" as a string) now fails the whole unmarshal, discarding an otherwise-valid action/body/findings/head_sha. The review is then posted as a comment-only review with the raw JSON blob as its body, and the new missing-risk-assessment diagnostic still fires (since RiskAssessment decoded to nil). No test covers a malformed risk_assessment value alongside an otherwise-valid result.
    Remediation: Decode risk_assessment separately (e.g. via json.RawMessage or a two-pass decode) so a type mismatch there doesn't fail the top-level parse. On a bad nested value, log a warning and leave RiskAssessment nil rather than discarding the whole result. Add a test asserting {"body":"ok","action":"approve","risk_assessment":"oops"} still yields Action="approve", Body="ok".

  • [incomplete-doc] docs/agents/review.md:85 — Step 4 of "How the agent works" was updated to describe REVIEW_RISK_ASSESSMENT_ENABLED and the new sticky diagnostic, but the ### Variables reference table later in the same file still lists only REVIEW_FINDING_SEVERITY_THRESHOLD. Operators who use that table to discover overridable review env vars will miss a default-on flag that posts a new PR comment.
    Remediation: Add a row to the Variables table for REVIEW_RISK_ASSESSMENT_ENABLED: purpose (gates the risk-assessment pre-pass and the missing-assessment diagnostic), default (true), valid values (true/false, or the false-synonyms the CLI recognizes: false/0/no/off).

  • [cross-repo-contract] internal/cli/postreview.go:306 — riskAssessmentEnabled() reads REVIEW_RISK_ASSESSMENT_ENABLED from the fullsend post-review CLI process's own environment (default true), and postMissingRiskAssessment unconditionally posts a new sticky diagnostic whenever that flag is true and risk_assessment is absent. "Was the persona expected?" is decided in a different process (the sandbox orchestrator, gated via post-review.sh in fullsend-ai/agents, not checked out in this sandbox) than the one that posts the diagnostic. If an install sets REVIEW_RISK_ASSESSMENT_ENABLED=false only for the sandbox step but the runner-side post-review step doesn't inherit that override, every review from that install starts receiving an unrequested sticky comment — a default-on behavior change for existing callers, not just ones opting into ADR 0089. This can't be verified from this repo alone.
    Remediation: Confirm with fullsend-ai/agents that REVIEW_RISK_ASSESSMENT_ENABLED is present in the same environment fullsend post-review runs in for every harness-composition path that can override it. Consider carrying an explicit "risk assessment expected" bit in the review-result JSON, set by the orchestrator that actually decided whether to run the persona, instead of having the CLI independently re-read a same-named env var it doesn't control the propagation of.

Low

  • [injection-pattern] internal/cli/postreview.go:320 — postMissingRiskAssessment interpolates parsed.RiskAssessment.Level into printer.StepInfo, which only prefixes the first line with indentation and does not strip embedded newlines. sanitizeReviewResult scans Rationale through the security output pipeline but not Level; that pipeline does not strip newlines/:: sequences anyway, so it wouldn't fully mitigate a log-line breakout even if applied. Level is a 5-value enum, which makes this a straightforward hygiene fix rather than a deep defect.
    Remediation: Allowlist Level against the ADR 0089 enum (low/moderate/elevated/high/critical) before interpolating it into any printer call, substituting a fixed placeholder for anything else.

  • [injection-pattern] internal/runtime/pi_personas.go:292 — The new skip marker for discovery-skipped personas logs the candidate name with %s without stripping newlines, unlike the sibling skip-marker for registered personas (whose name has already passed ValidSubagentKey) and unlike the reason parameter (which is %q-escaped).
    Remediation: Format the name with %q like reason, or gate emission on the name matching ValidSubagentKey.

  • [incorrect-doc] docs/ADRs/0089-pr-risk-assessment-scoring.md:161 — The new "Silent-miss observability" section says the CLI posts a diagnostic "instead of omitting the comment." The main review comment was never omitted when risk_assessment is missing; what's new is an independent second sticky comment that previously didn't exist at all. The phrasing is ambiguous about which comment was previously omitted — docs/runtimes/pi.md already uses the clearer "instead of staying silent."
    Remediation: Reword to "instead of staying silent about the missing assessment" or "instead of omitting any diagnostic notice."


Next steps:

  • /fs-fix — agent addresses review findings automatically
  • /fs-fix <your instruction> — agent fixes with your specific guidance
  • Push commits directly — review re-runs automatically on push
  • /fs-fix-stop — disable automatic fix runs for this PR

fullsend-ai-review[bot]

This comment was marked as outdated.

fullsend-ai-review[bot]

This comment was marked as outdated.

@fullsend-ai-coder

fullsend-ai-coder Bot commented Sep 16, 2026 •

Copy link
Copy Markdown
Contributor Author

🤖 Finished Fix · ✅ Success · Started 10:28 PM UTC · Completed 10:47 PM UTC

Commit: 3ea7745 · View workflow run →

Runtime: claude · Model: sonnet → claude-sonnet-5 · Effort: high · Cost: $3.07

Fix findings from the review agent on PR #7388 (silent risk-assessment
omissions, #7387):

- fullsend-agent.js: shutdown() decided "never-dispatched" from
  dispatchedPersonas, which is only populated at invoke/reject/skip time.
  A persona still queued behind maxConcurrent when shutdown() fired had
  none of those yet, so it was mislabeled never-dispatched, and its own
  skipNamed call then no-opped because the name was already recorded.
  Track "run() entered" separately (startedPersonas) so shutdown only
  labels personas whose run() was never called, and a queued call now
  gets its own accurate skip reason (e.g. aborted) instead.
- postreview.go: parseReviewResult failed the whole JSON parse on any
  type mismatch in the optional risk_assessment field, discarding an
  otherwise-valid action/body/findings. Decode risk_assessment via
  json.RawMessage in a second pass so a malformed nested value logs a
  warning and leaves RiskAssessment nil instead of losing the result.
- postreview.go: allowlist RiskAssessment.Level against the ADR 0089
  enum before interpolating it into printer output.
- pi_personas.go: format the discovery-skip persona name with %q (like
  reason) instead of %s, since that name comes from an unvalidated
  filename.
- docs/agents/review.md: add REVIEW_RISK_ASSESSMENT_ENABLED to the
  Variables reference table.
- docs/ADRs/0089: reword "instead of omitting the comment" (ambiguous
  about which comment) to "instead of staying silent about the missing
  assessment".

Not addressed (disagreement recorded): the cross-repo-contract finding
on whether REVIEW_RISK_ASSESSMENT_ENABLED propagates consistently
between the sandbox orchestrator and the fullsend post-review process —
that can only be verified against fullsend-ai/agents, which isn't
checked out here.

Addresses #7388
@fullsend-ai-coder

Copy link
Copy Markdown
Contributor Author

🔧 Fix agent — iteration 1 (bot-triggered)

Fixed the logic error where shutdown() mislabeled a queued persona as never-dispatched (with a regression test), fixed parseReviewResult discarding a whole valid result on a malformed nested risk_assessment value, allowlisted RiskAssessment.Level before log interpolation, escaped the discovery-skip persona name with %q, added the missing REVIEW_RISK_ASSESSMENT_ENABLED doc row, and reworded the ambiguous ADR sentence. Recorded a disagreement on the cross-repo-contract finding, which requires coordination with fullsend-ai/agents that this sandbox can't perform. All new and existing tests pass (node --test: 57 pass/1 skip; go test ./internal/cli/... ./internal/runtime/... pass except two pre-existing, unrelated failures confirmed present before this fix's changes).

Fixed (6):

  1. [logic-error] shutdown() mislabels a persona queued behind maxConcurrent as never-dispatched (internal/runtime/pi_extension/fullsend-agent.js): Added a separate startedPersonas set marked at the top of run() (once lookupPersona matches, before await acquire()). shutdown()'s never-dispatched sweep now checks startedPersonas instead of dispatchedPersonas, so a persona still queued when shutdown fires is no longer double-counted as never attempted; its own skipNamed call (previously a no-op because dispatchedPersonas already held the name) now logs an accurate reason (e.g. aborted). Added a regression test that queues a persona behind maxConcurrent=1 and shuts down while queued, asserting the skip reason is not never-dispatched.
  2. [api-contract] parseReviewResult discards a valid result on a risk_assessment type mismatch (internal/cli/postreview.go): Introduced reviewResultRaw with RiskAssessment as json.RawMessage so the top-level unmarshal no longer fails on a malformed nested risk_assessment value. risk_assessment is now decoded in a second pass; on failure it logs a warning to stderr and leaves RiskAssessment nil instead of discarding action/body/findings. Added TestParseReviewResult_MalformedRiskAssessmentDoesNotDiscardResult covering {"body":"ok","action":"approve","risk_assessment":"oops"}.
  3. [incomplete-doc] Variables table in docs/agents/review.md omits REVIEW_RISK_ASSESSMENT_ENABLED (docs/agents/review.md): Added a row for REVIEW_RISK_ASSESSMENT_ENABLED (default true, valid values true/false plus the false-synonyms 0/no/off the CLI recognizes) to the Variables reference table.
  4. [injection-pattern] postMissingRiskAssessment interpolates RiskAssessment.Level into printer.StepInfo without allowlisting (internal/cli/postreview.go): Added sanitizeRiskLevel(), which allowlists Level against the ADR 0089 enum (low/moderate/elevated/high/critical) and substitutes "unknown" for anything else, and used it at the Level interpolation site. Added TestSanitizeRiskLevel covering valid values, an embedded-newline/log-injection attempt, and empty.
  5. [injection-pattern] discovery-skip marker logs the candidate persona name with %s instead of %q (internal/runtime/pi_personas.go): Changed the discovery-skip fullsend:persona:skip marker to format name with %q (matching reason), since that name comes from an unvalidated file basename rather than an already-registered, ValidSubagentKey-passed persona name. Updated the existing marker test for the now-quoted output and added TestResolvePersonaModels_DiscoverySkipEscapesName asserting a name containing a newline is escaped on one line rather than breaking the marker across lines.
  6. [incorrect-doc] ADR 0089 "instead of omitting the comment" is ambiguous about which comment (docs/ADRs/0089-pr-risk-assessment-scoring.md): Reworded to "instead of staying silent about the missing assessment", matching the phrasing already used in docs/runtimes/pi.md, to make clear the new diagnostic is an independent second comment rather than a previously-omitted main review comment.

Disagreed (1):

  1. [cross-repo-contract] riskAssessmentEnabled() reads an env var whose cross-process propagation this repo can't verify: The finding itself states it can't be verified from this repo alone: whether REVIEW_RISK_ASSESSMENT_ENABLED is present in the same environment fullsend post-review runs in is decided by post-review.sh in fullsend-ai/agents, which is not checked out in this sandbox. The suggested alternative (carry an explicit 'risk assessment expected' bit in the review-result JSON, set by the orchestrator) is a cross-repo protocol change beyond this PR's scope of surfacing silent misses, and making it unilaterally here risks getting the new field's producer/consumer contract wrong without fullsend-ai/agents coordination. Left as-is; flagging for follow-up coordination with that repo.

Tests: passed

Decision points
  • Track a persona's run()-entry separately from its terminal-event dedup set rather than reusing dispatchedPersonas (alternatives: Mark dispatchedPersonas early (at run() start) and reuse it for both purposes, Add a new startedPersonas set used only to gate shutdown's never-dispatched sweep; rationale: Reusing dispatchedPersonas for early marking would make skipNamed's existing dedup check (!dispatchedPersonas.has(personaName)) always false for a queued call, silently suppressing its skip log entirely. A second set keeps the existing invoke/reject/skip dedup semantics intact while giving shutdown an accurate signal for 'never entered run() at all'.)
  • Log the malformed risk_assessment warning via fmt.Fprintf(os.Stderr, "WARNING: ...") instead of threading a *ui.Printer into parseReviewResult (alternatives: Change parseReviewResult's signature to accept a printer and update all ~9 call sites in postreview_test.go, Use the repo's existing fmt.Fprintf(os.Stderr, "WARNING: ...") convention (used elsewhere in internal/cli, e.g. bootstrap_scan.go) without changing the signature; rationale: Matches an existing in-repo convention for this kind of local warning and keeps the fix minimal — avoids an unrelated signature change rippling through every existing test call site.)

Next steps:

  • /fs-review — request a re-review of the changes
  • /fs-fix <your instruction> — run another fix pass with specific guidance
  • Push commits directly — review re-runs automatically on push
    Updated by fullsend fix agent

@fullsend-ai-review

fullsend-ai-review Bot commented Sep 16, 2026 •

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 10:49 PM UTC · Completed 11:12 PM UTC

Commit: f588a79 · View workflow run →

Runtime: pi · Model: sonnet → claude-sonnet-5 · Effort: high · Cost: $8.34

fullsend-ai-review[bot]

This comment was marked as outdated.

@fullsend-ai-coder

fullsend-ai-coder Bot commented Sep 16, 2026 •

Copy link
Copy Markdown
Contributor Author

🤖 Finished Fix · ✅ Success · Started 11:14 PM UTC · Completed 11:29 PM UTC

Commit: f588a79 · View workflow run →

Runtime: claude · Model: sonnet → claude-sonnet-5 · Effort: high · Cost: $3.00

- Supersede the stale "Risk assessment unavailable this run" sticky
  diagnostic once a later run on the same PR does produce a
  risk_assessment, instead of leaving it describing a run that in
  fact succeeded.
- Correct docs/agents/review.md: REVIEW_RISK_ASSESSMENT_ENABLED is
  read by fullsend post-review from the runner's own environment
  (env.runner), not the sandbox — unlike
  REVIEW_FINDING_SEVERITY_THRESHOLD, which is read in-sandbox. Drop
  the inaccurate "no separate configuration is needed" claim.
- Unify the fullsend:persona:skip marker's name= quoting across all
  three emission sites (Go discovery-skip, Go registered-persona
  skip, JS logPersona) on one convention: collapse control characters
  and quote only when whitespace remains. Add a Go and a JS test that
  both match emitted lines against the same shared regex.

Carries forward the cross-repo-contract finding on
REVIEW_RISK_ASSESSMENT_ENABLED propagation as a recorded disagreement
(out of scope for this repo; requires a tracked follow-up against
fullsend-ai/agents, not a code change here).

Addresses #7388
@fullsend-ai-coder

Copy link
Copy Markdown
Contributor Author

🔧 Fix agent — iteration 2 (bot-triggered)

Fixed the three current-round findings: superseding the stale risk-assessment sticky once a later run succeeds, correcting docs/agents/review.md's env.runner vs env.sandbox guidance for REVIEW_RISK_ASSESSMENT_ENABLED, and unifying the fullsend:persona:skip name= quoting convention across the Go and JS emission sites with a shared-regex test on each side. Recorded a disagreement on the carried-forward cross-repo-contract finding, which requires action in a different repository and was already declined as out of scope in the prior iteration.

Fixed (3):

  1. postMissingRiskAssessment never clears a stale 'Risk assessment unavailable this run' sticky once a later run produces risk_assessment (internal/cli/postreview.go:333) (internal/cli/postreview.go): Added clearMissingRiskAssessmentDiagnostic, called from the present-value path in postMissingRiskAssessment. It looks up the existing risk-assessment sticky via sticky.FindMarkedComment and, only if one exists, supersedes it in-place with a 'Risk assessment now present' body via sticky.Post under the same marker. It never creates a new comment when none exists. Added TestPostMissingRiskAssessment_SupersedesStaleDiagnosticWhenPresent asserting the stale comment is updated in-place (not duplicated) and no longer shows the stale text.
  2. Docs say to set REVIEW_RISK_ASSESSMENT_ENABLED in env.sandbox, but the CLI reads it from env.runner (internal/cli/postreview.go:307) (docs/agents/review.md): Split the guidance paragraph in the Variables section: REVIEW_FINDING_SEVERITY_THRESHOLD (read in-sandbox) still documents env.sandbox; REVIEW_RISK_ASSESSMENT_ENABLED now documents env.runner, with an explicit note that setting it only in env.sandbox will not affect the post-script's diagnostic. Removed the inaccurate 'no separate configuration is needed' claim for this flag.
  3. Three fullsend:persona:skip emission sites use three different name= quoting rules (internal/runtime/pi_personas.go:297) (internal/runtime/pi_personas.go): Added a shared formatPersonaLogName helper in pi_personas.go that collapses embedded control characters and quotes the name only when whitespace remains — the same convention the JS runtime's logPersona already uses. Both Go emission sites now call it (discovery-skip previously always used %q; registered-persona skip previously always used bare %s). Updated the two existing Go tests whose assertions encoded the old inconsistent output, and added a new test (Go: TestResolvePersonaModels_SkipMarkerNameMatchesSharedRegex; JS: 'logPersona name field matches the shared skip-name regex') that both apply the identical shared regex to their respective emitted lines, proving one scraper pattern now matches every site.

Disagreed (1):

  1. cross-repo-contract at internal/cli/postreview.go:307: file and link an explicit tracked follow-up against fullsend-ai/agents to confirm REVIEW_RISK_ASSESSMENT_ENABLED propagation into env.runner for every harness-composition path: This is carried forward from the prior review and was already explicitly declined as out of scope in the previous fix iteration. The remediation asks for filing a tracked issue in a different repository (fullsend-ai/agents) and/or a design change to the review-result JSON contract owned by that repo's orchestrator — neither is a code change available to make in this repository, and the fix agent has no ability to file cross-repo issues. The related api-contract/doc finding at the same location (this PR's own doc contradiction) has been fixed.

Tests: passed

Decision points
  • Chose to make clearMissingRiskAssessmentDiagnostic only update an existing diagnostic sticky, never create one, on the present-risk_assessment path (alternatives: Always call sticky.Post with a 'present' body on every run, creating a new comment when none exists, Only update in-place when a stale diagnostic is already present (chosen); rationale: A present risk assessment is the success/expected path and does not itself warrant a new PR comment; only superseding an existing stale diagnostic avoids adding noise to PRs where risk assessment was never missing.)
  • Chose the JS logPersona's 'collapse control chars, quote only if whitespace remains' convention as the single quoting rule for the name= field, applied to both Go emission sites (alternatives: Always double-quote name at all three sites, Never quote name at all three sites (drop safety escaping), Collapse-then-conditionally-quote at all three sites (chosen); rationale: Matches the already-documented example format (unquoted for ordinary names) for the common case, while still preventing an untrusted discovery-skip filename with embedded newlines from breaking the grep-stable marker across lines.)

Next steps:

  • /fs-review — request a re-review of the changes
  • /fs-fix <your instruction> — run another fix pass with specific guidance
  • Push commits directly — review re-runs automatically on push
    Updated by fullsend fix agent

@fullsend-ai-review

fullsend-ai-review Bot commented Sep 16, 2026 •

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 11:30 PM UTC · Completed 11:52 PM UTC

Commit: 7a0acc0 · View workflow run →

Runtime: pi · Model: sonnet → claude-sonnet-5 · Effort: high · Cost: $7.23

fullsend-ai-review[bot]

This comment was marked as outdated.

@fullsend-ai-coder

fullsend-ai-coder Bot commented Sep 16, 2026 •

Copy link
Copy Markdown
Contributor Author

🤖 Finished Fix · ✅ Success · Started 11:54 PM UTC · Completed 12:05 AM UTC

Commit: 7a0acc0 · View workflow run →

Runtime: claude · Model: sonnet → claude-sonnet-5 · Effort: high · Cost: $1.52

- Use a distinct marker (<!-- fullsend:risk-assessment-missing -->) for
  the missing-risk-assessment diagnostic sticky, separate from
  <!-- fullsend:risk-assessment --> which post-review.sh (in
  fullsend-ai/agents) uses to find and update the ADR 0089 score-card
  comment. sticky.FindMarkedComment matches via strings.Contains, so
  the shared marker could let this CLI select and overwrite the score
  card. Added a regression test asserting the two markers cannot
  collide as substrings, and a short ADR 0089 note documenting the
  split. Renamed the constant to missingRiskAssessmentMarker throughout.
- Clarified docs/agents/review.md: REVIEW_RISK_ASSESSMENT_ENABLED is
  read independently by the sandbox orchestrator (gates whether the
  risk-assessment pre-pass is dispatched) and by fullsend post-review
  in env.runner (gates the missing-assessment diagnostic). Both must
  be set together to fully enable or disable the feature.

Pre-commit could not fetch remote hook repositories (sandboxed network
policy); ran the equivalent hooks directly instead (gofmt, go vet,
lint-adr-status, lint-adr-numbers, lint-adr-frontmatter, lint-docs-links,
lychee, trailing-whitespace/EOF/merge-conflict checks) — all pass.

Addresses #7388
@fullsend-ai-review

fullsend-ai-review Bot commented Sep 17, 2026 •

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 12:07 AM UTC · Completed 12:31 AM UTC

Commit: bb6331b · View workflow run →

Runtime: pi · Model: sonnet → claude-sonnet-5 · Effort: high · Cost: $8.65

fullsend-ai-review[bot]

This comment was marked as outdated.

@fullsend-ai-coder

fullsend-ai-coder Bot commented Sep 17, 2026 •

Copy link
Copy Markdown
Contributor Author

🤖 Finished Fix · ✅ Success · Started 12:33 AM UTC · Completed 12:57 AM UTC

Commit: bb6331b · View workflow run →

Runtime: claude · Model: sonnet → claude-sonnet-5 · Effort: high · Cost: $4.56

- sticky.FindMarkedComment: match a marker as a body prefix instead of
  an unanchored substring. Post always writes marker+"\n"+body (and
  BuildUpdatedBody preserves that prefix across updates), so a
  legitimate sticky comment's body always starts with its marker.
  Contains-based matching let postMissingRiskAssessment /
  clearMissingRiskAssessmentDiagnostic select (and overwrite) an
  unrelated bot-authored comment that merely quoted another marker
  string mid-body — plausible now that ADR 0089 and this PR document
  the literal diagnostic marker text. Added regression tests in both
  internal/sticky and internal/cli covering the exact collision
  scenario (a review comment quoting the diagnostic marker).
- postreview.go: parseReviewResult now rejects a successfully-decoded
  risk_assessment that lacks a usable score (1-5) and level (the ADR
  0089 enum) — e.g. {} or an object missing those keys decodes into a
  zero-value struct rather than leaving the pointer nil, which was
  silently treated as a present assessment. Added
  hasUsableRiskAssessment and tests for {}, a missing-fields object, an
  out-of-range score, and an unknown level, plus an end-to-end
  postMissingRiskAssessment test for the {} case.
- pi_personas.go / fullsend-agent.js: route the fields interpolated
  into fullsend:persona:skip and fullsend:persona:reject marker lines
  (persona name, skip/reject reason) through the same substitution
  rules as runtime.sanitizeOutput before writing to stderr, not just a
  \r\n\t collapse. A skipped persona's file-derived name (never passed
  ValidSubagentKey) and reason text can carry "::"-delimited GHA
  workflow-command syntax or %0A-encoded newlines that the old
  collapse-only formatPersonaLogName let through unbroken. Mirrored the
  same rules in the JS logPersona helper's sanitizePersonaLogField,
  since subagent_type, reject reasons, model, and stopReason are
  equally agent-controlled there. Extended the Go and JS test suites
  with cases for "::error::"-style and %0A-encoded payloads in both the
  name and reason fields.

Pre-commit could not fetch remote hook repositories (sandboxed network
policy); ran the equivalent hooks directly instead (gofmt, go vet,
gitleaks, trailing-whitespace/EOF/merge-conflict/CRLF checks) — all
pass. go test and node --test both pass for every changed package;
golangci-lint (installed ad hoc via `go run`) reports zero findings in
any changed file. Two pre-existing, unrelated test failures were
observed and left alone (TestPostScriptEnv_NoSchemaAppendedWhen* and
TestStartFetchService_*) — verified to fail identically on the
pre-fix commit and traced to this sandbox's own environment (a leaked
FULLSEND_OUTPUT_SCHEMA var and a network policy returning 403 for a
local health probe), not to any code this PR touches.

Addresses #7388
@fullsend-ai-coder

Copy link
Copy Markdown
Contributor Author

🔧 Fix agent — iteration 4 (bot-triggered)

Addressed all three findings from the re-review: hardened sticky.FindMarkedComment to match by body prefix instead of unanchored substring (fixing the marker-collision risk for postMissingRiskAssessment/clearMissingRiskAssessmentDiagnostic and any other sticky caller), made parseReviewResult reject a risk_assessment object lacking a usable score/level instead of treating a zero-value struct as present, and routed persona-skip/reject marker-line fields (name and reason in Go, name/reason/model/stopReason in JS) through the same substitution rules as runtime.sanitizeOutput so GHA workflow-command syntax and encoded newlines can no longer reach the workflow log unbroken. Added targeted regression tests for each fix in both Go and JS. No findings were disagreed with — all three were verified against the code and fixed at the root cause each remediation pointed to.

Fixed (3):

  1. [injection-pattern] postreview.go:368 — sticky.FindMarkedComment's unanchored Contains match could select and overwrite a bot comment that merely quotes the diagnostic marker mid-body. (internal/sticky/sticky.go): Changed FindMarkedComment to match a marker as a body prefix (strings.HasPrefix) instead of an unanchored substring, mirroring how Post always writes marker+"\n"+body and how BuildUpdatedBody strips markers by prefix. Added a regression test in internal/sticky/sticky_test.go (marker embedded mid-body must not match) and an end-to-end regression test in internal/cli/postreview_test.go simulating the exact collision scenario from the finding: a bot review comment quoting the literal missingRiskAssessmentMarker text is left untouched and the diagnostic is posted as a separate new comment.
  2. [logic-error] postreview.go:349 — parseReviewResult treats any successfully-decoded JSON object as a present risk_assessment, including {} or an object missing score/level, which decode into a zero-value struct rather than leaving the pointer nil. (internal/cli/postreview.go): Added hasUsableRiskAssessment, which requires score in 1-5 and a level in the ADR 0089 enum (reusing sanitizeRiskLevel's allowlist), and used it in parseReviewResult to leave RiskAssessment nil when the decoded value isn't usable, logging a warning the same way the malformed/wrong-typed path already does. Added parse tests for {}, an object missing score/level, an out-of-range score, and an unknown level, plus an end-to-end postMissingRiskAssessment test for the {} case.
  3. [injection-pattern] pi_personas.go:185 — fullsend:persona:skip lines only collapse \r\n\t, letting a skipped persona's name/reason carry "::" GHA workflow-command syntax or %0A newlines into the log unbroken. (internal/runtime/pi_personas.go): formatPersonaLogName now routes the name through the existing runtime.sanitizeOutput (same package) instead of a bespoke control-char collapse, and both fullsend:persona:skip Fprintf call sites also route the reason through sanitizeOutput before %q formatting. Mirrored the identical substitution rules (ANSI/OSC strip, "::" breaking to a fixed point, %0A/%0a/%0D/%0d replacement, control-char-to-space) in a new sanitizePersonaLogField helper in fullsend-agent.js's logPersona. Removed the now-unused personaLogControlCharsRe var and its regexp import. Extended the Go test suite (pi_personas_test.go) with cases for "::error::"-style and %0A-encoded payloads in both the discovery-skip name and the reason field, and extended the JS test suite (fullsend-agent.test.mjs) with equivalent cases via the unknown-persona and skipped-persona reject paths.

Tests: passed

Decision points
  • Fixed FindMarkedComment's Contains-vs-prefix matching globally in sticky.go rather than adding a second, diagnostic-only lookup function (alternatives: Add a separate prefix-matching helper used only by postMissingRiskAssessment/clearMissingRiskAssessmentDiagnostic, leaving FindMarkedComment's Contains behavior for other callers (e.g. the main review sticky), Change FindMarkedComment itself to match by prefix; rationale: Post always writes every marker as a literal body prefix and BuildUpdatedBody preserves that prefix across updates, so no legitimate marker lookup (for any marker, including reviewMarker) ever depends on Contains semantics; switching the shared function closes the same class of collision for every sticky comment type with a one-line change, and all existing sticky tests already assert marker-as-prefix bodies so nothing broke.)
  • Reused sanitizeRiskLevel's existing allowlist inside the new hasUsableRiskAssessment check instead of introducing a second level allowlist (alternatives: Duplicate the low/moderate/elevated/high/critical list in a new map, Call sanitizeRiskLevel and treat its "unknown" sentinel as invalid; rationale: Avoids two allowlists drifting apart; sanitizeRiskLevel already exists a few lines above for the same enum.)

Next steps:

  • /fs-review — request a re-review of the changes
  • /fs-fix <your instruction> — run another fix pass with specific guidance
  • Push commits directly — review re-runs automatically on push
    Updated by fullsend fix agent

@fullsend-ai-coder fullsend-ai-coder Bot added the needs-human Agent loop needs human intervention label Sep 17, 2026
@fullsend-ai-review

fullsend-ai-review Bot commented Sep 17, 2026 •

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 12:59 AM UTC · Completed 1:15 AM UTC

Commit: 18acace · View workflow run →

Runtime: pi · Model: sonnet → claude-sonnet-5 · Effort: high · Cost: $6.54

@fullsend-ai-review fullsend-ai-review Bot added risk/elevated PR risk: elevated and removed risk/moderate PR risk: moderate labels Sep 17, 2026
fullsend-ai-review[bot]

This comment was marked as outdated.

@fullsend-ai-coder

fullsend-ai-coder Bot commented Sep 17, 2026 •

Copy link
Copy Markdown
Contributor Author

🤖 Finished Fix · ✅ Success · Started 1:17 AM UTC · Completed 1:26 AM UTC

Commit: 18acace · View workflow run →

Runtime: claude · Model: sonnet → claude-sonnet-5 · Effort: high · Cost: $0.94

Allowlist ra.Level through sanitizeRiskLevel before interpolating it
into the unusable-risk_assessment WARNING line in parseReviewResult,
matching the allowlisting this PR already applies at other print
sites. Adds a regression test asserting a level of "::error::pwned"
appears in stderr only as "unknown".

Note: pre-commit could not run (network-blocked hook-repo fetch);
ran the local hooks directly instead (gofmt, go vet, lint-broken-symlinks).

Addresses #7388
@fullsend-ai-coder

Copy link
Copy Markdown
Contributor Author

🔧 Fix agent — iteration 5 (bot-triggered)

This re-review round surfaced one new low-severity finding (the rest of the review body is prior-round history already marked resolved). Fixed by routing ra.Level through the existing sanitizeRiskLevel allowlist before it reaches the unusable-risk_assessment WARNING line, and added a test covering a malicious level value.

Fixed (1):

  1. injection-pattern: parseReviewResult's unusable-risk_assessment WARNING at internal/cli/postreview.go:1001 writes raw ra.Level to GHA-visible stderr with only %q, bypassing sanitizeRiskLevel (internal/cli/postreview.go): Interpolated sanitizeRiskLevel(ra.Level) instead of the raw ra.Level into the WARNING line, matching the allowlisting already applied at other print sites (e.g. printer.StepInfo). Added a regression test in internal/cli/postreview_test.go asserting that a level of "::error::pwned" appears in stderr only as "unknown" and never verbatim.

Tests: passed

Next steps:

  • /fs-review — request a re-review of the changes
  • /fs-fix <your instruction> — run another fix pass with specific guidance
  • Push commits directly — review re-runs automatically on push
    Updated by fullsend fix agent

@fullsend-ai-review

fullsend-ai-review Bot commented Sep 17, 2026 •

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 1:28 AM UTC · Completed 1:46 AM UTC

Commit: 9e44f38 · View workflow run →

Runtime: pi · Model: sonnet → claude-sonnet-5 · Effort: high · Cost: $6.67

@fullsend-ai-review
fullsend-ai-review Bot dismissed stale reviews from themself September 17, 2026 01:46

Superseded by updated review

@github-actions

Copy link
Copy Markdown

This pull request has been automatically marked as stale because it has not had any activity in the last month. It will be closed in 2 weeks if no further activity occurs. Remove the stale label to reset the inactivity timer.

@github-actions github-actions Bot added the stale label Oct 18, 2026

This branch had an error being deployed

1 failed and 1 active deployments
site-preview — 9e44f387 Deployed Sep 17, 2026 by github-actions[bot]
dev — 9e44f387 Deployed Sep 17, 2026 by fullsend-ai-coder[bot] via behaviour #13043
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs-human Agent loop needs human intervention ready-for-merge All reviewers approved — ready to merge ready-for-review Triggers review agent dispatch risk/elevated PR risk: elevated stale

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Risk-assessment review sub-agent silently omits its comment on ~15% of PRs (e.g. #7385)

0 participants