Skip to content

docs(adr): assess vulnerabilities with a read-only agent before the code agent - #9309

Draft
ascerra wants to merge 1 commit into
mainfrom
docs/adr-0169-cve-assessment-agent
Draft

ascerra wants to merge 1 commit into
mainfrom
docs/adr-0169-cve-assessment-agent

Conversation

@ascerra

@ascerra ascerra commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Records the decision to add a first-party cve agent that assesses one public vulnerability against one branch without changing anything, and to hand any fix to the existing code agent, which works only from a current verdict. Draft, for discussion alongside #9266, #9264 and #9265.

Related Issue

No tracking issue yet. Related: #9266 (dispatch to several targets), #9264 (recovery), #9265 (Pi tool selection), #2513 and #8944 (private tracker content and public repositories).

Changes

  • Add ADR 0169.
  • docs/architecture.md: one "Decided, not yet implemented" line under Agent Registry.
  • docs/problems/agent-architecture.md: a short role entry that links the ADR and names what stays open.

No implementation is in this PR.

What the ADR does not decide

Evidence for claims about current behavior

fullsend at 8ef5d3d954653a7316e2a1af953ca4eac7cdf175, agents at 3b40ac718d29c3190b71e68447b9f4e938f19974, the experiment repository at 42b1a2eec8ef3b4513f56a576b09e82154483d01.

Claim in the ADR Evidence
No existing agent produces a vulnerability verdict; triage labels and routes The agents repository ships seven harnesses: code, fix, prioritize, retro, review, scribe, triage. Triage's result is an action from a fixed list with no verdict or branch (schema L10–L13)
Triage's time budget is sized for labelling timeout_minutes: 10
The experiment composes triage, registers it as triage, and keeps a controller in the repository config.yaml, harness/triage-jira.yaml, scripts/cve_workflow.py
A harness can mount the repository read-only readonly_repo
A pre-script can stop a run before a sandbox exists FULLSEND_PRESCRIPT_OUTPUT and skipped=true
The code agent starts from ready-to-code or /fs-code harness/code.yaml trigger
The code agent may target a branch other than the default, within an allowlist ADR 0053; CODE_ALLOWED_TARGET_BRANCHES in the post-script
A run does not receive a branch from dispatch, so the agent resolves it The dispatch record has no ref (ExecutionRef); a Jira-sourced GitLab run is "an agent pipeline on the pinned default branch"
Jira support is Cloud only Jira guide, prerequisites

Prototype

A prototype is on the cve-assessment-agent branch of fullsend-ai/agents. It is not part of this PR and is not proposed for merge as it stands.

What was run with it (notes):

  • Hosted, on a private GitLab project with Jira trackers and fullsend v0.55.0, on the claude and pi runtimes: an affected verdict on a release branch, a fixed verdict on the default branch, and the handoff to the code agent, which opened a change against the release branch within the planned files.
  • Offline: 103 script tests, 32 tests for the code agent's checks, 11 for the evidence tool.

What it did not show, and what went wrong, is listed in the same notes. The main gaps are that the fix agent is not held to the verdict's plan, that nothing maps a tracker to its repository and branch, and that GitHub was not run hosted.

Testing

  • make lint passes (stage changes first, then run)
  • Tests added/updated for new or modified logic

Ran the repository's pre-commit hooks over this change (pre-commit run --from-ref origin/main --to-ref HEAD), which is what make lint runs: all passed, including the three ADR hooks, the docs and Markdown link checks and secret detection. gitlint passes on the commit. No code changes, so no tests.

Checklist

  • PR title follows Conventional Commits (correct type, ! for breaking changes)
  • Commits are signed off (DCO) — human and human-directed agent sessions only
  • I wrote this contribution myself and can explain all changes in it

🤖 Generated with Claude Code

…ode agent

Record the decision to add a first-party cve agent that assesses one public
vulnerability against one branch without changing anything, and to hand any
fix to the existing code agent, which works only from a current verdict.

Adds ADR 0169 and the matching lines in architecture.md and the
agent-architecture problem document. No implementation is included. A
prototype is on the cve-assessment-agent branch of fullsend-ai/agents.

Signed-off-by: Adam Scerra <ascerra@redhat.com>
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@fullsend-ai-review

fullsend-ai-review Bot commented Nov 11, 2026 •

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 3:44 AM UTC · Completed 3:54 AM UTC

Commit: 81c7090 · View workflow run →

Runtime: pi · Model: sonnet → claude-sonnet-5-5 · Effort: high · Cost: $2.45

@github-actions

Copy link
Copy Markdown

Site preview

Preview: https://9b668fdd-site.fullsend-ai.workers.dev

Commit: 81c7090e7a4d1739c55957758882433f7855064c

@fullsend-ai-review

Copy link
Copy Markdown

Risk Assessment: moderate (2/5)

Details

Documentation-only PR adding ADR 0169 with low metadata risk, elevated to moderate by high commit churn, author contention, and co-change coupling on docs/architecture.md.

@fullsend-ai-review

Copy link
Copy Markdown

Review

Findings

Low

  • [auth-bypass] docs/ADRs/0169-assess-vulnerabilities-with-a-read-only-agent-before-the-code-agent.md:135 — Decision item 4 (Handoff) has a host check reject code-agent changes outside the files in the verdict plan, but Consequences says the check "covers the code agent only; the fix agent ... needs the same check." The ADR names the gap without deciding to close it. It is already disclosed and this PR is docs-only, so nothing is bypassable today; advisory only.
    Remediation: Move the fix-agent requirement into Decision item 4 (same current-verdict and file-scope checks enforced on fix-agent changes; a change needing more files requires a new assessment), or state that the workflow must not be enabled until a later decision covers it.

Labels: ADR-only documentation PR; ADR PRs in this repo carry type/adr and component/docs.

vulnerability reaches a verdict before either acts on it.
- A fix takes two runs with a person between them by default, which is slower
than one run and keeps a person on the verdict.
- The check that holds a change to the verdict's plan covers the code agent

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[low] auth-bypass

Decision item 4 (Handoff) has a host check reject code-agent changes outside the files in the verdict plan. Consequences says the check "covers the code agent only; the fix agent, which acts on review findings, needs the same check." The ADR names this gap but does not decide to close it, so the Decision never says whether the fix agent will be bound to the current verdict and its file list. The ADR already discloses the gap and this PR is docs-only, so nothing is bypassable today. This is advisory; wording the requirement is a design choice for the author.

Suggested fix: Move the fix-agent requirement into Decision item 4 (host enforces the same current-verdict and file-scope checks on fix-agent changes, and a change needing more files requires a new assessment), or state that the workflow must not be enabled until a later decision covers it.

This branch was successfully deployed

1 active and 1 inactive deployments
site-preview — 81c7090e Deployed Nov 11, 2026 by github-actions[bot]
dev — 81c7090e Deployed Nov 11, 2026 by ascerra via playback #19642
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

component/docs User-facing documentation requires-manual-review Review requires human judgment risk/moderate PR risk: moderate type/adr

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant