Repository navigation
Description
What happens
The review agent currently approves PRs without checking whether CI status checks are passing. This can result in approving changes that fail tests, linting, or other automated verification.
Example: konflux-ci/oras-container#295 (review) — the review agent approved this PR while CI was failing.
What should happen
Before approving a PR, the review agent should check the status of all CI checks and factor that into its review decision.
Potential approaches:
- Block approval on any failing check: Refuse to approve if any status check is failing
- Distinguish required vs. optional: Only block on required checks (per branch protection rules)
- Approve with caveat: Approve the code quality but add a comment noting failing checks
- Comment only: Don't approve, but post review comments noting both code assessment and CI status
Pending/running checks: If checks haven't completed yet (pending or in-progress), the review agent should acknowledge this state and either:
- Wait for checks to complete before reviewing
- Review code quality but note that CI verification is incomplete
- Skip review entirely until CI provides signals
Fork PR edge case: When CI requires manual approval to run (e.g., on fork PRs), the agent should note that CI status is unavailable rather than treating it as passing.
Context
This is an enhancement for the review agent, not a blocker for initial deployment. However, it's important for preventing the agent from giving false signals about PR readiness.
Related work:
- Problem: how should fullsend model "definition of done"? #334 discusses modeling "definition of done" more broadly
- This issue is narrower and standalone: just checking CI status as part of review
Metadata
Metadata
Assignees
Labels
Type
Projects
- StatusShow more project fieldsDone
Activity
deboer-tim commentedon Apr 28, 2026
This happens fairly often with human reviewers in Podman Desktop. If the tests are still running then often a reviewer will just approve and rely on required tests passing. If a minor check has failed then humans will often approve with a comment - sometimes even if the check is not required, and trusting that the person will fix. If the failure was major, some will do a review already; others won't even start.
My point is mostly about conventions - teams may have different expectations. Fullsend reviewer doesn't use up human time but may want to wait until PR is ready to save $. So may need to be configurable or pick up conventions from the repo.
fullsend-ai-triage commentedon May 5, 2026
RICE Priority Score: 0.8
Score breakdown
Formula: (1.5 × 1 × 0.8) / 1.5 = 0.8
maruiz93 commentedon Jun 25, 2026
An example of this: konflux-ci/tekton-integration-catalog#185
The review agent says LGTM but the CI is failing.
ralphbean commentedon Sep 8, 2026
/fs-triage lots of things have changed between when this was originally filed and now across both the https://github.com/fullsend-ai/fullsend and https://github.com/fullsend-ai/agents repos. Is this still relevant?
fullsend-ai-triage commentedon Sep 8, 2026
🤖 Finished Triage · ✅ Success · Started 9:34 PM UTC · Completed 9:40 PM UTC
Commit:
e4749da· View workflow run →Runtime: claude · Model: opus → claude-opus-4-6 · Effort: high · Cost: $0.98
fullsend-ai-triage commentedon Sep 8, 2026
This feature request is still relevant. Investigation of the current review agent implementation in both
fullsend-ai/fullsendandfullsend-ai/agentsconfirms that no CI status checking has been added — the review agent's sub-agent roster (correctness, security, intent-coherence, style-conventions, docs-currency, cross-repo-contracts, risk-assessment, challenger) has no CI dimension, and neither the pre-review nor post-review scripts query check status.Related work
Several narrower issues track specific aspects of this problem:
All remain open with no PRs addressing them.
Implementation notes
The
post-review.shscript already has a pattern for downgradingapprove→comment(used for protected-path enforcement). A similar gate for CI status could be added there as a defense-in-depth layer. For richer integration, a CI-status sub-agent or pre-check step in thepr-revieworchestrator could feed CI signals into the review synthesis.Per community feedback, the behavior should likely be configurable per repo to accommodate different team conventions around CI expectations.
Labels: Removing type/idea since the issue has been validated by multiple users with concrete examples and is a confirmed feature gap, not an exploratory idea.
Next steps:
/fs-code— agent creates a PR to implement this issue/fs-code <your instruction>— agent implements with your specific guidancedeboer-tim commentedon Sep 9, 2026
FYI we had some discussion on the contributor call today, and @mrizzi and I chatted offline.
Review agent can be extended to look at additional aspects (my naive assumption is that would be generally more efficient than multiple review agents), but if we're building a mostly-automated platform the fundamental behaviour difference is between 'smart coder' or 'smart reviewer' - or more simply 'who is responsible for the next action'?
On most of the teams I work with, review/CI are asynchronous and the coder is fully responsible for getting the PR in an approved state - deciding how to respond to all feedback including failed PR checks. Hence issues like #642, #5350, fullsend-ai/agents#399, and fullsend-ai/agents#565 - i.e. code/fix automatically reacting by itself.
On Marco's team the review(bot) does more: essentially waiting to look at all sources of feedback, summarizing, and then triggering the coder with more specific instructions. This isn't just configuration of one agent, but a difference in responsibilities between the three current agents.
I think this merits discussion as we should either decide to only support one path (which would cancel a whole class of issues) or this is a new multi-agent 'who is responsible for reacting when X' config requirement.
ifireball commentedon Sep 17, 2026
See fullsend-ai/agents#1333 - we are making the fix agent also fix CI. Also further discussion on agent should go in the agents repo. Closing this here for now.