Repository navigation
Description
What happened
On PR #1840, the code agent modified files under internal/scaffold/fullsend-repo/skills/pr-review/ — these are generic skill definitions deployed to all repos via scaffolding. The changes hardcoded docs/plans/ path references in both SKILL.md and correctness.md. The review bot ran twice and posted findings about internal consistency, scope creep, and pattern ordering, but never flagged the portability problem. The human reviewer (ralphbean) caught it in 3 inline comments requesting removal of all path-specific references, leading to a CHANGES_REQUESTED verdict and a fix cycle.
What could go better
The review bot should have a portability/generality check for changes to scaffolded skill definitions (internal/scaffold/). These files are templates deployed across many repos — hardcoded paths like docs/plans/ will break or be misleading in repos with different directory structures. The bot's first run came close (it noticed the docs/plans/ path and raised a design question) but framed it as a detection strategy ambiguity rather than a portability violation. This is a high-confidence finding: the human explicitly called this out as the primary issue, and it required a full fix cycle to resolve.
Proposed change
Add a portability check to the review agent's analysis when the diff touches files under internal/scaffold/. This could be implemented as:
-
In the pr-review skill's
SKILL.md(internal/scaffold/fullsend-repo/skills/pr-review/SKILL.md): Add a dispatch rule or review dimension for changes to scaffold templates that checks for repo-specific assumptions (hardcoded paths, org-specific references, repo-structure dependencies). -
Alternatively, in the correctness sub-agent (
internal/scaffold/fullsend-repo/skills/pr-review/sub-agents/correctness.md): Add guidance that when reviewing changes to files underinternal/scaffold/, the agent should verify that all references are generic and portable across different repo structures.
The check should flag any literal directory paths (e.g., docs/plans/, src/components/) in scaffold templates as potential portability issues, unless they reference well-known conventional paths (e.g., .github/, CLAUDE.md).
Validation criteria
The next 3 PRs that modify files under internal/scaffold/ and introduce repo-specific path references should have the review agent flag the portability concern without requiring human intervention. Measure by checking whether human reviewers still need to manually catch path-specificity issues in scaffold changes.
Generated by retro agent from #1840
Metadata
Metadata
Assignees
Labels
Type
Projects
- StatusShow more project fieldsTodo
Activity
fullsend-ai-triage commentedon Jun 4, 2026
This is a stale-issue relevance check rather than an implementation trigger, per the request above. Here's what changed and what's still true:
Still relevant. The underlying gap -- no review-time check for hardcoded, repo-specific paths introduced into generic skill/sub-agent definitions -- remains unaddressed. Current SKILL.md dispatch classification (step 3b/3c) and the correctness, cross-repo-contracts, style-conventions, docs-currency, and intent-coherence sub-agent definitions were all inspected directly; none contain a portability check for templated skill content.
What moved. The file paths named in the original proposal no longer exist.
internal/scaffold/fullsend-repo/in this repo no longer has askills/subtree at all (onlyproviders/,templates/,scripts/,profiles/,.github/). Thepr-reviewskill and its sub-agents (includingcorrectness.md) now live in thefullsend-ai/agentsrepo underskills/pr-review/. Any implementation needs to target that repo/path instead.Related work. #1909 tracks the code-agent side of the same PR #1840 incident and is also still open/unresolved -- the two remain complementary as previously noted.
No conflicting work in flight. No open PR in either fullsend-ai/fullsend or fullsend-ai/agents references this issue or implements a portability check.
No ready-to-code trigger is being recommended -- this is left for human prioritization as requested.
Labels: Aligns labeling with the repo's convention for other 'review agent should detect X' feature issues (e.g. #1622, #1660, #3477, #1562), which are consistently tagged feature and component/skills.
Next steps:
/fs-code— agent creates a PR to implement this issue/fs-code <your instruction>— agent implements with your specific guidancePrevious run
Triage Summary
Category: Feature — add a portability check to the review agent for scaffold template changes.
Problem: When PRs modify files under
internal/scaffold/, the review agent does not check whether the changes introduce repo-specific references (hardcoded paths, org-specific assumptions). On PR #1840, the correctness sub-agent noticed adocs/plans/reference but framed it as a design ambiguity rather than a portability violation, requiring a human reviewer to catch the real issue.Recommended approach: Add scaffold-portability guidance to either the pr-review
SKILL.mddispatch rules or thecorrectness.mdsub-agent definition. When the diff touchesinternal/scaffold/, the review should flag literal directory paths that aren't well-known conventional paths (.github/,CLAUDE.md, etc.).Related but distinct: #1909 addresses the same root incident but targets the code agent (preventing it from introducing repo-specific paths in scaffold files). This issue targets the review agent (catching such paths during review). Both are complementary.
Proposed test case:
Labels: Retro-generated feature request targeting the review agent's handling of scaffold templates, medium priority since it requires human fallback but is not blocking.
fullsend-ai-prioritize commentedon Jun 8, 2026
RICE Priority Score: 2.4
Score breakdown
Formula: (1.5 x 1 x 0.8) / 0.5 = 2.4
ralphbean commentedon Sep 22, 2026
/fs-triage lots of things have changed between when this was originally filed and now across both the https://github.com/fullsend-ai/fullsend and https://github.com/fullsend-ai/agents repos. Is this still relevant? This is a stale-issue review, not an implementation request: even if the issue is still a valid bug and has sufficient information, do not recommend or trigger ready-to-code yet. Leave it for human prioritization after confirming whether it remains relevant. Do not close a still-relevant bug merely to avoid coding it.
fullsend-ai-triage commentedon Sep 22, 2026
🤖 Finished Triage · ✅ Success · Started 12:05 AM UTC · Completed 12:10 AM UTC
Commit:
b9e5099· View workflow run →Runtime: claude · Model: sonnet → claude-sonnet-5 · Effort: high · Cost: $0.46