Repository navigation
feat(verify-pr): re-sync FULLSEND_OUTPUT_DIR sandbox dual-mode onto SKILL.md - #286
Conversation
…KILL.md
Re-apply the sandbox dual-mode adaptation onto the current verify-pr
SKILL.md, gated on the FULLSEND_OUTPUT_DIR env var so interactive
(marketplace) behavior is unchanged when it is unset.
When set (fullsend sandbox): skip Jira/GitHub API calls, read pre-fetched
task and PR data from the mounted verify-pr-input.json, accumulate every
write as an action in { "report": {}, "actions": [] }, and write the
result to $FULLSEND_OUTPUT_DIR/agent-result.json. Adds Step 0.6 (Sandbox
Mode Detection), Step 0.7 (Load Pre-Fetched Data), inline sandbox-mode
notes on each read/write step, and a Step 9 Sandbox Mode Output block.
Every emitted action type matches the verify-pr-result.schema.json enum
(create_subtask, create_link, post_pr_reply, post_pr_comment,
create_root_cause_task, post_comment, post_report); link_type values use
the schema's Blocks/Related enum.
Implements TC-5812
Assisted-by: Claude Code
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Reviewer's GuideThis PR re-syncs verify-pr with the FULLSEND_OUTPUT_DIR sandbox contract: trusted prefetching supplies all task, PR, and idempotency reads; sandbox execution validates inputs, avoids credentials and network access, emits schema-aligned deferred actions, and writes a structured report, while interactive execution remains unchanged. Sequence diagram for tokenless verify-pr sandbox flowsequenceDiagram
participant Runner as Trusted pre-script runner
participant Sandbox as verify-pr sandbox
participant Output as agent-result.json
participant Post as post-script executor
participant Jira as Jira
participant GitHub as GitHub
Runner->>Runner: pre-verify-pr.sh fetches task, PR, and related issues
Runner->>Sandbox: provide verify-pr-input.json
Sandbox->>Sandbox: validate input schema
alt input valid
Sandbox->>Sandbox: inspect prefetched task, GitHub, and idempotency data
Sandbox->>Sandbox: accumulate create_subtask, create_link, and comment actions
Sandbox->>Sandbox: build report and append post_report
Sandbox->>Output: write report and actions
Post->>Output: read result
Post->>GitHub: execute post_report using report_md
Post->>Jira: execute post_report using report_adf
Post->>Jira: execute deferred Jira actions
Post->>GitHub: execute deferred PR actions
else input missing or invalid
Sandbox->>Output: write structured error result
Output-->>Post: validation failure
end
Flow diagram for verify-pr dual-mode selectionflowchart TD
Start[Start verify-pr] --> Mode{FULLSEND_OUTPUT_DIR set?}
Mode -->|No| Interactive[Interactive mode]
Interactive --> Direct[Read Jira and GitHub; execute writes directly]
Mode -->|Yes| Sandbox[Sandbox mode]
Sandbox --> Validate[Validate verify-pr-input.json]
Validate -->|Invalid| Fail[Write error result and stop]
Validate -->|Valid| Prefetched[Use prefetched task, PR, and idempotency data]
Prefetched --> Actions[Accumulate schema-aligned actions]
Actions --> Report[Write report and actions to agent-result.json]
File-Level Changes
Possibly linked issues
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
There was a problem hiding this comment.
Hey - I've found 4 issues
Prompt for AI Agents
Please address the comments from this code review:
## Individual Comments
### Comment 1
<location path="plugins/sdlc-workflow/skills/verify-pr/SKILL.md" line_range="111" />
<code_context>
+(where the tokens live) and mounts it read-only into the sandbox. Read it:
+
+```bash
+cat /sandbox/workspace/.pre-script/verify-pr-input.json | python3 -m json.tool > /dev/null 2>&1 && echo "Pre-fetched data available" || echo "ERROR: Pre-fetched data missing or invalid"
+```
+
</code_context>
<issue_to_address>
**issue (bug_risk):** The pre-fetched input check validates only JSON syntax, not `verify-pr-input.schema.json`; a syntactically valid object with missing or malformed `task` or `github` fields is accepted and later field accesses fail or use incomplete data instead of taking the documented invalid-input path.
**Triggers:** When the mounted file contains valid JSON that violates the input schema.
**Suggested fix:** Validate the file against `verify-pr-input.schema.json` before treating it as available.
</issue_to_address>
### Comment 2
<location path="plugins/sdlc-workflow/skills/verify-pr/SKILL.md" line_range="128" />
<code_context>
+ Step 9's HEAD-SHA retrieval — using the bundle fields instead.
+
+If the file is missing or invalid, log a warning and fall back to Steps 1–3 and the
+direct GitHub reads (interactive behavior).
+
## Inputs
</code_context>
<issue_to_address>
**issue (broader_impact):** The documented fallback from a missing or invalid prefetch file returns to direct Jira and GitHub API behavior even though `FULLSEND_OUTPUT_DIR` explicitly denotes a sandbox with no tokens, no `gh` CLI, and no network egress; the fallback therefore fails in the very environment this mode is intended to support.
**Triggers:** When the pre-fetched input file is missing or invalid in sandbox mode.
**Suggested fix:** Fail clearly with a structured result, or have the trusted pre-script repair/report the prefetch failure; do not fall back to credentialed APIs inside the sandbox.
</issue_to_address>
### Comment 3
<location path="plugins/sdlc-workflow/skills/verify-pr/SKILL.md" line_range="590-591" />
<code_context>
jira.create_issue_link(type="Blocks", inwardIssue=<sub-task-id>, outwardIssue=<parent-task-id>)
+**Sandbox mode:** Instead of calling `jira.create_issue` and `jira.create_issue_link`,
+append a `create_subtask` action and a `create_link` action. Use this pattern for **all**
+sub-task categories below (review feedback, CI failure, eval failure):
+
</code_context>
<issue_to_address>
**issue (bug_risk):** Sandbox mode replaces sub-task and root-cause writes with actions but provides no sandbox path for the Jira reads required by the idempotency checks: the prefetch contains only shallow `issue_links`, not linked sub-task labels/descriptions or root-cause comments, and Step 7c still requires fetching linked-task comments. The agent therefore cannot perform the existing deduplication checks without attempting unavailable Jira reads and will either fail or emit duplicate tasks.
**Triggers:** When prior review-feedback or root-cause tasks already exist and the run reaches the corresponding idempotency check.
**Suggested fix:** Prefetch the linked-task metadata and root-cause comments, or add explicit sandbox read actions/data and use them for these checks.
</issue_to_address>
### Comment 4
<location path="plugins/sdlc-workflow/skills/verify-pr/SKILL.md" line_range="1215-1218" />
<code_context>
+### Sandbox Mode Output
+
+**Sandbox mode only:** Instead of posting to GitHub and Jira directly, populate the
+`report` object and append a single `post_report` action. The runner's `post_script`
+posts the GitHub PR comment (from `report_md`, applying the commit-scoped
+find-then-update-or-create path above) and the Jira comment (from `report_adf`) after
+the sandbox exits.
+
+Populate `report` (all fields required by
</code_context>
<issue_to_address>
**issue (bug_risk):** The sandbox contract says the runner posts the Jira comment from `report_adf`, but the actual `execute_post_report` implementation posts `report_md` to Jira and ignores `report_adf`; the deferred report therefore loses the required ADF representation and can lose the mandated Jira footnote/formatting.
**Triggers:** On every successful sandbox run that emits `post_report`.
**Suggested fix:** Make the runner consume `report_adf` for the Jira comment, or change the skill contract to emit and validate the representation the runner actually uses.
</issue_to_address>Sourcery assessment
Needs a human reviewer. 4 findings to address first, and if the sandbox instructions or action mappings are wrong, the agent could create incorrect Jira subtasks/links or post incorrect GitHub and Jira comments that persist after this file is reverted. Those records and messages require manual cleanup; reverting only prevents future executions from using the behavior.
Blocking findings: plugins/sdlc-workflow/skills/verify-pr/SKILL.md:111, plugins/sdlc-workflow/skills/verify-pr/SKILL.md:128, plugins/sdlc-workflow/skills/verify-pr/SKILL.md:591, plugins/sdlc-workflow/skills/verify-pr/SKILL.md:1218
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
verify-pr report — TC-5812Overall:
Review Feedback (WARN)Four inline review comments from
All four sub-tasks block TC-5812. Root-Cause Investigation (DONE)The four defects share one systemic root cause in the plan-feature phase: TC-5787's deliverable is a single contract split across four sibling artifacts (schema, producer, consumer, orchestration doc), but decomposition scoped each task's acceptance criteria to one artifact in isolation, so no task owned cross-artifact consistency. TC-5984 captures the fix: plan-feature must detect contract-split Features and require cross-artifact-consistency criteria (reads-have-a-source, inputs-validated-against-schema, documented-behaviors-feasible-under-the-trust-model, doc-vs-runner field agreement). Other checksDiff is purely additive to
This comment was AI-generated by sdlc-workflow/verify-pr v0.13.9. |
Step 0.7 treated verify-pr-input.json as usable if it merely parsed as JSON. A syntactically valid but structurally incomplete prefetch (missing the github bundle or task fields) passed the check and then failed deep inside a later step. Replace the json.tool syntax check with jsonschema validation against verify-pr-input.schema.json so a structurally invalid-but-parseable prefetch is treated as invalid, mirroring the existing validate-output-schema.sh pattern. Implements TC-5980 Assisted-by: Claude Code Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…aled fallback Step 0.7 documented that a missing/invalid prefetch falls back to Steps 1-3 and direct Jira/GitHub reads. That "interactive" fallback cannot work in sandbox mode (FULLSEND_OUTPUT_DIR set): the sandbox has no tokens, no gh CLI, and no network egress, so credentialed calls fail obscurely or hang. Split the failure handling: in sandbox mode write a structured error to $FULLSEND_OUTPUT_DIR/agent-result.json and stop (the runner's output validation surfaces it as a hard failure), while interactive mode keeps the existing warn-and-fall-back behavior unchanged. Implements TC-5981 Assisted-by: Claude Code Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Steps 6d/6f/7c dedupe against the task's existing sub-tasks and linked root-cause issues via Jira reads, but the sandbox has no Jira token, so those idempotency checks had no data source and risked creating duplicate sub-tasks/root-cause tasks on re-run. Prefetch the read data on the trusted runner (split-trust): pre-verify-pr.sh enumerates the task's related keys and fetches each (summary, labels, description, issuetype, comments), and pre_verify_pr.py embeds them under a new `idempotency` bundle in verify-pr-input.json. SKILL.md Steps 0.7/6d/6f/7c now read `idempotency.related_issues` in sandbox mode instead of calling Jira; the input schema documents the new (optional) bundle. No token or egress is added to the sandbox. Implements TC-5982 Assisted-by: Claude Code Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
execute_post_report posted the GitHub-only report_md (with its embedded commit marker) to Jira and never read report_adf, contradicting SKILL.md Step 9's documented contract. Render the Jira comment body from report_adf via adf_to_markdown so the tracker-native report (and its Comment Footnote) is used; GitHub keeps report_md plus the commit-scoped marker. Implements TC-5983 Assisted-by: Claude Code Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
@sourcery-ai review |
There was a problem hiding this comment.
Hey - I've found 1 issue
Prompt for AI Agents
Please address the comments from this code review:
## Individual Comments
### Comment 1
<location path="plugins/sdlc-workflow/skills/verify-pr/SKILL.md" line_range="127-128" />
<code_context>
+ the already-checked-out PR-head tree. **Skip every GitHub read step** — Step 3
+ (checkout), Step 4a (review/comment fetches), Step 5a (diff/stat/commits), and
+ Step 9's HEAD-SHA retrieval — using the bundle fields instead.
+- Read the `idempotency.related_issues` array (each entry has `key`, `summary`,
+ `labels`, `description`, `issuetype`, `comments`) — the task's existing sub-tasks
+ and linked issues, prefetched on the runner. **Use it for every idempotency read**
+ (Steps 6d, 6f, 7c) instead of calling Jira; the sandbox has no token. The array is
+ empty when the task has no sub-tasks or linked issues yet.
+
+If the validation above fails (file missing, not valid JSON, or not conforming to the
</code_context>
<issue_to_address>
**issue (broader_impact):** Sandbox input validation accepts a structurally valid prefetch that omits the optional `idempotency` bundle, but the sandbox instructions then unconditionally rely on `idempotency.related_issues` for Steps 6d, 6f, and 7c. Those idempotency checks have no tokenless data source in that case and the run proceeds without deduplication, creating duplicate sub-tasks or root-cause tasks on reruns.
**Triggers:** When an older or manually supplied input bundle contains the required task/GitHub fields but omits `idempotency`.
**Suggested fix:** Require `idempotency.related_issues` in sandbox input validation, or treat a missing bundle as invalid and fail fast instead of continuing without idempotency data.
```suggestion
validate(instance=instance, schema=schema)
if (
not isinstance(instance, dict)
or not isinstance(instance.get("idempotency"), dict)
or not isinstance(instance["idempotency"].get("related_issues"), list)
):
raise ValidationError(
"required idempotency.related_issues array is missing"
)
print("Pre-fetched data available")
```
</issue_to_address>Sourcery assessment
Needs a human reviewer. 1 finding to address first, and if the sandbox-mode branching or prefetched idempotency data is wrong, the runner could create duplicate Jira tasks or links and post incorrect Jira/GitHub comments without live human intervention; those records and messages outlive a revert and require cleanup. Reverting restores the prior behavior but does not remove actions already executed.
Blocking findings: plugins/sdlc-workflow/skills/verify-pr/SKILL.md:128
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
Verification Report for TC-5812 (commit df76f1b)
Overall: WARNAll sourcery feedback is now fully enumerated and classified: the 5 inline threads and both review-body summaries. Every finding is tracked by an existing sub-task (TC-5980–5983, TC-6026) — no duplicates created. Domain sub-agent verdicts are unchanged from the prior verification at this same commit (df76f1b). The sole open merge blocker is TC-6026; the systemic root cause is tracked by TC-5984. Scope Containment, Commit Traceability, and Test Quality warnings are informational or justified. No FAILs. This comment was AI-generated by sdlc-workflow/verify-pr v0.13.9. |
|
[sdlc-workflow/verify-pr] Re: @sourcery-ai[bot] review (5090904620) — Classified as code change request (4 findings). This review body aggregates the four inline comments already tracked by sub-tasks: TC-5980 (SKILL.md:111 — validate prefetch against schema), TC-5981 (:128 — no credentialed fallback in sandbox), TC-5982 (:591 — sandbox path for idempotency Jira reads), and TC-5983 (:1218 — report_adf/report_md contract). Existing sub-tasks cover this feedback — no new sub-task created (deduplicated). |
|
[sdlc-workflow/verify-pr] Re: @sourcery-ai[bot] review (5093004840) — Classified as code change request — this review body aggregates the inline comment at SKILL.md:127-128 already tracked by sub-task TC-6026 (schema makes the |
The TC-5982 fix added an optional top-level `idempotency` object to verify-pr-input.schema.json, but the sandbox SKILL.md steps read `idempotency.related_issues` unconditionally for the Steps 6d/6f/7c dedup checks. A prefetch that passed Step 0.7 validation while omitting the bundle left those tokenless checks with no data source, silently skipping dedup and creating duplicate sub-tasks / root-cause tasks on reruns. Option 1 (require it): the producer always runs on the trusted runner with a Jira token, so it can always emit the bundle (empty related_issues when the task has no relations). Make the schema require `idempotency` and its `related_issues`, so Step 0.7 rejects a prefetch lacking it before any consuming step runs — consistent with the TC-5980/TC-5981 fail-fast contract. transform_to_input now always emits the bundle; SKILL.md states the invariant. Implements TC-6026 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Assisted-by: Claude Code
|
@sourcery-ai review |
There was a problem hiding this comment.
Hey - I've found 2 issues
Prompt for AI Agents
Please address the comments from this code review:
## Individual Comments
### Comment 1
<location path="plugins/sdlc-workflow/skills/verify-pr/SKILL.md" line_range="127-128" />
<code_context>
+- Read the `idempotency.related_issues` array (each entry has `key`, `summary`,
</code_context>
<issue_to_address>
**issue (bug_risk):** The input schema requires the `idempotency.related_issues` array itself but does not require `key`, `labels`, `description`, `issuetype`, or `comments` on each array item. A structurally incomplete entry therefore passes Step 0.7 and the sandbox dedup checks silently treat it as non-matching, allowing duplicate subtasks or root-cause tasks to be created.
**Triggers:** When a prefetch contains a related-issue object with missing metadata fields.
**Suggested fix:** Require the fields consumed by Steps 6d, 6f, and 7c in each `related_issues` item, or reject incomplete entries before execution.
```suggestion
validate(instance=instance, schema=schema)
required_related_fields = {"key", "labels", "description", "issuetype", "comments"}
for index, related_issue in enumerate(instance["idempotency"]["related_issues"]):
missing_fields = required_related_fields - related_issue.keys()
if missing_fields:
missing = ", ".join(sorted(missing_fields))
raise ValidationError(
f"related_issues[{index}] is missing required fields: {missing}"
)
print("Pre-fetched data available")
```
</issue_to_address>
### Comment 2
<location path="plugins/sdlc-workflow/skills/verify-pr/SKILL.md" line_range="66" />
<code_context>
+
+Check whether the `FULLSEND_OUTPUT_DIR` environment variable is set:
+
+```bash
+echo ${FULLSEND_OUTPUT_DIR:-not-set}
+```
+
</code_context>
<issue_to_address>
**issue (bug_risk):** The detection example uses `${FULLSEND_OUTPUT_DIR:-not-set}`, which treats an exported-but-empty `FULLSEND_OUTPUT_DIR` as unset. That causes the skill to select interactive mode and attempt credentialed Jira/GitHub operations even though the sandbox mode gate is present in the environment.
**Triggers:** When the runner exports `FULLSEND_OUTPUT_DIR` with an empty value instead of omitting it.
**Suggested fix:** Detect presence with `${FULLSEND_OUTPUT_DIR+x}` and separately reject an empty output directory, or require a non-empty value before entering sandbox mode.
```suggestion
if [ "${FULLSEND_OUTPUT_DIR+x}" = x ]; then
if [ -z "$FULLSEND_OUTPUT_DIR" ]; then
echo "FULLSEND_OUTPUT_DIR is set but empty"
exit 1
fi
echo "$FULLSEND_OUTPUT_DIR"
else
echo not-set
fi
```
</issue_to_address>Sourcery assessment
Needs a human reviewer. 2 findings to address first, and if the sandbox-mode workflow or prefetched idempotency data is wrong, it can create duplicate Jira sub-tasks and links or post incorrect Jira and GitHub comments after the runner executes the actions. Reverting stops future runs but does not automatically remove records, links, or externally posted messages already created.
Blocking findings: plugins/sdlc-workflow/skills/verify-pr/SKILL.md:128, plugins/sdlc-workflow/skills/verify-pr/SKILL.md:66
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
|
[sdlc-workflow/verify-pr] Re: @sourcery-ai[bot] review (5101905003) — Classified as code change request (2 findings).
Both Blocks TC-5812. The review body restates inline comments 3924536285 and 3924536305, which are tracked by the sub-tasks above. |
verify-pr report — TC-5812 (commit 5bb07cc)Overall:
Root-Cause Investigation: SKIPPED (idempotent). TC-6032 (schema does not require per-item New merge blockers (this run):
Classification replies: inline 3924866359 (→TC-6032), 3924866594 (→TC-6033); review-body 5526289494 (→both). verify-pr did not merge the PR and did not transition the Jira issue. |
The idempotency.related_issues array is required (TC-6026) but its items declared properties without a required list, so a prefetched related-issue entry missing key/summary/labels/description/issuetype/comments passed Step 0.7 schema validation. The tokenless-sandbox dedup checks (Steps 6d/6f/7c) then silently treated the incomplete entry as non-matching, letting duplicate sub-tasks and root-cause tasks be created on reruns — the item-level analogue of the array-level gap TC-6026 closed. Add a per-item required list of the six fields the dedup checks consume. build_idempotency_bundle already emits all six unconditionally, so the producer stays schema-valid. Add a test asserting an incomplete entry raises ValidationError. TC-6032 Assisted-by: Claude Code Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Step 0.6 detected sandbox mode with `echo ${FULLSEND_OUTPUT_DIR:-not-set}`.
The `:-` operator treats an exported-but-empty value the same as unset, so a
runner exporting the gate with an empty value would select the credentialed
interactive path despite the sandbox gate being present — the opposite of the
intended tokenless behavior.
Use presence detection (`${FULLSEND_OUTPUT_DIR+x}`) and reject an empty value
with a non-zero exit. Sandbox mode is selected whenever the variable is present
and non-empty; interactive mode only when it is genuinely unset.
TC-6033
Assisted-by: Claude Code
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
@sourcery-ai review |
Verification Report for TC-5812 (commit 7ca7e8c)
Overall: WARNAll acceptance criteria are satisfied and verification commands pass. Warnings are informational:
This comment was AI-generated by sdlc-workflow/verify-pr v0.13.9. Previous runVerification Report for TC-5812 (commit 7ca7e8c)
Overall: WARNAll acceptance criteria are satisfied and verification commands pass. Warnings are informational:
This comment was AI-generated by sdlc-workflow/verify-pr v0.13.9. Previous run (2)🔍 verify-pr report —
|
| Check | Verdict | Notes |
|---|---|---|
| Review Feedback | ✅ PASS | All 10 classifiable items (7 inline threads + 3 review bodies) already classified on prior runs; no new items on this HEAD. The two most recent code-change-requests — TC-6032 (per-item schema required) and TC-6033 (empty-FULLSEND_OUTPUT_DIR fail-fast) — are delivered in this commit. No new sub-tasks, no new replies. |
| Scope Containment | Task-declared file skills/verify-pr/SKILL.md present; 6 extra files (verify-pr-input.schema.json, pre-verify-pr.sh, pre_verify_pr.py, execute-actions.py, test_pre_verify_pr.py, test_execute_actions.py) all trace to review-feedback sub-tasks TC-5980..TC-6033. Justified extras, no missing task files. |
|
| Diff Size | ✅ PASS | +751 / −14 across 7 files — proportionate for a multi-sub-task review-feedback PR (doc + schema + scripts + matching tests). |
| Commit Traceability | ✅ PASS | All 8 commits reference a Jira ID (parent TC-5812 or child sub-task) and carry both Assisted-by + Co-Authored-By trailers. |
| Sensitive Patterns | ✅ PASS | No secrets/keys/credentials in any added line across 7 files. Near-matches (env var names, fabricated SHA/issue-key fixtures, template placeholders) correctly excluded. |
| CI Status | ✅ PASS | All 5 checks pass: Script Unit Tests (3.11–3.14) + Sourcery review. |
| Acceptance Criteria | ✅ PASS | Dual-mode gated on FULLSEND_OUTPUT_DIR via presence detection ${FULLSEND_OUTPUT_DIR+x} with exported-but-empty fail-fast; interactive path untouched when unset; all 7 emitted action types in the result-schema enum; both commit trailers present. |
| Verification Commands | ✅ PASS | Step 0.6 + output block present; orphan-check clean (only benign ADF-node false positives); pytest 109 passed. |
| Test Quality | Repetitive-test pairs in test_pre_verify_pr.py (idempotency-bundle field tests; schema-rejection tests) — parameterization candidates. 3 new tests lack docstrings (test_related_keys_empty_when_no_relations, test_build_idempotency_bundle_empty, test_transform_with_idempotency_attaches_key). |
|
| Test Change Classification | 🟢 ADDITIVE (informational) | Both modified test files only add/tighten coverage — +14 tests, negative schema-rejection cases, stronger report_adf assertions; no removals, relaxed matchers, skips, or mock-scope narrowing. |
| Root-Cause Investigation | N/A (informational) | No new sub-tasks created this run — nothing to investigate. |
Overall:
verify-pr did not modify code, merge, or transition the parent task.
This report was AI-generated by sdlc-workflow/verify-pr v0.13.9.
e696d00
into
RHEcosystemAppEng:TC-5787-verify-pr-fullsend-v037
Verification Report for TC-5812 (commit 7ca7e8c)
Overall: FAILCommit Traceability is FAIL: None of the 8 commits on the PR branch reference TC-5812 in their commit messages. Commits use conventional commit format ( Scope Containment and Diff Size are WARN: The task specifies only SKILL.md, but the PR also modifies 6 supporting files (schema, prefetch scripts, tests). These are plausibly related but exceed the task description scope. CI Status is WARN: Cannot be verified from the sandbox environment (no Test Quality is WARN: 3 of 14 new test functions in This comment was AI-generated by sdlc-workflow/verify-pr v0.13.9. |
Summary
Re-applies the sandbox dual-mode adaptation onto the current
verify-prSKILL.md, gated on theFULLSEND_OUTPUT_DIRenvironment variable. When the var is unset (marketplace/interactive users), behavior is unchanged. When set (fullsend sandbox), the skill runs without Jira/GitHub tokens: it reads pre-fetched task and PR data and accumulates every write as a structured action.Changes
Purely additive to
plugins/sdlc-workflow/skills/verify-pr/SKILL.md(228 insertions, no deletions):FULLSEND_OUTPUT_DIR; in sandbox mode, no direct Jira writes / GitHub reads / PR comment posts, and initializes the{ "report": {}, "actions": [] }accumulator./sandbox/workspace/.pre-script/verify-pr-input.json(the TC-5810 prefetch contract), skips Steps 1–3 and all GitHub reads, with a fallback to direct APIs if the file is missing/invalid.reportobject, appends apost_reportaction, and writes the complete result to$FULLSEND_OUTPUT_DIR/agent-result.json.Contract alignment
Every emitted action
typematches theverify-pr-result.schema.jsonenum:create_subtask,create_link,post_pr_reply,post_pr_comment,create_root_cause_task,post_comment,post_report.link_typevalues use the schema'sBlocks/Relatedenum.Verification
uvx skillsaw— 0 errors (Grade A); context-budget finding is warn-level (non-blocking; CI strict mode disabled).claude plugin validate plugins/sdlc-workflow— validation passed.scripts/changes → pytest not required.FULLSEND_OUTPUT_DIR).Implements TC-5812
🤖 Generated with Claude Code
Review feedback fixes (Sourcery-ai on PR #286)
Four follow-up sub-tasks address review comments on this PR:
verify-pr-input.jsonagainstverify-pr-input.schema.json(jsonschema), not just JSON syntax; malformed-but-parseable input is rejected.{"error": ...}result (no credentialed fallback is possible tokenless) so the runner's output validation surfaces it as a hard failure. Interactive mode keeps warn-and-fall-back.pre-verify-pr.shprefetches related-issue metadata (summary, labels, description, issuetype, comments) intoverify-pr-input.jsonunder a new optionalidempotencykey.execute_post_reportnow posts the Jira comment fromreport_adf(viaadf_to_markdown), matching Step 9's contract; GitHub still getsreport_md+ its commit marker.idempotencybundle: the schema now requiresidempotency.related_issues, so Step 0.7 rejects a prefetch omitting it before the tokenless Steps 6d/6f/7c dedup checks run (fail-fast, per TC-5980/TC-5981).pre_verify_pr.pyalways emits the bundle (empty list when the task has no relations); SKILL.md states the always-present invariant.idempotency.related_issues.itemsnow requires the six fields the dedup checks consume (key,summary,labels,description,issuetype,comments), so a structurally incomplete prefetched entry fails Step 0.7 instead of being silently treated as non-matching.build_idempotency_bundlealready emits all six; a test asserts an incomplete entry raisesValidationError.${FULLSEND_OUTPUT_DIR+x}) and fails fast on an exported-but-empty value, instead of${FULLSEND_OUTPUT_DIR:-not-set}which treated exported-but-empty as unset and fell back to the credentialed interactive path despite the sandbox gate being present.Verification:
python3 -m pytest plugins/sdlc-workflow/scripts/ -q(109 passed),uvx skillsaw(0 errors, Grade A),claude plugin validate plugins/sdlc-workflow(passed).Summary by Sourcery
Enable verify-pr to run safely in tokenless sandboxes while retaining the existing interactive workflow.
New Features:
Bug Fixes:
Enhancements:
Documentation:
Tests: