[safeoutputs] Clarify auto-targeting behavior for add_comment, add_labels, and add_reviewer when no workflow context exists

[github-actions]

Analysis of the last 24 hours of workflow runs identified 1 confirmed error where an agent incorrectly used the add_comment safe output tool without an explicit target in a workflow_dispatch context. The same ambiguity affects add_labels and add_reviewer. The workflow prompt is not the root cause — the tool descriptions are misleading about when auto-targeting works vs. silently fails.

Error Analysis

Error Pattern: Omitting item_number in Non-PR/Issue/Discussion Trigger Contexts

Occurrences: 1 confirmed error (run §22229256501, Smoke Copilot, workflow_dispatch)

What the agent did wrong:
Called add_comment without item_number when the workflow was triggered by workflow_dispatch — a context with no triggering issue, PR, or discussion to auto-resolve.

Example from workflow Smoke Copilot (Run §22229256501):

{
  "body": "### Smoke Test Results...",
  "type": "add_comment"
}

Expected behavior: Either include item_number explicitly, or skip the add_comment call entirely when there is no triggering PR.

Why this happened: The item_number description for add_comment says:

"If omitted, the tool will attempt to resolve the target from the current workflow context (triggering issue, PR, or discussion)."

The agent interpreted "attempt to resolve" as "will succeed" and omitted item_number. For workflow_dispatch triggers there is no triggering issue/PR/discussion, so the resolution silently fails — the server returns {"result":"success"} but no comment is posted. The agent receives no feedback that the operation was a no-op.

Contrast with smoke-claude: The equivalent Claude smoke workflow (smoke-claude.md) works around this correctly because it contains an explicit conditional instruction:

"Only if this workflow was triggered by a pull_request event: Use the add_comment tool ... (omit the item_number parameter to auto-target the triggering PR)"

The Copilot smoke workflow lacks this conditional, and the tool description's vague language about auto-targeting gave the agent no reason to add it. This pattern is likely to recur in other workflows written by less-careful prompt authors.

Current Tool Descriptions

Relevant excerpts from pkg/workflow/js/safe_outputs_tools.json

add_comment → item_number:

{
  "type": "number",
  "description": "The issue, pull request, or discussion number to comment on. This is the numeric ID from the GitHub URL (e.g., 123 in github.com/owner/repo/issues/123). If omitted, the tool will attempt to resolve the target from the current workflow context (triggering issue, PR, or discussion)."
}

add_labels → item_number:

{
  "type": "number",
  "description": "Issue or PR number to add labels to. This is the numeric ID from the GitHub URL (e.g., 456 in github.com/owner/repo/issues/456). If omitted, adds labels to the item that triggered this workflow."
}

add_reviewer → pull_request_number:

{
  "type": ["number", "string"],
  "description": "Pull request number to add reviewers to. This is the numeric ID from the GitHub URL (e.g., 876 in github.com/owner/repo/pull/876). If omitted, adds reviewers to the PR that triggered this workflow."
}

Root Cause Analysis

Three related tool description problems:

1. Auto-targeting claim is unconditional: "If omitted, the tool will attempt to resolve..." / "If omitted, adds to the item that triggered this workflow." These phrases imply the field is always safely optional. They don't mention the condition for success.

2. Silent failure is undocumented: When auto-targeting fails (no triggering context), the server returns {"result":"success"} and does nothing. Agents have no signal that the operation failed.

3. Trigger type restriction is unspecified: Auto-targeting only works for issues, pull_request, discussion, issue_comment, and similar event triggers. It does not work for workflow_dispatch, schedule, push, workflow_run, or similar non-item triggers. This constraint is completely absent from the descriptions.

Recommended Improvements

Update Field Descriptions in pkg/workflow/js/safe_outputs_tools.json

For add_comment → item_number:

• Current: "If omitted, the tool will attempt to resolve the target from the current workflow context (triggering issue, PR, or discussion)."
• Suggested: "If omitted, the tool auto-targets the issue, PR, or discussion that triggered this workflow. Auto-targeting only works for issue, pull_request, discussion, and comment event triggers — it does NOT work for schedule, workflow_dispatch, push, or workflow_run triggers. For those trigger types, always provide item_number explicitly, or the comment will be silently discarded."

For add_labels → item_number:

• Current: "If omitted, adds labels to the item that triggered this workflow."
• Suggested: "If omitted, adds labels to the issue or PR that triggered this workflow. Only works for issue or pull_request event triggers. For schedule, workflow_dispatch, or other triggers, item_number is required — omitting it will silently skip the label operation."

For add_reviewer → pull_request_number:

• Current: "If omitted, adds reviewers to the PR that triggered this workflow."
• Suggested: "If omitted, adds reviewers to the PR that triggered this workflow. Only works for pull_request event triggers. For workflow_dispatch, schedule, or other triggers, pull_request_number is required — omitting it will silently skip the reviewer assignment."

Also update the smoke-copilot.md prompt

The Smoke Copilot workflow's OUTPUT section should be updated to mirror the conditional logic in smoke-claude.md:

• Add "Only if this workflow was triggered by a pull_request event:" before the add_comment instruction for the PR summary
• Add "Only if triggered by a pull_request event:" before the add_labels/remove_labels instructions

Affected Workflows

The following workflows had errors with this pattern:

• Smoke Copilot — 1 error (run §22229256501, workflow_dispatch)

Testing Plan

After updating the tool descriptions:

1. Trigger smoke-copilot via workflow_dispatch and verify the agent either skips the PR comment or provides an explicit item_number
2. Trigger smoke-copilot via a PR labeled event and verify auto-targeting still works (existing behavior preserved)
3. Monitor logs for 2-3 days to verify the pattern does not recur
4. Check other workflows using add_comment, add_labels, add_reviewer without explicit targets to identify any other affected prompts

Implementation Checklist

• Update item_number description for add_comment in pkg/workflow/js/safe_outputs_tools.json
• Update item_number description for add_labels in pkg/workflow/js/safe_outputs_tools.json
• Update pull_request_number description for add_reviewer in pkg/workflow/js/safe_outputs_tools.json
• Update smoke-copilot.md Output section to add trigger-type conditional guards
• Run make build to rebuild binary
• Run make recompile to update all workflows
• Run make test to ensure no regressions
• Deploy and monitor error rates

References

• Tool schema: pkg/workflow/js/safe_outputs_tools.json
• MCP server loader: actions/setup/js/safe_outputs_tools_loader.cjs
• Validator: actions/setup/js/safe_output_validator.cjs
• Smoke Claude prompt (good example): .github/workflows/smoke-claude.md lines 186–194
• Smoke Copilot prompt (needs fix): .github/workflows/smoke-copilot.md lines 148–160

Run IDs with errors: §22229256501

Generated by Daily Safe Output Tool Optimizer

• expires on Feb 22, 2026, 4:01 PM UTC