Skip to content

feat(#582): support exit code 78 as neutral pre-script skip - #5961

Merged
maruiz93 merged 4 commits into
mainfrom
agent/582-prescript-neutral-exit-code
Aug 18, 2026
Merged

maruiz93 merged 4 commits into
mainfrom
agent/582-prescript-neutral-exit-code

Conversation

@fullsend-ai-coder

Copy link
Copy Markdown
Contributor

Summary

  • Add exit code 78 ("neutral") support to pre-script handling in fullsend run, so pre-scripts can signal "nothing to do, skip cleanly" without using the file-based output protocol
  • When a pre-script exits 78, the run skips sandbox creation and LLM invocation, reports a ⏭ skipped status, and exits 0 — saving inference cost and avoiding false error noise
  • The output file is still parsed best-effort for reason and other outputs; when no file reason exists, the last non-empty stdout line becomes the skip reason

Changed files

  • internal/prescript/prescript.go — add ExitCodeNeutral = 78 constant
  • internal/cli/run.go — detect exit 78 in runPreScript(), capture stdout for fallback reason, add lastNonEmptyLine() helper
  • internal/cli/prescript_run_test.go — 10 new tests covering exit 78 behavior (stdout reason, file reason, override of skipped=false, malformed file tolerance, relay, regression guard for other exit codes)
  • docs/normative/prescript-output/v1/README.md — document exit code 78 semantics in the normative spec
  • docs/guides/user/building-custom-agents.md — add user-facing documentation with usage example

Testing

  • All 10 new pre-script tests pass with -race
  • All existing pre-script tests pass (no regressions)
  • All internal/prescript package tests pass
  • go vet clean on both changed packages

Note

Pre-commit could not run in sandbox (network access denied for hook installation). The post-script runs an authoritative pre-commit check on the runner.


Closes #582

Post-script verification

  • Branch is not main/master (agent/582-prescript-neutral-exit-code)
  • Secret scan passed (gitleaks — 2c340de079acaf412144c5b421630f63b7511c71..HEAD)
  • PR body secret scan passed (gitleaks — no-git)

@fullsend-ai-coder
fullsend-ai-coder Bot requested a review from a team August 5, 2026 22:15
@fullsend-ai-coder fullsend-ai-coder Bot added the ready-for-review Triggers review agent dispatch label Aug 5, 2026
@fullsend-ai-review

fullsend-ai-review Bot commented Aug 5, 2026 •

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 10:16 PM UTC · Completed 10:31 PM UTC
Commit: 3fd6517 · View workflow run →

@github-actions

github-actions Bot commented Aug 5, 2026 •

Copy link
Copy Markdown

Site preview

Preview: https://17e4801d-site.fullsend-ai.workers.dev

Commit: a0e7a02b3bd2a9d96e6ae9da3de16654c35e8658

@codecov

codecov Bot commented Aug 5, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@fullsend-ai-review

fullsend-ai-review Bot commented Aug 5, 2026 •

Copy link
Copy Markdown

Review

Findings

Low

  • [naming-consistency] internal/cli/run.go — lastNonEmptyLine bundles line extraction with byte-length truncation and UTF-8 boundary repair alongside the delegated stripControlChars call. The surrounding codebase tends toward single-purpose helpers, though the concerns here are tightly coupled to stdout-reason extraction and stripControlChars is already factored out. No action required.

Labels: PR adds new user-facing feature (exit code 78 pre-script skip) in the harness CLI runtime

Previous run

Review

Findings

Low

  • [edge-case] internal/cli/run.go:2712 — lastNonEmptyLine truncates at 1024 bytes using len(l) and l[:1024], which operates on byte offsets. If the stdout-derived reason contains multi-byte UTF-8 characters and the 1024th byte falls within a multi-byte sequence, the resulting string will end with an incomplete (invalid) UTF-8 codepoint. This does not cause a runtime failure — validateOutputs passes it through, and Relay writes it to GITHUB_OUTPUT — but the relayed reason will have a garbled trailing character. Consider truncating at a valid rune boundary.

  • [incomplete-implementation-documentation] docs/guides/dev/cli-internals.md:382 — The CLI internals diagram shows the pre-script step with the comment "skipped=true exits 0 here (Remove redundant pre-code.sh / pre-fix.sh calls from reusable workflows #4718)" but does not mention that exit code 78 is also a valid skip signal at this pipeline stage. A brief update (e.g., "skipped=true or exit 78 signals skip (Remove redundant pre-code.sh / pre-fix.sh calls from reusable workflows #4718, Support graceful pre-script skip (neutral exit code) #582)") would keep the diagram current.

Previous run (2)

Review

Findings

Medium

  • [incomplete-specification] docs/normative/prescript-output/v1/README.md — The normative spec's "Stdout as fallback reason" section specifies that the last non-empty line of stdout is used and that file-based reason takes precedence, but does not specify maximum line length or character restrictions for stdout-derived reasons. The implementation applies TrimSpace but no explicit length or character validation on stdout-derived reasons (unlike file-based values, which ParseFile validates against control characters). Note: the Relay function's validateOutputs does catch control characters when relaying to GITHUB_OUTPUT, so this gap only affects non-CI contexts where Relay is not called.

Low

  • [incomplete-feature-documentation] docs/guides/user/customizing-agents.md:70 — The pre-script protocol reference only mentions file-based skip signaling. A brief mention of exit code 78 with a link to the normative spec would help users discover the simpler approach.

  • [incomplete-feature-documentation] docs/architecture.md:155 — The architecture doc describes pre-script skip signaling as writing skipped=true but does not mention exit code 78 as an alternative.

Previous run (3)

Review

Findings

Low

  • [edge case / test gap] internal/cli/prescript_run_test.go — No test covers exit 78 when the pre-script deletes the output file. The existing TestRunPreScript_Exit78_MalformedOutputFileStillSkips exercises the parseErr!=nil branch via malformed content, but a dedicated deleted-file test would document the intentionally different behavior between exit 0 (deleted file = hard error) and exit 78 (deleted file = skip anyway).

  • [protocol-versioning-clarity] docs/normative/prescript-output/v1/README.md:32 — The PR replaces the invariant "A non-zero script exit remains a hard failure" with an exit-78 exception. The existing Versioning section covers this via "relax the grammar" (non-breaking), but an explicit note clarifying that exit code semantics are part of the v1 protocol surface would strengthen the spec.


Labels: PR modifies pre-script handling in the harness runtime (internal/cli/run.go, internal/prescript/) and adds a new feature

fullsend-ai-review[bot]

This comment was marked as outdated.

@fullsend-ai-review fullsend-ai-review Bot added ready-for-merge All reviewers approved — ready to merge component/harness Agent harness, config, and skills loading labels Aug 5, 2026
@maruiz93
maruiz93 force-pushed the agent/582-prescript-neutral-exit-code branch from 3fd6517 to 091a552 Compare August 18, 2026 00:10
@fullsend-ai-review

fullsend-ai-review Bot commented Aug 18, 2026 •

Copy link
Copy Markdown

🤖 Review · ⚠️ Cancelled · Started 12:11 AM UTC · Ended 12:22 AM UTC

Commit: 091a552 · View workflow run →

@maruiz93

Copy link
Copy Markdown
Contributor

Addressed both findings in 4a5a285:

  • edge case / test gap: Added TestRunPreScript_Exit78_DeletedOutputFileStillSkips — exercises the parseErr!=nil branch via a missing (not malformed) file, documenting the difference from exit 0.
  • protocol-versioning-clarity: Added exit code semantics to the Versioning table and a note explaining why exit 78 is non-breaking.

@fullsend-ai-review

fullsend-ai-review Bot commented Aug 18, 2026 •

Copy link
Copy Markdown

🤖 Review · ⚠️ Cancelled · Started 12:23 AM UTC · Ended 12:25 AM UTC

Commit: 4a5a285 · View workflow run →

@fullsend-ai-review

fullsend-ai-review Bot commented Aug 18, 2026 •

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 12:26 AM UTC · Completed 12:42 AM UTC

Commit: 0c62270 · View workflow run →

@fullsend-ai-review fullsend-ai-review Bot added requires-manual-review Review requires human judgment and removed ready-for-merge All reviewers approved — ready to merge labels Aug 18, 2026
@maruiz93

Copy link
Copy Markdown
Contributor

Addressed all three findings from the re-review in bee05d6:

  • [incomplete-specification]: lastNonEmptyLine now strips control characters (matching file-based validation) and caps at 1024 bytes. Documented in the normative spec's stdout section.
  • [incomplete-feature-documentation] customizing-agents.md: Added exit 78 mention alongside skipped=true with link to normative spec.
  • [incomplete-feature-documentation] architecture.md: Added exit 78 as an alternative skip mechanism.

@fullsend-ai-review

fullsend-ai-review Bot commented Aug 18, 2026 •

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 12:50 AM UTC · Completed 1:05 AM UTC

Commit: bee05d6 · View workflow run →

fullsend-ai-review[bot]

This comment was marked as outdated.

@fullsend-ai-review fullsend-ai-review Bot added ready-for-merge All reviewers approved — ready to merge and removed requires-manual-review Review requires human judgment labels Aug 18, 2026
@maruiz93

Copy link
Copy Markdown
Contributor

Addressed both new findings from round 3 in f2fafb2:

  • [edge-case] UTF-8 truncation: lastNonEmptyLine now backs off from the 1024-byte cut point to a valid rune boundary, preventing split multi-byte characters.
  • [incomplete-implementation-documentation] cli-internals.md: Updated the pre-script diagram to mention exit 78 alongside skipped=true.

The re-raised [incomplete-specification], [incomplete-feature-documentation] (customizing-agents.md, architecture.md) findings were already addressed in bee05d6.

@fullsend-ai-review

fullsend-ai-review Bot commented Aug 18, 2026 •

Copy link
Copy Markdown

🤖 Review · ⚠️ Cancelled · Started 1:11 AM UTC · Ended 1:13 AM UTC

Commit: f2fafb2 · View workflow run →

fullsend-ai-coder Bot and others added 4 commits August 18, 2026 03:12
Pre-scripts can now signal "nothing to do" by exiting with code 78
(the CI convention for neutral). When fullsend run sees this exit code
it skips sandbox creation and LLM invocation, reports a skipped status,
and exits 0 — avoiding both the cost of a no-op agent run and the
noise of a false failure.

Exit 78 is complementary to the existing file-based skip protocol
(skipped=true in FULLSEND_PRESCRIPT_OUTPUT). Either mechanism alone
is sufficient. When both are present, exit 78 is authoritative.

The output file is still parsed best-effort on exit 78 for reason and
other outputs. When no reason key is found, the last non-empty line
of the script's stdout becomes the skip reason, enabling simple
scripts like: echo "No issues need scoring" && exit 78

Other non-zero exit codes remain hard failures (unchanged).

Closes #582
Address review findings: add a test covering exit 78 when the
pre-script deletes the output file (documenting the intentional
difference from exit 0), and note in the normative spec's Versioning
section that exit code semantics are part of the v1 protocol surface.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Marta Anon <manon@redhat.com>
Strip control characters and cap stdout-derived reasons at 1024 bytes,
matching the file-based value validation. Document the constraints in
the normative spec. Add exit 78 mentions to customizing-agents.md and
architecture.md so users can discover the simpler skip mechanism.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Marta Anon <manon@redhat.com>
…ernals

Fix UTF-8 truncation in lastNonEmptyLine: back off from the 1024-byte
cut point to a valid rune boundary so multi-byte characters are not
split. Update the CLI internals diagram to mention exit 78 alongside
skipped=true.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Marta Anon <manon@redhat.com>
@maruiz93
maruiz93 force-pushed the agent/582-prescript-neutral-exit-code branch from f2fafb2 to a0e7a02 Compare August 18, 2026 01:13
@fullsend-ai-review

fullsend-ai-review Bot commented Aug 18, 2026 •

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 1:14 AM UTC · Completed 1:31 AM UTC

Commit: a0e7a02 · View workflow run →

@fullsend-ai-review fullsend-ai-review Bot added the type/feature New capability request label Aug 18, 2026
@maruiz93
maruiz93 added this pull request to the merge queue Aug 18, 2026
Merged via the queue into main with commit 05416e9 Aug 18, 2026
24 checks passed
@maruiz93
maruiz93 deleted the agent/582-prescript-neutral-exit-code branch August 18, 2026 01:59
@fullsend-ai-retro

fullsend-ai-retro Bot commented Aug 18, 2026 •

Copy link
Copy Markdown

🤖 Finished Retro · ❌ Failure · Started 2:01 AM UTC · Completed 2:03 AM UTC

Commit: a0e7a02 · View workflow run →

This branch was successfully deployed

1 active deployment
site-preview — a0e7a02b Deployed Aug 18, 2026 by github-actions[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

component/harness Agent harness, config, and skills loading ready-for-merge All reviewers approved — ready to merge ready-for-review Triggers review agent dispatch type/feature New capability request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support graceful pre-script skip (neutral exit code)

1 participant