Skip to content

fix(#9449): recognize oversized Pi agent_end events - #9507

Open
Benkapner wants to merge 1 commit into
mainfrom
fix/9449-oversized-pi-agent-end
Open

Benkapner wants to merge 1 commit into
mainfrom
fix/9449-oversized-pi-agent-end

Conversation

@Benkapner

Copy link
Copy Markdown
Member

Summary

Pi includes the full message history in agent_end. Once that line exceeds 1 MiB, Fullsend can report incomplete and exit 1 after Pi has finished successfully.

Related Issue

Closes #9449.

Changes

  • Read oversized histories one message at a time and keep the final assistant outcome and willRetry flag.
  • Confirm a recovered result on agent_settled. Turn counts, tokens, and cost still come from the individual message_end events.

Testing

  • make lint and make go-vet passed.
  • Pi parser tests passed with -race, including retries, errors, aborts, and interrupted streams.
  • Replayed the original 206-turn transcript through PiRuntime.Run. main exited 1 with incomplete. This branch exited 0 with stop and reported the same token counts and cost.

Checklist

  • PR title follows Conventional Commits.
  • Commit is signed off (DCO).
  • I wrote this contribution myself and can explain all changes in it.

Pi repeats the message history on agent_end. Read large histories one
message at a time to retain the last assistant outcome and willRetry.
Confirm recovered results on agent_settled and keep usage accounting on
the individual message_end events.

Signed-off-by: Ben Kapner <benkapner@gmail.com>
@fullsend-ai-review

fullsend-ai-review Bot commented Nov 15, 2026 •

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 6:32 PM UTC · Completed 6:42 PM UTC

Commit: 500218a · View workflow run →

Runtime: pi · Model: sonnet → claude-sonnet-5-5 · Effort: high · Cost: $1.58

@fullsend-ai-review fullsend-ai-review Bot added the risk/moderate PR risk: moderate label Nov 15, 2026
@fullsend-ai-review

fullsend-ai-review Bot commented Nov 15, 2026 •

Copy link
Copy Markdown

Risk Assessment: moderate (2/5)

Details

Small two-file bug fix with 50% test coverage and no protected paths, moderately elevated by recent fix churn in pi_progress.go and absence of a feature flag.

Previous run

Risk Assessment: moderate (2/5)

Details

Small two-file bug fix with 50% test coverage and no protected paths, moderately elevated by recent fix churn in pi_progress.go and absence of a feature flag.

@fullsend-ai-review

fullsend-ai-review Bot commented Nov 15, 2026 •

Copy link
Copy Markdown

Looks good to me

Previous run

<!-- fullsend:review-findings-v2:eyJ2ZXJzaW9uIjoyLCJhY3Rpb24iOiJhcHByb3ZlIiwiZmluZGluZ3MiOltdfQ== -->

Looks good to me

Previous run (2)

<!-- fullsend:review-diff-hash:cd90016bd3b0aa2bf6b293acf4a4b38521d77e90734c62ff58d1d405bc776e23 -->
<!-- fullsend:review-head:500218ad7aca3f4cda4807b51f5886be6bc2cd2a -->
<!-- fullsend:review-findings-v2:eyJ2ZXJzaW9uIjoyLCJhY3Rpb24iOiJhcHByb3ZlIiwiZmluZGluZ3MiOltdfQ== -->

Looks good to me


Labels: The PR changes the Go Pi stream parser in the agent runner (internal/runtime).

Previous run (3)

<!-- fullsend:review-findings-v2:eyJ2ZXJzaW9uIjoyLCJhY3Rpb24iOiJhcHByb3ZlIiwiZmluZGluZ3MiOltdfQ== -->

Looks good to me


Labels: The PR changes the Go Pi stream parser in the agent runner (internal/runtime).

fullsend-ai-review[bot]

This comment was marked as outdated.

@fullsend-ai-review fullsend-ai-review Bot added requires-manual-review Review requires human judgment component/runner Agent runner behavior and lifecycle go Pull requests that update go code labels Nov 15, 2026
@codecov

codecov Bot commented Nov 15, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 91.35802% with 7 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
internal/runtime/pi_progress.go 91.35% 3 Missing and 4 partials ⚠️

📢 Thoughts on this report? Let us know!

@Benkapner
Benkapner marked this pull request as ready for review November 15, 2026 19:17
@Benkapner
Benkapner requested a review from a team as a code owner November 15, 2026 19:17
@qodo-code-review

Copy link
Copy Markdown

PR Summary by Qodo

Recognize oversized Pi agent_end events

🐞 Bug fix 🧪 Tests 🕐 20-40 Minutes

Grey Divider

AI Description

• Recover terminal outcomes from oversized Pi histories instead of reporting successful runs as
 incomplete.
• Require agent_settled to confirm recovered results; keep usage accounting on message_end events.
• Test long sessions, retries, errors, aborts, and interrupted streams.
Diagram

graph TD
  S["Pi NDJSON"] --> U["message_end usage"] --> D{"Oversized agent_end?"}
  D -->|Yes| R["Recover outcome"] --> T["agent_settled"] --> O["Result event"]
  D -->|No| N["Normal end"] --> O
Loading
High-Level Assessment

The following are alternative approaches to this PR:

1. Buffer the entire agent_end line
  • ➕ Would reuse ordinary JSON decoding.
  • ➖ Memory would grow with the full message history.
2. Stream-decode every history message
  • ➕ Could avoid hand-written boundary scanning.
  • ➖ Adds decoder complexity and still needs limits for oversized individual messages.

Recommendation: Keep the bounded recovery approach: it preserves the fields needed to determine the outcome without buffering the full history, and it fails closed when an assistant outcome cannot be recovered.

Files changed (2) +221 / -8

Bug fix (1) +117 / -8
pi_progress.goRecover outcomes from oversized Pi agent_end lines +117/-8

Recover outcomes from oversized Pi agent_end lines

• Reads oversized histories one message at a time, retaining the last assistant message and trailing fields such as willRetry. Recovered results remain pending until agent_settled; unreadable outcomes or interrupted streams remain incomplete. Usage accounting continues to use message_end events.

internal/runtime/pi_progress.go

Tests (1) +104 / -0
pi_progress_test.goCover oversized histories and terminal-state edge cases +104/-0

Cover oversized histories and terminal-state edge cases

• Adds parser tests for a 206-turn history, usage preservation, assistant errors, aborted settlement, successful retries, oversized assistant messages, and interrupted or truncated streams.

internal/runtime/pi_progress_test.go

@fullsend-ai-review

fullsend-ai-review Bot commented Nov 15, 2026 •

Copy link
Copy Markdown

🤖 Review · ❌ Terminated · Started 7:19 PM UTC · Ended 7:24 PM UTC

Commit: 500218a · View workflow run →

@qodo-code-review

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (1) 📘 Rule violations (0) 📎 Requirement gaps (0) 🔗 Cross-repo conflicts (0) 📜 Skill insights (0)

Grey Divider


Remediation recommended

1. Large earlier replies still fail runs 🐞 Bug ≡ Correctness
Description
readPiAgentEnd rejects the entire history when any assistant entry exceeds the buffer, even if a
later, readable assistant entry contains the final outcome. If an earlier turn contains a large
assistant message, the terminal event is discarded and a successfully settled run is reported as
incomplete.
Code

internal/runtime/pi_progress.go[R116-119]

+		if oversized {
+			// A single oversized assistant may have lost its message_end
+			// as well. Its stop reason cannot be recovered safely.
+			valid = piNonAssistantMessage.Match(message)
Relevance

●● Moderate

Real correctness bug in the new parser path, but no close historical precedent either way. Accepted
and dismissed hits are on different parser logic, so this stays uncertain.

PR-#9186
PR-#9147

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The new per-entry cap marks an oversized message, and the oversized branch sets valid=false for
any assistant role. That makes recovery return no event before the parser can inspect a later
assistant; without a terminal event, its EOF fallback reports incomplete. The normal parser
selects the last assistant from the history, while the added test covers only an oversized final
assistant.

internal/runtime/pi_progress.go[110-126]
internal/runtime/pi_progress.go[178-182]
internal/runtime/pi_progress.go[325-333]
internal/runtime/pi_progress.go[638-668]
internal/runtime/pi_progress_test.go[493-498]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
An oversized assistant entry earlier in `agent_end.messages` invalidates the entire event, even when a later assistant entry supplies a readable final outcome.
## Fix Focus Areas
- internal/runtime/pi_progress.go[110-130]
- internal/runtime/pi_progress.go[178-182]
- internal/runtime/pi_progress_test.go[493-498]
## Recommended Fix
Continue scanning after an oversized assistant entry and track whether the latest assistant remains unreadable. Clear that condition when a later valid assistant entry is read; reject the recovered event only if its final assistant outcome is unreadable. Add a test with an oversized earlier assistant followed by a small final assistant and `agent_settled`.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Dismiss ↗ | View ↗


Grey Divider

Context sources
✅ Compliance rules (platform): 96 rules
✅ Skills: 5 invoked
  writing-user-docs
  writing-adrs
  check-patch-coverage
  cutting-releases
  adr-corner
✅ Cross-repo context — repo relationships
  Explored: repo: fullsend-ai/agents (sha: 0869ded8) — View relationship
  Explored: repo: fullsend-ai/mlflow-assessments (sha: 0058f877) — View relationship
Review mode: Auto: ⚖️ Balanced: Downgraded extended -> standard: change is below the extended eligibility bar (hunks 9/18, lines 229/200; both must reach the floor). Router rationale: Complex streaming parser changes span recovery, retries, errors, and settlement state.

Grey Divider

Tip of the day
💡 Did you know, you can reply 'qodo' on any finding to push back, ask questions, or dig deeper

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

Grey Divider

Qodo Logo

Comment on lines +116 to +119
if oversized {
// A single oversized assistant may have lost its message_end
// as well. Its stop reason cannot be recovered safely.
valid = piNonAssistantMessage.Match(message)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remediation recommended

1. Large earlier replies still fail runs 🐞 Bug ≡ Correctness

readPiAgentEnd rejects the entire history when any assistant entry exceeds the buffer, even if a
later, readable assistant entry contains the final outcome. If an earlier turn contains a large
assistant message, the terminal event is discarded and a successfully settled run is reported as
incomplete.
Agent Prompt
## Issue description
An oversized assistant entry earlier in `agent_end.messages` invalidates the entire event, even when a later assistant entry supplies a readable final outcome.
## Fix Focus Areas
- internal/runtime/pi_progress.go[110-130]
- internal/runtime/pi_progress.go[178-182]
- internal/runtime/pi_progress_test.go[493-498]
## Recommended Fix
Continue scanning after an oversized assistant entry and track whether the latest assistant remains unreadable. Clear that condition when a later valid assistant entry is read; reject the recovered event only if its final assistant outcome is unreadable. Add a test with an oversized earlier assistant followed by a small final assistant and `agent_settled`.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Dismiss ↗ | View ↗

@fullsend-ai-review
fullsend-ai-review Bot dismissed their stale review November 15, 2026 19:24

Superseded by updated review

@fullsend-ai-review fullsend-ai-review Bot added ready-for-merge All reviewers approved — ready to merge and removed requires-manual-review Review requires human judgment labels Nov 15, 2026
@fullsend-ai-review

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 7:19 PM UTC · Completed 7:24 PM UTC

Commit: 500218a · View workflow run →

Runtime: pi · Model: sonnet → claude-sonnet-5-5 · Effort: high · Cost: $0.70

This branch was previously deployed

1 inactive deployment
dev — 500218ad Deployed Nov 15, 2026 by Benkapner via behaviour #20346
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

component/runner Agent runner behavior and lifecycle go Pull requests that update go code ready-for-merge All reviewers approved — ready to merge risk/moderate PR risk: moderate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

parsePiStream: recognize oversized agent_end events instead of failing stream as incomplete

1 participant