Skip to content

fix(base-action): don't reject successful runs when num_turns exceeds maxTurns - #1762

Open
gunjanjaswal wants to merge 1 commit into
anthropics:mainfrom
gunjanjaswal:fix/1758-num-turns-guard
Open

gunjanjaswal wants to merge 1 commit into
anthropics:mainfrom
gunjanjaswal:fix/1758-num-turns-guard

Conversation

@gunjanjaswal

Copy link
Copy Markdown

Fixes #1758.

run-claude-sdk.ts re-checked the SDK's result with num_turns > maxTurns and threw on a run the SDK had already reported as subtype: "success", is_error: false. The two quantities aren't comparable: maxTurns bounds model round trips, while num_turns counts every turn in the transcript, and batched tool calls inflate it well past the round-trip count. The issue has a concrete trace, a --max-turns 14 run returning num_turns: 27, reported as a success by the SDK and then failed by this guard, leaving the tracking comment stuck.

Dropping the guard doesn't lose genuine enforcement. A real overrun is reported by the SDK as a non-success subtype (error_max_turns), which the existing isSuccess check (subtype === "success" && !is_error) already treats as a failure and throws on. So typed maxTurns enforcement, is_error handling, and timeouts all stay intact; the only thing removed is the second-guess against a metric that doesn't line up.

Changes

  • Remove the post-result num_turns > maxTurns throw in base-action/src/run-claude-sdk.ts, with a comment on why the comparison isn't valid.
  • Tests: the old fixture asserted the wrong behavior (num_turns: 73 / maxTurns: 60 rejected). Replaced it with a batching-aware fixture (num_turns: 27 / maxTurns: 14 stays a success, matching the issue's trace), and added a test that an error_max_turns result still fails.

Testing

  • bun test base-action/test/run-claude-sdk.test.ts — 5 pass
  • bun run typecheck clean, prettier --check clean

Follow-up to #1607 / #1577. Per the issue, this keeps genuine error_max_turns enforcement rather than raising budgets or accepting failures.

… maxTurns

maxTurns bounds model round trips, but result.num_turns counts every turn in
the transcript, which batched tool calls inflate well past the round-trip
count. The post-result guard compared the two and threw on runs the SDK had
already reported as success within the limit (a --max-turns 14 run returning
num_turns 27, per anthropics#1758).

A genuine overrun is reported by the SDK as a non-success subtype
(error_max_turns) and already fails via the isSuccess check, so the num_turns
comparison was redundant and wrong. Remove it, and replace the
num_turns:73/maxTurns:60 test with a batching-aware fixture that stays a
success plus a test that a max-turns overrun still fails.

Closes anthropics#1758
@gunjanjaswal

Copy link
Copy Markdown
Author

Gentle nudge on this one. run-claude-sdk.ts re-checks the SDK result with num_turns > maxTurns and throws on a run the SDK already reported as success/is_error: false — but the two aren't comparable (maxTurns bounds model round trips, num_turns counts everything), so legitimate completions get flagged as failures (#1758). Small, contained fix. Would appreciate a first review when someone has time.

@gunjanjaswal

Copy link
Copy Markdown
Author

Following up on this, it's been open about five weeks without a review. It's a small, self-contained fix: run-claude-sdk.ts was throwing on runs the SDK had already reported as successful, because it compared num_turns against maxTurns, which bound different things (model round trips vs the turn budget). It still merges clean. Happy to walk through it or rebase if that makes a review easier.

@gunjanjaswal

Copy link
Copy Markdown
Author

Following up again here. Still a small, self-contained fix, and I'm happy to rebase if it's drifted since. Let me know if there's anything you'd like adjusted before it can go in.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Post-result num_turns > maxTurns guard can reject batched-tool traces within the round-trip limit

1 participant