Skip to content

fix(anthropic): fall back to auto tool_choice for Opus 5.5, Sonnet 5.5, Fable 5.1 and Mythos 5.1 - #7511

Merged
davidzhao merged 1 commit into
livekit:mainfrom
omChauhanDev:fix/anthropic-forced-tool-choice
Oct 17, 2026
Merged

davidzhao merged 1 commit into
livekit:mainfrom
omChauhanDev:fix/anthropic-forced-tool-choice

Conversation

@omChauhanDev

@omChauhanDev omChauhanDev commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

Problem

LLM.chat() translates a forced tool_choice the same way for every model: "required" becomes {"type": "any"} and a named function becomes {"type": "tool", "name": ...}. Claude Opus 5.5, Sonnet 5.5, Fable 5.1 and Mythos 5.1 don't support forced tool use (Opus 5.5, Sonnet 5.5, Fable 5.1 and Mythos 5.1 migration guides), so on these models every such request fails with a 400: tool_choice: type "tool" and "any" are not supported for this model. AMD and keyterm detection force "required" on their own, so they fail on these models even when the app never sets a tool choice.

Changes

For these four models a forced choice is now sent as {"type": "auto"}, keeping disable_parallel_tool_use when it's set, and the plugin logs a warning once per LLM instance. Requests for all other models are unchanged. These models have no forced mode, so the model may now answer without calling a tool, or call a different tool than the one named.

@omChauhanDev
omChauhanDev requested a review from a team as a code owner September 28, 2026 02:58

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Devin Review

@toni-anthropic

Copy link
Copy Markdown

Thanks for this fix. One addition to consider before it merges: Claude Sonnet 5.5 (claude-sonnet-5-5) rejects a forced tool_choice the same way (migration guide, "Forced tool use is not supported"). I ran this branch (862a936) against the live API: through LLM.chat() with tool_choice="required", which AMD and keyterm detection use, Sonnet 5.5 still gets {"type": "any"} and the API returns:

400 invalid_request_error: tool_choice: type "tool" and "any" are not supported for this model.

With "claude-sonnet-5-5" added to _NO_FORCED_TOOL_CHOICE_PATTERNS, the request goes out as auto and the model calls the tool. claude-sonnet-5, claude-haiku-5-5 and claude-sonnet-4-6 still accept any and call the tool, so the entry doesn't need to be wider. #7631 makes the same change for the Bedrock plugin.

Happy to test the branch again once it's updated.

…5, Fable 5.1 and Mythos 5.1

Claude Opus 5.5, Sonnet 5.5, Fable 5.1 and Mythos 5.1 reject a forced tool_choice (`any` or `tool`) with a 400. For these models, send `auto` instead, keep `disable_parallel_tool_use`, and log a warning once per LLM instance.
@omChauhanDev omChauhanDev changed the title fix(anthropic): fall back to auto tool_choice for Opus 5.5, Fable 5.1 and Mythos 5.1 fix(anthropic): fall back to auto tool_choice for Opus 5.5, Sonnet 5.5, Fable 5.1 and Mythos 5.1 Oct 17, 2026
@omChauhanDev
omChauhanDev force-pushed the fix/anthropic-forced-tool-choice branch from 862a936 to e6fc2ad Compare October 17, 2026 03:42
@toni-anthropic

Copy link
Copy Markdown

Re-tested at e6fc2ad against the live API, with tool_choice="required" and with a named function. On main (f548149), Sonnet 5.5, Opus 5.5 and Fable 5.1 all return the 400. With this PR, all three send auto and call the tool. Sonnet 5 and Haiku 5.5 still send the forced choice, as before. Thanks for adding Sonnet 5.5.

@davidzhao
davidzhao merged commit 18f9505 into livekit:main Oct 17, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants