Skip to content

fix(aws): fall back to auto tool_choice for Claude Sonnet 5.5 on Bedrock - #7631

Open
kimnamu wants to merge 1 commit into
livekit:mainfrom
kimnamu:fix/bedrock-sonnet-5-5-tool-choice
Open

kimnamu wants to merge 1 commit into
livekit:mainfrom
kimnamu:fix/bedrock-sonnet-5-5-tool-choice

Conversation

@kimnamu

@kimnamu kimnamu commented Oct 5, 2026

Copy link
Copy Markdown
Contributor

Thanks for merging #7411 so quickly, @longcw. Claude Sonnet 5.5 rejects a forced toolChoice on Bedrock the same way Opus 5.5 does, but _MODELS_REJECTING_FORCED_TOOL_CHOICE only lists claude-opus-5-5 and claude-fable-5-1. So tool_choice="required" or a named tool still goes out as any/tool, and Bedrock returns a 400. This adds claude-sonnet-5-5 to that tuple. The sampling-params gate already covers Sonnet 5.5 through its claude-sonnet-5 substring.

case Before After
us.anthropic.claude-sonnet-5-5, tool_choice="required" ❌ ValidationException (400) ✅ sent as auto, the model called get_weather
same, named tool get_weather ❌ ValidationException (400) ✅ sent as auto, the model called get_weather
Opus 5.5, Fable 5.1 ✅ auto ✅ unchanged
Opus 5, Sonnet 5, Fable 5, Sonnet 4.6: required → any, named → tool ✅ ✅ unchanged (Sonnet 5 row added to the test)

Problem

ValidationException: An error occurred (ValidationException) when calling the ConverseStream operation:
The model returned the following errors: tool_choice: type "tool" and "any" are not supported for this model.

Fix

One entry in _MODELS_REJECTING_FORCED_TOOL_CHOICE. The substring match doesn't reach claude-sonnet-5, which still accepts any (now covered by test_forced_tool_choice_kept_for_other_models).

Tests

$ uv run pytest tests/test_plugin_aws_llm.py -q
16 passed in 1.50s
$ make type-check
✓ Type checking passed
$ uv run ruff check livekit-plugins/livekit-plugins-aws tests/test_plugin_aws_llm.py
All checks passed!
RED on main (source reverted, new test rows kept) and live runs
$ uv run pytest tests/test_plugin_aws_llm.py -q      # llm.py from main
>           assert await _tool_choice(model, "required") == {"auto": {}}
E           AssertionError: assert {'any': {}} == {'auto': {}}
1 failed, 15 passed in 0.47s

Live, us-east-1, aws.LLM(model="us.anthropic.claude-sonnet-5-5").chat(..., tools=[get_weather], tool_choice=...):

main     tool_choice='required'          sent={'any': {}}                   -> APIStatusError: ... tool_choice: type "tool" and "any" are not supported for this model. status_code=400
main     tool_choice=<named get_weather>  sent={'tool': {'name': 'get_weather'}} -> APIStatusError: ... status_code=400
this PR  tool_choice='required'          sent={'auto': {}}                  -> OK tool_calls=['get_weather']
this PR  tool_choice=<named get_weather>  sent={'auto': {}}                  -> OK tool_calls=['get_weather']

Sonnet 5.5 rejects a forced toolChoice (any/tool) with a ValidationException,
like Opus 5.5 and Fable 5.1. Add it to _MODELS_REJECTING_FORCED_TOOL_CHOICE.

Co-Authored-By: Claude <noreply@anthropic.com>
@kimnamu
kimnamu requested a review from a team as a code owner October 5, 2026 07:40

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Devin Review

@kimnamu

kimnamu commented Nov 5, 2026

Copy link
Copy Markdown
Contributor Author

Thanks @longcw for reviewing #7411, which added the forced tool_choice list that this PR extends with Claude Sonnet 5.5.

State open, no conflicts with main
CI all checks green on head 09274a6
main @ 296f28b Sonnet 5.5 with tool_choice="required" still gets a 400 ValidationException (live, us-east-1). With this branch applied, tests/test_plugin_aws_llm.py: 16 passed

Would you mind taking a look when you have a moment? If this isn't a priority right now, tell me and I'll close it to keep your queue clean.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants