Bug: tool_choice is not enforced and malformed tool calls are returned as successful responses #649

Summary

POST /v1/chat/completions accepts OpenAI-style tools, tool_choice, and parallel_tool_calls, but FastFlowLM does not enforce the requested tool-selection policy or validate the parsed tool call before returning it.

Across Gemma 4 and two Qwen families, deliberately truncating a required tool call produces HTTP 200 with finish_reason: "tool_calls" even though the function name and/or arguments are invalid. A client will attempt to dispatch these fabricated calls and fail downstream.

This is distinct from model quality: the server owns the API contract and should never label an unusable parser result as a completed tool call.

Environment

• FastFlowLM: stock Linux release 0.9.46
• Hardware: AMD Ryzen AI NPU / XDNA2
• Endpoint: /v1/chat/completions, non-streaming
• Context length: 8192 for the reproduction matrix
• Temperature: 0.1
• parallel_tool_calls: false
• Test date: 2026-08-03

Minimal reproduction

Start any tested model, for example:

flm serve qwen3-it:4b --ctx-len 8192 --port 52625

Then send a required tool call with an intentionally small output budget:

curl -sS http://127.0.0.1:52625/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "qwen3-it:4b",
    "messages": [
      {"role": "user", "content": "Get the weather for Miami using the function."}
    ],
    "tools": [{
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Get the current weather for a location",
        "parameters": {
          "type": "object",
          "properties": {"location": {"type": "string"}},
          "required": ["location"]
        }
      }
    }],
    "tool_choice": "required",
    "parallel_tool_calls": false,
    "temperature": 0.1,
    "max_tokens": 1
  }'

Actual results

All three tool-capable model families returned HTTP 200 and finish_reason: "tool_calls" for an invalid, truncated call:

Representative Qwen response:

{
  "choices": [{
    "message": {
      "role": "assistant",
      "tool_calls": [{
        "type": "function",
        "function": {"name": "", "arguments": ""}
      }]
    },
    "finish_reason": "tool_calls"
  }]
}

There is a second policy-enforcement symptom on gemma3:1b: with tool_choice: "required" and a normal token budget, FastFlowLM returned plain assistant text with finish_reason: "stop". If that model does not support tools, the server should reject the request rather than silently ignore the required policy.

A longer Gemma 4 workflow also leaked raw template syntax as ordinary content instead of a structured call:

list directory{path:<|"|>.<|"|>}<tool_call|>

Expected behavior

• tool_choice: "none" must prevent tool calls.
• tool_choice: "required" must not return a successful response without at least one valid declared tool call.
• A named function choice must call exactly that declared function.
• parallel_tool_calls: false must allow at most one call in the response.
• Every returned call must name a declared function and contain a JSON-encoded object in function.arguments.
• If generation is truncated or parsing fails, return a non-dispatchable result such as finish_reason: "length", or a clear model/server error. Do not return finish_reason: "tool_calls" with an invalid call.
• If a model lacks tool support, reject requests that require tool use with a clear error.

Cross-check

The same test corpus includes a normal required call and a three-turn chain (read_dataset -> calculate_total -> final answer). Those succeed on the tested Gemma 4 and Qwen models, demonstrating that the models and parsers can produce valid calls when generation completes. The defect is that invalid results are accepted without enforcing the request contract.

A local response-validation prototype was also tested with gemma4-it:e4b; it passed the normal required call, fail-closed truncation, multi-turn chain, and a longer skill workflow. That suggests this can be fixed centrally in the OpenAI REST boundary instead of with per-model argument repair.

Prototype commit: waw2637@4978c1a

Related issues

• Bug: Tool calls with Object arguments are serialized as string #537 covers argument normalization/serialization.
• The behavior of gemma4-it:e4b running in FastFlowLM and llama.cpp is very different: the model running in FFLM does not use the appropriate tools #559 and [Issue]: Subpar Gemma 4 E4B output quality on FLM #641 cover model workflow/output quality.
• Gemma4 NPU2 conversions predate Google's July 2026 template fixes — porting notes for a refresh (2 parser shims) #634 covers updated Gemma 4 templates.

Those may improve how often a model emits valid calls, but they do not ensure that the REST API enforces tool_choice or fails closed when parsing produces an invalid call.