flm serve: streaming /api/chat segfaults the server on every request (insert() twice, generate() never called) #757

What happens

Every POST /api/chat with "stream": true crashes flm serve with SIGSEGV. The client gets no bytes back, and every other client loses the server too.

flm serve llama3.2:1b
curl -sN http://127.0.0.1:52625/api/chat -H 'Content-Type: application/json' \
  -d '{"model":"llama3.2:1b","messages":[{"role":"user","content":"Count from 1 to 5."}],"stream":true}'
# connection closes with no response; the flm process has exited (SIGSEGV)

It reproduces on llama3.2:1b and gemma4-it:e4b. The server log shows one Prefill chunk 1/1 line, then nothing. Backtraces from the core dumps:

llama3.2:1b                                  gemma4-it:e4b
#0 Sampler::sample_greedy(buffer<bf16>&)     #0 Sampler::sample_greedy(buffer<bf16>&)
#1 AutoModel::_shared_insert(...)            #1 AutoModel::_shared_insert(...)
#2 Llama3::insert(...)                       #2 Gemma4e::insert(...)
#3 RestHandler::handle_chat(...)             #3 RestHandler::handle_chat(...)

Cause

The streaming branch of handle_chat runs the insert() try block twice (rest_handler.cpp:792-819). It never calls generate() before ostream.finalize_chat() (:823).

The first insert() writes the whole prompt to token_history. The second one then prefix-matches all of it in AutoModel::_shared_insert (automodel.cpp:184-201). It erases every token, prefills an empty vector, and samples an empty logits buffer (:234).

This looks like the same code as #436, which reported empty chunks on v0.9.36 and was closed because the Ollama API is no longer maintained. On v1.0.6 the result is no longer an empty stream: the whole process crashes, including for clients that only use /v1/*. If the Ollama routes are staying unmaintained, removing /api/chat would also close this.

Fix we are carrying

Replace the second insert() block with the generate() block that streaming /api/generate already uses (:690-697):
https://github.com/noamsto/nix-amd-ai/blob/46d7a73c1044435cec0ff08410e1a3bc87cbc544/pkgs/fastflowlm/patches/stream-chat-generate.patch

The diff is small. Our copy of the file carries other patches, so the context lines differ from upstream's. With the fix, both models stream token chunks and then one done: true chunk with stats. The streamed text is byte-identical to the non-streaming reply, and there is one prefill per request. I can send this as a PR against main if that's useful.

Probes and logs: https://github.com/noamsto/nix-amd-ai/blob/46d7a73c1044435cec0ff08410e1a3bc87cbc544/bench-logs/oflm-api-conformance-2026-09-28-after-187/README.md

Environment

FLM v1.0.6 built from the tag, with our downstream request-handling patches applied. In this block those patches change only the bodies of the catch clauses. The double insert() and the missing generate() are upstream code.
Ryzen AI MAX+ 395 (Strix Halo), XDNA2 NPU, 8 columns. NixOS, Linux 7.2.8, amdxdna 0.10, NPU FW 1.1.2.65.