flm serve: a request body with invalid UTF-8 leaves the NPU locked, and every later request queues forever #756

What happens

One request whose body is not valid JSON and contains an invalid UTF-8 byte takes the NPU lock and never releases it. Every later request to an NPU route (/v1/chat/completions, /api/chat, /api/generate, /v1/embeddings, /v1/audio/transcriptions) waits in the queue until the server is restarted.

flm serve llama3.2:1b
printf '{"messages": "\xe5"' | curl -s -m 10 http://127.0.0.1:52625/v1/chat/completions \
  -H 'Content-Type: application/json' --data-binary @-
curl -s -m 60 http://127.0.0.1:52625/v1/chat/completions -H 'Content-Type: application/json' \
  -d '{"model":"llama3.2:1b","messages":[{"role":"user","content":"Say ok."}],"max_tokens":8}'
# the second request never completes

The server log shows NPU Locked! for the first request, then invalid UTF-8 byte at index 174, and never NPU Lock Released!.

Expected

A 400 for the malformed body, and the lock released.

Cause

1. handle_request tries to parse the body for logging (server.cpp:646-661). The parse fails, so is_json stays false, and process_task then skips its own guarded parse (:689-712).
2. The route lambda parses the body again with no guard (:1021-1023). The resulting parse_error lands in the catch at :782-797.
3. That catch builds {"error": "Handler exception: " + e.what()} and calls .dump() (:787). nlohmann quotes the offending bytes in e.what(), and the default dump() throws type_error.316 on invalid UTF-8.
4. The second throw leaves the catch before process_next_npu_request() runs. It is caught only by the per-I/O-thread try around ioc.run() (:502-509), so the lock is never released.

We found a second path to the same leak by reading the source; we did not measure it. request_id is read into a std::string at server.cpp:719, outside every try, while the lock is held. A request with a non-string request_id ("request_id": 5) throws there and leaks the lock the same way. Also from reading the source: each such escape seems to end the ioc.run() loop of the I/O thread it happened on, so the thread pool shrinks by one per request.

Fix we are carrying

https://github.com/noamsto/nix-amd-ai/blob/46d7a73c1044435cec0ff08410e1a3bc87cbc544/pkgs/fastflowlm/patches/server-error-handling.patch

• safe_dump() dumps with json::error_handler_t::replace, so building the error body cannot throw. This is ported from OpenFlowLM-Next's server.cpp (Vegard Berget).
• A small RAII guard releases the NPU lock exactly once on every exit from process_task, including an exception nothing catches.
• A non-string request_id is refused with a 400 before it is read.
• A json::parse_error escaping a route lambda is answered with 400 "Invalid JSON".

With the patch, the request above gets a 400, and the log shows NPU Locked! and NPU Lock Released! around it. The next chat request completes.

Environment

FLM v1.0.6 built from the tag, plus one downstream patch that only changes how send_response maps error.code to an HTTP status. That patch does not touch these paths.
Ryzen AI MAX+ 395 (Strix Halo), XDNA2 NPU, 8 columns. NixOS, Linux 7.2.8, amdxdna 0.10, NPU FW 1.1.2.65. Model: llama3.2:1b.

We found this with test_a_body_that_is_not_json_is_refused_and_the_server_keeps_serving from OpenFlowLM-Next's server-api conformance suite.