serve: a request finalized after the next one starts clears its status (KeyError: 'tail', client sees RemoteDisconnected) #266

Summary

A request whose generator is finalized only after the next request has already started clears the new request's
status: it pops status["tail"], clears its busy, and records its status as its own history entry. The new request
then raises KeyError: 'tail' in Service._note and the client gets no response (connection closed). This looks like
a regression from the #212 fix (the self.status.pop("tail", None) added to run()'s outer finally).

Environment

• Strata 0.1.29 (serve/server.py at d6708a4), engine 0.1.29 sm_75 build
• Ubuntu 26.04, kernel 7.0.0-28-generic, x86_64; Python 3.14 (the bundled .venv)
• 2× RTX 2080 Ti 22 GB, --layer-split auto, --max-context 262144, --kv int8 --kv-resident 32768, --spec 4
• Client: a coding agent over the OpenAI-compatible API (streaming), which cancels/retries requests

Actual behavior

Exception occurred during processing of request from ('127.0.0.1', 49318)
Traceback (most recent call last):
  ...
  File "serve/server.py", line 1382, in do_POST
    self._openai(req)
  File "serve/server.py", line 1470, in _openai
    return self._json(200, openai_collect(chunks))
  File "serve/server.py", line 1123, in openai_collect
    for c in chunks:
  File "serve/server.py", line 1081, in openai_chunks
    for kind, x in run if run is not None else svc.run(ids, thinking, tools, max_new, req, cancel):
  File "serve/server.py", line 870, in run
    self._note(n, evs)
  File "serve/server.py", line 801, in _note
    s["tail"] = (s["tail"] + (ev.text or ""))[-600:]
KeyError: 'tail'

The client sees RemoteDisconnected (no HTTP response). It happened right after a long agent request, and once so far
in a few hours of use — it is timing dependent, not deterministic.

Root cause

Service.run() is a generator:

1. with self.fifo: releases the lock as soon as the generation loop ends.
2. The status cleanup (self.status["busy"] = False, self.status.pop("tail"), self.status.pop("tool")) is in the
   outer finally, which only runs when the consumer closes or GCs the generator — which can be after the next
   request has already acquired the fifo and set tail="".
3. So request A's finally runs against request B's status: it removes B's tail, sets B's busy = False, and (via
   if self.status.get("busy")) appends a history entry and totals built from B's status.
4. B's next _note raises KeyError: 'tail'.

The window opens when a client aborts/disconnects mid-stream and immediately re-sends (common for agents). Switching
the client to non-streaming does not avoid it: the race is server-side, and both the streaming and the non-streaming
path leave the generator suspended after done until it is finalized.

Expected behavior

A finished request must not touch the status of a request that has already replaced it.

Suggested fix

Tag each request and let it clean up only its own status:

# Service.__init__
self.req_id = 0

# Service.run(), where the request starts (inside status_lock)
self.req_id += 1
my_id = self.req_id
self.status.update(busy=True, ..., tail="", max_tokens=max_new, req=my_id)

# Service.run()'s outer finally
with self.status_lock:
    mine = self.status.get("req") == my_id
    if mine and self.status.get("busy"):
        ...  # history / totals, unchanged
    if mine:
        self.status["busy"] = False
        self.status.pop("tail", None)
        self.status.pop("tool", None)

A defensive s["tail"] = ((s.get("tail") or "") + (ev.text or ""))[-600:] in _note is worth keeping as a second line
of defence. The my_id = 0 default plus the req equality check means a request that fails before the status is
initialized never cleans up a status it never owned.

Verification (local)

With that change, 5 rounds of "abort a streaming request after 3 chunks, then immediately send another request" give
0 exceptions and 0 KeyErrors, every follow-up request returns 200, and /status still omits tail after a
completed request (the #212 behavior is preserved).