fix(local): refuse a chat turn whose images outgrow the model's window

docs/architecture/chat.md

@@ -52,15 +52,15 @@ The list handed to the generator is `[system, *history within budget, user]`.
 - **History** is the thread's stored turns in `created_at` order, flattened to role and text. Stored citations and reasoning are for the UI, not the model. The newest earlier turn that carried images also carries them, unless the new turn brings its own ([Images](#images)).
 - **Sliding window.** The system message and the new user message are pinned; the most recent prior turns that fit the history budget are kept and older ones dropped.
 - **Budget** ([`budget.py`](../../surfsense_local/backend/modules/chat/budget.py)). One context window is shared by the system prompt (priced at 400 tokens), the excerpts (2,400: five hits of up to 480 tokens), the question (1,024, which `MessageText` enforces at the wire as 4,096 characters at the same four-characters-a-token estimate, refusing more with a 422 before any model is resolved) and a 1,024-token answer reserve, which is claimed first because llama.cpp stops a reply wherever the window runs out. History gets what remains of the model's window, floored at zero, so a narrow window means a shorter history rather than an overflowing turn. llama.cpp reports the window it actually allocated; an OpenAI-compatible endpoint reports none, and then history gets 3,000 tokens and `max_tokens` is left to the endpoint.
-- Each prior turn is priced by the local runtime's own tokenizer when it answers, and by `len(text) // 4` otherwise, plus 1,400 tokens per image it carries. The new turn's images are paid for out of the history budget before any prior turn is priced.
+- Each prior turn is priced by the local runtime's own tokenizer when it answers, and by `len(text) // 4` otherwise, plus 1,400 tokens per image it carries. The new turn's images are paid for out of the history budget before any prior turn is priced. That 1,400 is a ceiling for the trim, not a limit: the turn is refused with a 409, before anything is stored, only when a lower bound overflows a known window, the answer reserve plus the turn's own excerpts and question plus 256 per image, Gemma 3's fixed cost. The detail names how many images could fit; an unknown window lets the turn through.
 - **Retrieval query** is the new user message.
 
 ## The stream
 
 `POST .../messages` does, in order:
 
 1. Resolve the generation selection. With none, it answers `409` "no chat model selected" before anything streams, and the frontend opens model setup. It also validates `document_ids`, answers `503` if the embedding model files are missing, loads the history and runs `retrieve()`.
-2. Ask the generator whether the model reads images. A turn carrying images to a model that answers no gets a `409` and nothing is stored; images that are not an accepted image get a `422`.
+2. Ask the generator whether the model reads images. A turn carrying images to a model that answers no gets a `409` and nothing is stored, and so does one whose images overflow a known window even at 256 each ([Message assembly](#message-assembly)); images that are not an accepted image get a `422`.
 3. Build the context and the message list, with any images and retrieved image sources, and read the model's context window.
 4. Mark the model in use, so it cannot be deleted mid-answer; a deletion already in progress makes this a `409`.
 5. Store the user turn, with its images' files, and an empty assistant turn together. Their ids stay stable for the whole stream.
@@ -168,12 +168,12 @@ The same routes then reach the agent ([`modules/agent/agent_threads/`](../../sur
 
 ## Known gaps
 
+- The images `409` prices each image at Gemma 3's 256, so a dearer projector can still overflow unrefused: four Qwen2.5-VL images at the 1,024 px cap with five excerpts outgrow the 8,192 floor and fail as `context_too_long`.
 - An agent thread ignores the thinking switch, and its composer still shows the button as if it applied.
 - An agent thread's session is deleted only while opencode is running; one deleted before any turn has started opencode in this run of the app stays in opencode's database.
 - An agent thread's first turn is named after its first words, not by the model as a chat's is.
 - An agent thread refuses images.
 - No live region announces streamed text, and focus does not move to the conversation heading after a thread switch; the dashboard design asks for both.
 - A thinking model spends the 1,024-token answer cap on its reasoning too: `max_tokens` counts what goes to `reasoning_content`, as the title measurement in [`local-models/runtime.md`](local-models/runtime.md#turning-thinking-off) shows, so on the local runtime a long think can cut the answer short or leave it empty. The trace now shows, so an empty answer is no longer unexplained, but how often it happens is unmeasured.
 - Nothing shows progress while a model loads beyond "Thinking": llama-server's prompt progress starts once the model is up ([`local-models/runtime.md`](local-models/runtime.md#prompt-progress)).
-- Nothing refuses a turn whose images alone outgrow the window. History floors at zero, but four attached images at 1,400 tokens each overflow a small local window before any history is kept, and the turn fails as `context_too_long`.
 - The per-image cost is priced, not measured: Gemma 3, the curated vision model, spends a fixed 256, and 1,400 covers Qwen2.5-VL at the 1,024 px cap. No measurement at the pinned build confirms either.

surfsense_local/backend/modules/chat/budget.py

@@ -46,6 +46,10 @@
 # drops a turn rather than overflowing the window.
 IMAGE_TOKENS = 1400
 
+# The cheapest image the code knows: Gemma 3's fixed 256. A refusal prices at
+# this, so it only stops a turn that overflows even at Gemma 3's 256.
+SMALLEST_IMAGE_TOKENS = 256
+
 # What MessageText refuses past, at the wire (modules/chat/schemas.py). One
 # global cap rather than one per window: it runs before any model is resolved,
 # and the budget prices the question at the same 1,024 for every window.
@@ -81,3 +85,16 @@ def answer_max_tokens(n_ctx: int | None) -> int | None:
     a limit that was never really theirs.
     """
     return ANSWER_RESERVE_TOKENS if n_ctx is not None else None
+
+
+def image_room(n_ctx: int | None, text_chars: int) -> int | None:
+    """The most images the window could take, or None when it is unknown.
+
+    A lower bound on the turn's cost: the answer reserve, the turn's own text
+    and Gemma 3's 256 per image. IMAGE_TOKENS stays the price the history trim
+    uses; as a limit it would refuse turns Gemma 3 answers.
+    """
+    if n_ctx is None:
+        return None
+    left = n_ctx - ANSWER_RESERVE_TOKENS - text_chars // CHARS_PER_TOKEN
+    return max(0, left // SMALLEST_IMAGE_TOKENS)

surfsense_local/backend/modules/chat/router.py

@@ -20,7 +20,12 @@
 from modules.agent.agent_threads.turn import agent_turn
 from modules.agent.dependencies import LaunchKeyDep
 from modules.agent.engine_choice import selected_model_can_run_agent
-from modules.chat.budget import IMAGE_TOKENS, answer_max_tokens, history_budget
+from modules.chat.budget import (
+    IMAGE_TOKENS,
+    answer_max_tokens,
+    history_budget,
+    image_room,
+)
 from modules.chat.dependencies import ThreadDep
 from modules.chat.errors import classify_chat_error, empty_reply_error
 from modules.chat.history import TokenCounter, build_messages
@@ -195,6 +200,7 @@ async def send_message(
         len(citations),
     )
     n_ctx = await _context_tokens_or_none(generator, selected.name)
+    _refuse_images_past_window(len(images), n_ctx, len(context) + len(payload.text))
     found = await _source_images(session, hits, sees, n_ctx, len(images))
     messages = await build_messages(
         context,
@@ -408,6 +414,27 @@ async def _accepted_images(
         ) from error
 
 
+def _refuse_images_past_window(
+    attached: int, n_ctx: int | None, text_chars: int
+) -> None:
+    """Refuse before anything is stored a turn whose images overflow even at
+    Gemma 3's 256 each: trimming history cannot make room for them, so the model would only
+    fail it later as `context_too_long`."""
+    room = image_room(n_ctx, text_chars)
+    if room is None or attached <= room:
+        return
+    raise HTTPException(
+        status.HTTP_409_CONFLICT,
+        (
+            "This model's context window has no room for images."
+            if room == 0
+            else f"This model's context window has room for at most {room} "
+            + ("image." if room == 1 else "images.")
+        )
+        + " Send fewer, or choose a model with a larger window.",
+    )
+
+
 async def _source_images(
     session: Session,
     hits: list[Hit],

surfsense_local/backend/tests/integration/chat/test_images.py

@@ -10,7 +10,7 @@
 from PIL import Image as Pillow
 from sqlalchemy import Engine
 
-from tests.integration.chat.conftest import set_sees
+from tests.integration.chat.conftest import set_props_n_ctx, set_sees
 from tests.integration.chat.test_chat import _open_thread, _seed
 
 pytestmark = pytest.mark.integration
@@ -98,6 +98,48 @@ async def test_a_model_that_cannot_see_is_refused_and_nothing_is_kept(
     assert llamacpp_server == []
 
 
+async def test_images_that_outgrow_the_window_are_refused_and_nothing_is_kept(
+    client: AsyncClient,
+    engine: Engine,
+    real_model: object,
+    llamacpp_server: list[dict],
+    data_dir: Path,
+) -> None:
+    """2,048 tokens hold at most four 256-token images once the answer is
+    reserved, fewer with the turn's text: four overflow even at 256 each."""
+    set_sees(True)
+    set_props_n_ctx(2048)
+    workspace_id, _ = _seed(engine)
+    thread_id = await _open_thread(client, workspace_id)
+
+    status, body = await send(client, thread_id, "what are these?", [picture()] * 4)
+
+    assert status == 409
+    assert "Send fewer" in json.loads("".join(body))["detail"]
+    assert (await client.get(f"/chat/threads/{thread_id}/messages")).json() == []
+    assert stored_images(data_dir) == []
+    assert llamacpp_server == []
+
+
+async def test_images_a_cheap_projector_fits_are_sent(
+    client: AsyncClient,
+    engine: Engine,
+    real_model: object,
+    llamacpp_server: list[dict],
+) -> None:
+    """Three images on an 8,192 window cost 768 on Gemma 3, so they are sent,
+    though the history trim prices each at 1,400."""
+    set_sees(True)
+    set_props_n_ctx(8192)
+    workspace_id, _ = _seed(engine)
+    thread_id = await _open_thread(client, workspace_id)
+
+    status, _ = await send(client, thread_id, "what are these?", [picture()] * 3)
+
+    assert status == 200
+    assert len(image_parts(llamacpp_server[-1]["messages"][-1])) == 3
+
+
 async def test_bytes_that_are_not_an_image_are_refused(
     client: AsyncClient, engine: Engine, real_model: object, llamacpp_server: list[dict]
 ) -> None:

surfsense_local/backend/tests/unit/chat/test_budget.py

@@ -9,8 +9,10 @@
     DEFAULT_HISTORY_TOKENS,
     QUESTION_CHARS,
     QUESTION_TOKENS,
+    SMALLEST_IMAGE_TOKENS,
     answer_max_tokens,
     history_budget,
+    image_room,
 )
 from modules.chat.schemas import MessageCreate
 
@@ -55,3 +57,32 @@ def test_the_question_cannot_exceed_its_share() -> None:
     MessageCreate(text="x" * QUESTION_CHARS + "   ")  # whitespace is stripped first
     with pytest.raises(ValidationError):
         MessageCreate(text="x" * (QUESTION_CHARS + 1))
+
+
+def test_image_room_is_unknown_without_a_window() -> None:
+    """A remote endpoint that reports no window is never refused on its images."""
+    assert image_room(None, 0) is None
+
+
+def test_three_images_fit_an_8192_window_at_the_cheapest_projector() -> None:
+    """Gemma 3 spends 256 per image: three cost 768, which a floor window holds."""
+    assert image_room(8192, len("what are these?")) >= 3
+
+
+def test_image_room_is_a_lower_bound_not_the_history_price() -> None:
+    """Only the reserve and the turn's own text are taken: a refusal must be sure."""
+    assert (
+        image_room(2048, 0) == (2048 - ANSWER_RESERVE_TOKENS) // SMALLEST_IMAGE_TOKENS
+    )
+
+
+def test_the_turns_own_text_takes_room_from_its_images() -> None:
+    """Long excerpts leave fewer images room."""
+    excerpts = 400 * CHARS_PER_TOKEN
+
+    assert image_room(2048, excerpts) < image_room(2048, 0)
+
+
+def test_a_window_smaller_than_the_reserve_has_no_room() -> None:
+    """Floors at zero rather than going negative."""
+    assert image_room(512, 0) == 0