fix(local): cap a chat question at its share of the window

docs/architecture/chat.md

@@ -49,7 +49,7 @@ The list handed to the generator is `[system, *history within budget, user]`.
 
 - **History** is the thread's stored turns in `created_at` order, flattened to role and text. Stored citations and reasoning are for the UI, not the model.
 - **Sliding window.** The system message and the new user message are pinned; the most recent prior turns that fit the history budget are kept and older ones dropped.
-- **Budget** ([`budget.py`](../../surfsense_local/backend/modules/chat/budget.py)). One context window is shared by the system prompt (priced at 400 tokens), the excerpts (2,400: five hits of up to 480 tokens), the question (1,024) and a 1,024-token answer reserve, which is claimed first because llama.cpp stops a reply wherever the window runs out. History gets what remains of the model's window, floored at zero, so a narrow window means a shorter history rather than an overflowing turn. llama.cpp reports the window it actually allocated; an OpenAI-compatible endpoint reports none, and then history gets 3,000 tokens and `max_tokens` is left to the endpoint.
+- **Budget** ([`budget.py`](../../surfsense_local/backend/modules/chat/budget.py)). One context window is shared by the system prompt (priced at 400 tokens), the excerpts (2,400: five hits of up to 480 tokens), the question (1,024, which `MessageText` enforces at the wire as 4,096 characters at the same four-characters-a-token estimate, refusing more with a 422 before any model is resolved) and a 1,024-token answer reserve, which is claimed first because llama.cpp stops a reply wherever the window runs out. History gets what remains of the model's window, floored at zero, so a narrow window means a shorter history rather than an overflowing turn. llama.cpp reports the window it actually allocated; an OpenAI-compatible endpoint reports none, and then history gets 3,000 tokens and `max_tokens` is left to the endpoint.
 - Each prior turn is priced by the local runtime's own tokenizer when it answers, and by `len(text) // 4` otherwise.
 - **Retrieval query** is the new user message.
 
@@ -126,7 +126,6 @@ The list handed to the generator is `[system, *history within budget, user]`.
 
 ## Known gaps
 
-- `budget.py` prices the question at 1,024 tokens and says `MessageText` enforces that, but `MessageText` has no length limit, so a long question can push a turn past the model's window.
 - No live region announces streamed text, and focus does not move to the conversation heading after a thread switch; the dashboard design asks for both.
 - No test covers a client disconnecting mid-reply. The assistant's text is written only when generation ends, inside the stream, so whether a disconnected reply is kept is unverified.
 - A thinking model spends the 1,024-token answer cap on its reasoning too: `max_tokens` counts what goes to `reasoning_content`, as the title measurement in [`local-models/runtime.md`](local-models/runtime.md#turning-thinking-off) shows, so on the local runtime a long think can cut the answer short or leave it empty. The trace now shows, so an empty answer is no longer unexplained, but how often it happens is unmeasured.

docs/architecture/local-models/fit.md

@@ -597,7 +597,7 @@ model's real window:
 | Answer reserve | 1024 | Reserved first. llama.cpp stops a reply wherever the window runs out, so spending this on history is how a good answer gets cut off mid-sentence with no error at all. |
 | System prompt | 400 | A rendered prompt plus the grounding header at the largest tier. Approximate. |
 | Excerpts | 2400 | Retrieval already caps chunks at 480 tokens and 5 hits, so this holds before retrieval runs. |
-| Question | 1024 | The user's message, so one turn cannot claim the whole window. A share, not a limit (see Known gaps). |
+| Question | 1024 | The user's message, so one turn cannot claim the whole window. Enforced at the wire: `MessageText` refuses more than 4,096 characters, the share at the four-characters-a-token estimate. |
 
 History gets what is left, `max(0, n_ctx - 4848)`: 3,344 tokens at the 8,192
 floor, growing with the window, so a narrower window means a shorter history
@@ -706,7 +706,6 @@ holds the badge to the load, and `tests/unit/chat/test_budget.py` the chat budge
 ## Known gaps
 
 - `Device.reports_live_memory`, which tells a live reading from a restated total, is never read, so a device whose `free` merely restates `total`, as WSL2 and macOS do, is trusted as if it had reported free memory. `system_memory.available_bytes()` has no test.
-- The question's 1,024-token share is not enforced: `MessageText` in `modules/chat/schemas.py` sets no maximum length, so a longer question can push a turn with a full history past the window.
 - The `q8_0` preference is unmeasured: nobody has timed a resident `q8_0` cache against a small `f16` spill on prompt rate, and published figures report quantized caches generating materially slower.
 - The light-spill boundary may be tight: `LIGHT_SPILL` ends at 0.25, which puts Qwen3 8B on a 6 GB RTX 3050 (0.33 by layers) out of the recommendation although its owner runs it without noticeable lag; one prefill and decode measurement of that configuration against the resident 4B settles it, and moving the boundary moves the badge and the star together.
 - The weights term is unconfirmed against the pinned files: the measured model buffers exceeded them (2,680 MiB against 2,382 for the 4B, 1,294 against 1,056 for the 1.7B), probably because those runs used another publisher's build, and if not, the largest term is under-estimated by 12 to 22%.

surfsense_local/backend/modules/chat/budget.py

@@ -31,11 +31,20 @@
 # measured so the budget holds even before retrieval runs.
 EXCERPTS_TOKENS = 2400
 
+# The characters-to-tokens estimate history.py prices turns with when the
+# model's own tokenizer is not asked. Dense scripts run richer than this, so
+# the cap below is loose for them, never tight.
+CHARS_PER_TOKEN = 4
+
 # A pasted question can be long; capped so one turn cannot claim the whole
-# window and leave nothing for history or the answer. MessageText enforces
-# this at the wire (modules/chat/schemas.py).
+# window and leave nothing for history or the answer.
 QUESTION_TOKENS = 1024
 
+# What MessageText refuses past, at the wire (modules/chat/schemas.py). One
+# global cap rather than one per window: it runs before any model is resolved,
+# and the budget prices the question at the same 1,024 for every window.
+QUESTION_CHARS = QUESTION_TOKENS * CHARS_PER_TOKEN
+
 # Today's number, kept as the fallback for a window this module cannot see:
 # unchanged so a remote model with no reported `n_ctx` behaves exactly as it
 # did before this budget existed.

surfsense_local/backend/modules/chat/history.py

@@ -1,6 +1,6 @@
 from collections.abc import Awaitable, Callable, Sequence
 
-from modules.chat.budget import DEFAULT_HISTORY_TOKENS
+from modules.chat.budget import CHARS_PER_TOKEN, DEFAULT_HISTORY_TOKENS
 from modules.chat.models import ChatMessage
 from modules.llm.providers.types import Message
 
@@ -65,4 +65,4 @@ async def _cost(text: str, token_count: TokenCounter | None) -> int:
 def _tokens(text: str) -> int:
     # ponytail: ~4 chars per token dodges loading the model's tokenizer. Ceiling:
     # fine for a soft trim; swap in the real count if the runtime starts truncating.
-    return len(text) // 4
+    return len(text) // CHARS_PER_TOKEN

surfsense_local/backend/modules/chat/schemas.py

@@ -3,12 +3,18 @@
 
 from pydantic import BaseModel, ConfigDict, Field, StringConstraints
 
+from modules.chat.budget import QUESTION_CHARS
 from modules.chat.models import MessageRole
 
 ThreadTitle = Annotated[
     str, StringConstraints(strip_whitespace=True, min_length=1, max_length=200)
 ]
-MessageText = Annotated[str, StringConstraints(strip_whitespace=True, min_length=1)]
+# The question's share of the window (budget.py), refused here before any
+# model is resolved or retrieval runs.
+MessageText = Annotated[
+    str,
+    StringConstraints(strip_whitespace=True, min_length=1, max_length=QUESTION_CHARS),
+]
 DocumentId = Annotated[int, Field(gt=0)]
 
 

surfsense_local/backend/tests/integration/chat/test_chat.py

@@ -6,7 +6,7 @@
 from httpx import AsyncClient
 from sqlalchemy import Engine
 
-from modules.chat.budget import ANSWER_RESERVE_TOKENS
+from modules.chat.budget import ANSWER_RESERVE_TOKENS, QUESTION_CHARS
 from modules.documents.models import Document, DocumentStatus, DocumentType
 from modules.llm.model_type import ModelType
 from modules.llm.models import SelectedModel
@@ -403,6 +403,22 @@ async def test_a_thread_with_no_model_selected_is_a_409(client: AsyncClient) ->
     assert reply.status_code == 409
 
 
+async def test_a_question_past_its_share_is_refused_at_the_wire(
+    client: AsyncClient,
+) -> None:
+    """Refused before a model is resolved or retrieval runs: with no model
+    selected, a message that got past the schema would be the 409 above."""
+    workspace = (await client.post("/workspaces", json={"name": "w"})).json()
+    thread_id = await _open_thread(client, workspace["id"])
+
+    reply = await client.post(
+        f"/chat/threads/{thread_id}/messages",
+        json={"text": "x" * (QUESTION_CHARS + 1)},
+    )
+
+    assert reply.status_code == 422
+
+
 async def test_missing_embedding_assets_are_an_actionable_503(
     client: AsyncClient, engine: Engine
 ) -> None:

surfsense_local/backend/tests/unit/chat/test_budget.py

@@ -1,13 +1,18 @@
 """How a turn's fixed parts, the answer, and history share one context window."""
 
 import pytest
+from pydantic import ValidationError
 
 from modules.chat.budget import (
     ANSWER_RESERVE_TOKENS,
+    CHARS_PER_TOKEN,
     DEFAULT_HISTORY_TOKENS,
+    QUESTION_CHARS,
+    QUESTION_TOKENS,
     answer_max_tokens,
     history_budget,
 )
+from modules.chat.schemas import MessageCreate
 
 pytestmark = pytest.mark.unit
 
@@ -39,3 +44,14 @@ def test_the_answer_reserve_is_capped_only_when_the_window_is_known() -> None:
     larger would truncate replies for a limit that does not apply there."""
     assert answer_max_tokens(None) is None
     assert answer_max_tokens(16384) == ANSWER_RESERVE_TOKENS
+
+
+def test_the_question_cannot_exceed_its_share() -> None:
+    """The question's 1,024 tokens are a cap the wire enforces, not a share it
+    hopes for: priced by the same characters-per-token estimate history uses,
+    a longer message is refused before any model or retrieval is touched."""
+    assert QUESTION_CHARS // CHARS_PER_TOKEN == QUESTION_TOKENS
+    MessageCreate(text="x" * QUESTION_CHARS)
+    MessageCreate(text="x" * QUESTION_CHARS + "   ")  # whitespace is stripped first
+    with pytest.raises(ValidationError):
+        MessageCreate(text="x" * (QUESTION_CHARS + 1))

surfsense_local/frontend/src/features/chat/chat-composer.test.tsx

@@ -0,0 +1,62 @@
+import { afterEach, describe, expect, it, vi } from "vitest"
+import { cleanup, screen } from "@testing-library/react"
+import userEvent from "@testing-library/user-event"
+import { AssistantRuntimeProvider, useLocalRuntime } from "@assistant-ui/react"
+
+import { render } from "@/test-utils"
+
+import { ChatComposer } from "./chat-composer"
+import { QUESTION_MAX_CHARS } from "./question-limit"
+
+const MODEL = {
+  model_type: "text_gen" as const,
+  provider: "llamacpp" as const,
+  connection_id: null,
+  name: "Qwen3-1.7B-Q4_K_M",
+  updated_at: "2026-09-05T00:00:00Z",
+}
+
+function Harness() {
+  const runtime = useLocalRuntime({ run: async () => ({ content: [] }) })
+  return (
+    <AssistantRuntimeProvider runtime={runtime}>
+      <ChatComposer
+        placement="center"
+        model={MODEL}
+        sourceCount={0}
+        isRunning={false}
+        isUploading={false}
+        providerAvailable
+        onModelSetup={() => undefined}
+        onModelSelected={() => undefined}
+        onUpload={() => undefined}
+      />
+    </AssistantRuntimeProvider>
+  )
+}
+
+afterEach(() => {
+  cleanup()
+  vi.unstubAllGlobals()
+})
+
+describe("chat composer", () => {
+  it("stops a question at its share of the window and says so", async () => {
+    vi.stubGlobal(
+      "fetch",
+      vi.fn(async () => Response.json([]))
+    )
+    const user = userEvent.setup()
+    render(<Harness />)
+
+    const input = screen.getByRole<HTMLTextAreaElement>("textbox", {
+      name: "Message",
+    })
+    expect(screen.queryByRole("status")).toBeNull()
+    await user.click(input)
+    await user.paste("x".repeat(QUESTION_MAX_CHARS + 1))
+
+    expect(input.value).toHaveLength(QUESTION_MAX_CHARS)
+    expect(screen.getByRole("status").textContent).toContain("4,096")
+  })
+})

surfsense_local/frontend/src/features/chat/chat-composer.tsx

@@ -1,5 +1,5 @@
 import { useRef, type ChangeEvent, type ReactNode } from "react"
-import { ComposerPrimitive } from "@assistant-ui/react"
+import { ComposerPrimitive, useAuiState } from "@assistant-ui/react"
 
 import { Button } from "@/components/ui/button"
 import { ArrowUp02Icon, CircleStopIcon, PlusIcon } from "@/components/ui/icons"
@@ -14,6 +14,7 @@ import { cn } from "@/lib/utils"
 import { intl } from "@/i18n/intl"
 
 import { ModelPicker, modelControlButtonClassName } from "./model-picker"
+import { QUESTION_MAX_CHARS } from "./question-limit"
 
 function ModelControl({
   model,
@@ -205,6 +206,20 @@ export function ChatComposer({
   onModelSelected: (selection: ModelSelection) => void
   onUpload: (files: File[]) => void
 }) {
+  // Said only once the cap is reached: that is the moment typing, or the tail
+  // of a paste, stops landing, and the one moment it needs explaining.
+  const atLimit = useAuiState(
+    ({ composer }) => composer.text.length >= QUESTION_MAX_CHARS
+  )
+  const limitNotice = atLimit
+    ? intl.formatMessage(
+        {
+          id: "chat_composer_length_limit_status",
+          defaultMessage: "Messages are limited to {max, number} characters.",
+        },
+        { max: QUESTION_MAX_CHARS }
+      )
+    : null
   return (
     <div
       className="relative mx-auto w-full max-w-xl"
@@ -262,6 +277,7 @@ export function ChatComposer({
           }
           submitMode="enter"
           rows={1}
+          maxLength={QUESTION_MAX_CHARS}
           aria-label={intl.formatMessage({
             id: "chat_composer_message_aria",
             defaultMessage: "Message",
@@ -291,22 +307,34 @@ export function ChatComposer({
           </>
         )}
       </ComposerPrimitive.Root>
+      {placement === "center" && limitNotice ? (
+        <p
+          role="status"
+          className="mt-1 px-2 text-[11px] text-muted-foreground select-none"
+        >
+          {limitNotice}
+        </p>
+      ) : null}
       {placement === "bottom" ? (
         <div className="mt-1 flex min-h-7 items-center justify-between gap-3 px-2">
           {/* Keeps its line and leaves the model name what is left, never less
           than 7rem, so a long translation truncates the name before wrapping. */}
           <p className="max-w-[calc(100%-7rem)] shrink-0 text-left text-[11px] text-muted-foreground select-none">
-            {!model || providerAvailable
-              ? intl.formatMessage({
-                  id: "chat_composer_disclaimer_body",
-                  defaultMessage:
-                    "SurfSense can make mistakes. Check important answers.",
-                })
-              : intl.formatMessage({
-                  id: "chat_composer_provider_offline_body",
-                  defaultMessage:
-                    "Historical chats remain available while the provider is offline.",
-                })}
+            {limitNotice ? (
+              <span role="status">{limitNotice}</span>
+            ) : !model || providerAvailable ? (
+              intl.formatMessage({
+                id: "chat_composer_disclaimer_body",
+                defaultMessage:
+                  "SurfSense can make mistakes. Check important answers.",
+              })
+            ) : (
+              intl.formatMessage({
+                id: "chat_composer_provider_offline_body",
+                defaultMessage:
+                  "Historical chats remain available while the provider is offline.",
+              })
+            )}
           </p>
           <ModelControl
             model={model}

surfsense_local/frontend/src/features/chat/question-limit.ts

@@ -0,0 +1,5 @@
+// Mirrors QUESTION_CHARS in surfsense_local/backend/modules/chat/budget.py:
+// the question's 1,024-token share of the window at four characters a token.
+// The wire refuses more with a 422; the composer stops first, so nobody finds
+// out on send.
+export const QUESTION_MAX_CHARS = 4096

surfsense_local/frontend/translations/de.json

@@ -62,6 +62,7 @@
   "chat_composer_disclaimer_body": "SurfSense kann Fehler machen. Überprüfe wichtige Antworten.",
   "chat_composer_egress_off_placeholder": "Erlaube das Senden an {host}, um zu chatten",
   "chat_composer_follow_up_placeholder": "Nachfrage zu dieser Antwort stellen",
+  "chat_composer_length_limit_status": "Nachrichten sind auf {max, number} Zeichen begrenzt.",
   "chat_composer_message_aria": "Nachricht",
   "chat_composer_provider_offline_body": "Frühere Chats bleiben verfügbar, solange der Anbieter offline ist.",
   "chat_composer_provider_offline_placeholder": "Verbinde deinen Modellanbieter erneut, um zu senden",

surfsense_local/frontend/translations/en.json

@@ -62,6 +62,7 @@
   "chat_composer_disclaimer_body": "SurfSense can make mistakes. Check important answers.",
   "chat_composer_egress_off_placeholder": "Allow sending to {host} to chat",
   "chat_composer_follow_up_placeholder": "Follow up on this answer",
+  "chat_composer_length_limit_status": "Messages are limited to {max, number} characters.",
   "chat_composer_message_aria": "Message",
   "chat_composer_provider_offline_body": "Historical chats remain available while the provider is offline.",
   "chat_composer_provider_offline_placeholder": "Reconnect your model provider to send",

surfsense_local/frontend/translations/es.json

@@ -62,6 +62,7 @@
   "chat_composer_disclaimer_body": "SurfSense puede equivocarse. Comprueba las respuestas importantes.",
   "chat_composer_egress_off_placeholder": "Permite el envío a {host} para chatear",
   "chat_composer_follow_up_placeholder": "Pregunta más sobre esta respuesta",
+  "chat_composer_length_limit_status": "Los mensajes están limitados a {max, number} caracteres.",
   "chat_composer_message_aria": "Mensaje",
   "chat_composer_provider_offline_body": "Los chats anteriores siguen disponibles mientras el proveedor esté sin conexión.",
   "chat_composer_provider_offline_placeholder": "Vuelve a conectar tu proveedor de modelos para enviar",

surfsense_local/frontend/translations/fr.json

@@ -62,6 +62,7 @@
   "chat_composer_disclaimer_body": "SurfSense peut faire des erreurs. Vérifiez les réponses importantes.",
   "chat_composer_egress_off_placeholder": "Autorisez l’envoi vers {host} pour discuter",
   "chat_composer_follow_up_placeholder": "Posez une question sur cette réponse",
+  "chat_composer_length_limit_status": "Les messages sont limités à {max, number} caractères.",
   "chat_composer_message_aria": "Message",
   "chat_composer_provider_offline_body": "Les chats passés restent accessibles pendant que le fournisseur est hors ligne.",
   "chat_composer_provider_offline_placeholder": "Reconnectez votre fournisseur de modèle pour envoyer",

surfsense_local/frontend/translations/hi.json

@@ -62,6 +62,7 @@
   "chat_composer_disclaimer_body": "SurfSense से गलतियां हो सकती हैं। ज़रूरी जवाब जांच लें।",
   "chat_composer_egress_off_placeholder": "चैट करने के लिए {host} को भेजने की अनुमति दें",
   "chat_composer_follow_up_placeholder": "इस जवाब पर आगे पूछें",
+  "chat_composer_length_limit_status": "संदेश {max, number} अक्षरों तक सीमित हैं।",
   "chat_composer_message_aria": "संदेश",
   "chat_composer_provider_offline_body": "प्रोवाइडर ऑफ़लाइन रहने पर भी पुरानी चैट उपलब्ध रहती हैं।",
   "chat_composer_provider_offline_placeholder": "भेजने के लिए अपने मॉडल प्रोवाइडर से फिर से कनेक्ट करें",