Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions docs/architecture/chat.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ A chat thread belongs to a workspace. Each user message retrieves its own contex

| Method | Path | Does |
|---|---|---|
| `POST` | `/workspaces/{workspace_id}/chat/threads` | open a thread, with an optional `source_scope`, `mode`, `basic` or `agentic`, and `remember`, which makes the mode the model's default ([Agent threads](#agent-threads)); the title defaults to "New chat" |
| `POST` | `/workspaces/{workspace_id}/chat/threads` | open a thread, with an optional `source_scope` and `mode`, `basic` or `agentic` ([Agent threads](#agent-threads)); the title defaults to "New chat" |
| `GET` | `/workspaces/{workspace_id}/chat/threads` | list, newest first, each with `running` while a reply is being generated for it, and `run_state`, `{state, position}`: `queued` with its place in line, `running`, or `needs-approval` while an agent turn waits on the user; `null` when nothing runs |
| `PATCH` | `/chat/threads/{thread_id}` | rename, 1 to 200 characters |
| `DELETE` | `/chat/threads/{thread_id}` | delete; a running reply is stopped first, waiting up to 5 seconds, then its messages cascade, its artifacts stay with the thread cleared, and its images' folder goes after the commit |
Expand Down Expand Up @@ -262,12 +262,12 @@ Interface strings are `dictation_<surface>_<purpose>`, and a server code is `dic
- The composer follows assistant-ui's keyboard behaviour.
- While the turn waits, the thinking header's `role="status"` region says "Thinking" or the reading quarter. A second region (`reply-announcer.tsx`) stays silent until it says "Reply finished" once, for a reply it watched run to completion. It is keyed by message, because messages render by index and an instance would otherwise watch one thread's reply and announce another's. Tokens are never fed to it, since a polite region read on every delta would interrupt without pause; a reply loaded from history, a turn that ended with no answer, and one that failed or was stopped after text arrived say nothing; its alert, or the stop, already does. A stopped reply is marked `incomplete` with reason `cancelled` by the runtime ([`use-chat-runtime.ts`](../../surfsense_local/frontend/src/features/chat/use-chat-runtime.ts)), since assistant-ui would otherwise call it complete once the run ends. A queued reply's header region says "Waiting for another reply" with its place. The Chats button's count is in its accessible name ("Chats, 2 replies being written"), and an unread row carries "New reply" as screen-reader text.
- The conversation has a heading, the thread's title, hidden visually because the visible title is the rename button. Focus does not move to it: on every thread switch the composer keeps the focus assistant-ui gives it, so someone can switch and type, and the composer is described by the heading (`aria-describedby`), so a screen reader names the conversation it writes into. A new chat has no heading until it is a thread.
- The startup loader and the typed-in thread title respect `prefers-reduced-motion`.
- The startup fade and the typed-in thread title follow Settings › General's Reduce motion, which matches the system by default ([overview](overview.md#frontend)).
- The mic carries `aria-pressed` and a name that follows its state: "Start voice input", "Stop voice input", or "Set up voice input"; its tooltip is visual only. While voice input runs, a `role="status"` line in the strip says "Starting the microphone…", "Listening", "Loading the speech recognition model…", "Transcribing…" or the 30-second warning; a problem is a `role="alert"`, the two limits a `role="status"`. Under `prefers-reduced-motion` the level bars give way to the still dot beside them.

## Agent threads

How the agent runs is in [agent](agent.md). A thread opened with `mode: "agentic"` is the agent's, and one opened with `mode: "basic"` the chat's; with no `mode` it takes the selected model's default ([model capabilities](model-capabilities.md#the-modes), [agent](agent.md#which-threads-get-it)). Agentic asked for and kept out by a technical gate is refused with `409`, `detail.code` `no_model`, `agent_not_installed`, `tool_calls_unsupported` or `window_below_floor`, and no thread is made. Opening an agent thread starts opencode if it is not running, waits until it serves SurfSense's configuration, and opens an opencode session under the thread's title; the session's id and the engine, `agent_engine` `opencode`, are stored on the thread, and `ThreadRead` reports `uses_agent`, true when the thread has an engine. When opencode does not become ready, a thread that asked for Agentic is removed and the request answers `503`, `detail.code` `agent_unavailable`; one that asked for nothing opens as a chat. A mode sent with `remember: true`, which the composer sends only for a mode the user picked, is remembered as the model's default. Which engine runs a thread's turns, and how it moves when the selected model changes, is in [agent](agent.md#two-engines).
How the agent runs is in [agent](agent.md). A thread opened with `mode: "agentic"` is the agent's, and one opened with `mode: "basic"` the chat's; with no `mode` it takes the selected model's default ([model capabilities](model-capabilities.md#the-modes), [agent](agent.md#which-threads-get-it)). Agentic asked for and kept out by a technical gate is refused with `409`, `detail.code` `no_model`, `agent_not_installed`, `tool_calls_unsupported` or `window_below_floor`, and no thread is made. Opening an agent thread starts opencode if it is not running, waits until it serves SurfSense's configuration, and opens an opencode session under the thread's title; the session's id and the engine, `agent_engine` `opencode`, are stored on the thread, and `ThreadRead` reports `uses_agent`, true when the thread has an engine. When opencode does not become ready, a thread that asked for Agentic is removed and the request answers `503`, `detail.code` `agent_unavailable`; one that asked for nothing opens as a chat. Opening a thread never changes the model's default; the composer's switch saves a pick itself ([model capabilities](model-capabilities.md#the-modes)). Which engine runs a thread's turns, and how it moves when the selected model changes, is in [agent](agent.md#two-engines).

The same routes then reach the agent ([`modules/agent/agent_threads/`](../../surfsense_local/backend/modules/agent/agent_threads/)):

Expand Down
2 changes: 1 addition & 1 deletion docs/architecture/chatgpt-subscription.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ The OpenAI catalog records the plan's models as calling tools, so Agentic mode i

## Frontend

Every model list reads `serves` rather than deciding: the Settings sections, their empty states and onboarding's server steps leave a ChatGPT connection out of every section but chat. A new ChatGPT connection is made only through **Use your ChatGPT plan**, below. The connection form lists **ChatGPT subscription** only while editing a ChatGPT connection, so a server's provider list never offers it. Editing one hides the URL and key fields, shows the account, and offers **Sign in again**, with **Open the sign-in page again** and Cancel while it waits, **Sign out**, and **Save changes** for a new name. Its provider cannot be changed. The model groups show the account's email in place of the URL.
Every model list reads `serves` rather than deciding: the Settings sections and their empty states leave a ChatGPT connection out of every section but chat. A new ChatGPT connection is made only through **Use your ChatGPT plan**, below. The connection form lists **ChatGPT subscription** only while editing a ChatGPT connection, so a server's provider list never offers it. Editing one hides the URL and key fields, shows the account, and offers **Sign in again**, with **Open the sign-in page again** and Cancel while it waits, **Sign out**, and **Save changes** for a new name. Its provider cannot be changed. The model groups show the account's email in place of the URL.

**Use your ChatGPT plan** sits below **Use a server** on Settings' **Add model** page and on onboarding's text step, wherever the sign-in's `serves` includes the slot ([`chatgpt-option.tsx`](../../surfsense_local/frontend/src/features/models/remote/connections/chatgpt/chatgpt-option.tsx)). One click signs in as **ChatGPT subscription**, with the dialog's progress in the row and Cancel in place of the button, then opens the new connection's models as connecting a server does. A plan already connected reads "Connected as" its email and offers **Show models** instead.

Expand Down
2 changes: 1 addition & 1 deletion docs/architecture/connections.md
Original file line number Diff line number Diff line change
Expand Up @@ -180,7 +180,7 @@ Keys are protected by envelope encryption ([ADR 0018](../adr/0018-keychain-envel

- Each model section in Settings shows, as a group under the local models, every connection whose `serves` includes that section's slot. `serves` comes from one backend rule ([`serves.py`](../../surfsense_local/backend/modules/llm/connections/serves.py)): every slot but `speech_to_text` for a key connection, `text_gen` alone for a ChatGPT one. The section's empty state counts only those connections. A group loads its models only when opened, so a slow or failed endpoint does not hold up the others, and lists only those whose `selectable_for` includes that section's slot; the model in use, if it comes from that server, stays shown above the list whether the group is open or closed. Open, its list scrolls inside its own capped-height area rather than lengthening the page. A model's row shows its type chips, and a Vision chip when `reads_images` is true. A model is assigned after an optional test; an exact ID the listing lacks can be typed in. A new connection is added from **Use a server** on the section's **Add model** page. Adding and editing a connection happen in a dialog over the page. Saving a new connection returns to the list with its group open on its models, since choosing one is why it was added; saving an edit only refreshes.
- Edit and Disconnect sit on each group. Edit opens the same form in that dialog. A connection can fill every slot but `speech_to_text`, so Disconnect names each model it will clear, Chat, Image or both, whichever section it is disconnected from.
- Onboarding's model steps use the same groups and the same dialog; with nothing connected yet, their **Connect** opens the dialog directly, and saving opens the server page on the new connection's models.
- Onboarding's chat step uses the same groups and the same dialog; with nothing connected yet, its **Connect** opens the dialog directly, and saving opens the server page on the new connection's models.
- The connection form picks a provider from the remote manifest, through `GET /llm/catalog/remote`, or "Local or custom server". A ready provider fills its URL, which stays editable so a proxy in front of it still works; a provider that needs account details asks for each field and builds the URL from its template; a provider that needs a URL leaves it to the user; an unreachable one is listed, disabled, with its reason. A provider that takes no key, a loopback server, hides the key field. "Local or custom server" leaves the URL to the user, since its port is whatever its owner set, with `http://localhost:11434/v1` as placeholder text only; a loopback server the manifest lists, such as LM Studio, fills the manifest's URL like any ready provider. The save sends `catalog_provider`.

## Known gaps
Expand Down
2 changes: 1 addition & 1 deletion docs/architecture/local-models/catalog.md
Original file line number Diff line number Diff line change
Expand Up @@ -506,7 +506,7 @@ counted rather than decoded

`POST /llm/installs` takes `{"catalog_id": "...", "select": true}`, and
optionally the `model_type` that `select` fills, the engine's first type when
absent, so onboarding's editing step installs straight into `image_edit`. Nothing
absent, so a speech model's download installs straight into `speech_to_text`. Nothing
else: no repo, file, URL, path or quantization, so the renderer cannot name an
arbitrary download. A curated build's id is minted once per process and keyed on
its repo and weights path; a searched build's is a ticket holding the whole build
Expand Down
32 changes: 15 additions & 17 deletions docs/architecture/local-models/selection.md
Original file line number Diff line number Diff line change
Expand Up @@ -202,13 +202,15 @@ route. Only the onboarding page's last step does, once a chat model is
persisted. Once the marker exists the app never shows onboarding again, and a
missing selection is fixed from Settings' Chat section.

The onboarding page opens on a welcome screen, then six steps: chat, image, image editing, audio, video and search model. The welcome is not counted as a step, but it is part of onboarding and gated by the same marker, so it is never shown again once onboarding is done.
The onboarding page opens on a welcome screen, then the privacy choices ([usage statistics](../usage-statistics.md#consent)), then two model steps: chat and search model. The welcome is not counted as a step, but it is part of onboarding and gated by the same marker, so it is never shown again once onboarding is done. Image, image editing, audio and video models are not asked for: they are chosen in Settings, and the feature that needs one says so.

The embedding step comes last and is not a slot, but it is the same component
as the model steps below, with one more entry in their tables
as the chat step below, with its own entry in their tables
([`model-step/`](../../../surfsense_local/frontend/src/features/onboarding/model-step/)).
What a step does on Use is its own hook: a slot's saves the selection, and the
embedding step's ([`use-embedding-step.ts`](../../../surfsense_local/frontend/src/features/onboarding/model-step/kinds/use-embedding-step.ts))
What a step does on Use is its own hook: the chat step's
([`use-chat-step.ts`](../../../surfsense_local/frontend/src/features/onboarding/model-step/kinds/use-chat-step.ts))
saves the selection, and the embedding step's
([`use-embedding-step.ts`](../../../surfsense_local/frontend/src/features/onboarding/model-step/kinds/use-embedding-step.ts))
only marks a choice, In use until another is used, bge-small by default and again
if the chosen one is deleted. Its downloads install with `select: false`; a
Hugging Face pick is labelled not tested by SurfSense; the bundled bge-small has
Expand All @@ -220,21 +222,17 @@ Finish sends the choice with the call that ends onboarding; Skip and finish, or
Finish with the choice untouched, sends none, which means bge-small
([embedding](../embedding.md)).

The model steps are one component for any slot
The model steps are one component
([`frontend/src/features/onboarding/model-step/`](../../../surfsense_local/frontend/src/features/onboarding/model-step/)),
built on the same hooks as Settings but with its own screens. Each lists every
model this computer can run at once: in the chat step, models downloaded from Hugging Face first, then the curated list with its starred row first, with
built on the same hooks as Settings but with its own screens. The chat step lists every
model this computer can run at once: models downloaded from Hugging Face first, then the curated list with its starred row first, with
Download, Use and Delete as in Settings, and no Delete on a model the app ships; a download's progress shows under its
row and never moves the page. The chat step also offers Settings' Hugging Face
search, closed until asked for; the image, image editing, audio and video steps have none, since
sd.cpp's and audio.cpp's models are the few the catalog ships. The image editing step lists first the model chosen for images earlier when it edits too, so FLUX.2 klein is one Use away, and its downloads fill `image_edit`. A server sits one line below the list and names
any connected earlier. Once the slot has a model, the footer names it beside
Continue. Onboarding installs with `select: true`, so a download is also the
choice; Settings installs with `select: false`. The chat step's Continue is
enabled only once a chat model is selected, local or from a server. The image,
image editing, audio and video steps are optional, and each enables its Continue or Finish only once
its slot has a model. Each step's Skip and Continue move on to the next, image
to image editing to audio to video. The last step's buttons read Skip and finish and Finish, and both post the marker.
row and never moves the page. It also offers Settings' Hugging Face
search, closed until asked for. A server sits one line below the list and names
any connected earlier. Once a chat model is chosen, the footer names it beside
Continue. The chat step installs with `select: true`, so a download is also the
choice; Settings installs with `select: false`. Its Continue is
enabled only once a chat model is selected, local or from a server, and it has no Skip. The embedding step's buttons read Skip and finish and Finish, and both post the marker.

## Resolution: local and remote

Expand Down
Loading
Loading