Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .agents/skills/translate/glossary.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,9 @@ SurfSense, Studio, llama.cpp, GGUF, Hugging Face, API, MCP, and every model and
| session log | セッションログ | 세션 로그 | 会话日志 | सेशन लॉग | Sitzungsprotokoll | registro de la sesión | journal de session | log da sessão | журнал сеанса |
| clipboard | クリップボード | 클립보드 | 剪贴板 | क्लिपबोर्ड | Zwischenablage | portapapeles | presse-papiers | área de transferência | буфер обмена |
| embedding (the model that turns the library into vectors) | 埋め込み | 임베딩 | 嵌入 | एम्बेडिंग | Embedding | embedding | embedding | embedding | эмбеддинг |
| sign in (to a ChatGPT account) | サインイン | 로그인 | 登录 | साइन इन | anmelden | iniciar sesión | se connecter | entrar | войти |
| plan (a ChatGPT plan, Plus or Pro) | プラン | 플랜 | 套餐 | प्लान | Plan | plan | forfait | plano | план |
| subscription (using a ChatGPT plan instead of a key) | サブスクリプション | 구독 | 订阅 | सब्सक्रिप्शन | Abo | suscripción | abonnement | assinatura | подписка |

## What the English means

Expand Down
2 changes: 1 addition & 1 deletion docs/adr/0015-openai-compatible-connections.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# ADR 0015: Remote models come through named OpenAI-compatible connections, discovered live

- **Status:** Accepted; the per-role selection is revised by the [model catalog proposal](../proposals/model-catalog.md), in progress: a selection is keyed by model type, with no roles
- **Status:** Accepted; the per-role selection is revised by the [model catalog proposal](../proposals/model-catalog.md), in progress: a selection is keyed by model type, with no roles; a connection signing in only with an optional bearer key and chatting only on `/chat/completions` is superseded for ChatGPT plans by [ADR 0038](0038-chatgpt-plans-sign-in-through-openai-not-codex.md)
- **Date:** 2026-09-10
- **Source:** [Connections plan L8–68](https://github.com/MODSetter/SurfSense/blob/431914fae066e0c42a38b1fdbbab64e8f92d3d00/plans/community-local/api/05b-openai-compatible-connections.md#L8-L68), [Connections plan L70–128](https://github.com/MODSetter/SurfSense/blob/431914fae066e0c42a38b1fdbbab64e8f92d3d00/plans/community-local/api/05b-openai-compatible-connections.md#L70-L128), [Connections plan L160–170](https://github.com/MODSetter/SurfSense/blob/431914fae066e0c42a38b1fdbbab64e8f92d3d00/plans/community-local/api/05b-openai-compatible-connections.md#L160-L170), [Connections plan L360–362](https://github.com/MODSetter/SurfSense/blob/431914fae066e0c42a38b1fdbbab64e8f92d3d00/plans/community-local/api/05b-openai-compatible-connections.md#L360-L362)

Expand Down
29 changes: 29 additions & 0 deletions docs/adr/0038-chatgpt-plans-sign-in-through-openai-not-codex.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
# ADR 0038: A ChatGPT plan is a connection signed in through OpenAI's open-source flow, not the Codex CLI's client

- **Status:** Accepted
- **Date:** 2026-10-02
- **Source:** [ChatGPT subscription proposal L22–34](https://github.com/MODSetter/SurfSense/blob/e9a1bd801fc9ac74a3c492fff40241701d6b9515/docs/proposals/chatgpt-subscription.md#L22-L34)

## Context

A ChatGPT Plus or Pro subscriber pays for model usage that a key connection ([ADR 0015](0015-openai-compatible-connections.md)) cannot reach: there is no API key, only an account. Two ways in exist. OpenAI's "Sign in with ChatGPT" for open-source apps issues each user a client on first sign-in and serves `POST https://api.openai.com/v1/responses`, but refuses `instructions`, `reasoning`, `text.format`, `max_output_tokens` and `tools`, and has no device-code flow. The Codex CLI's public client id reaches `chatgpt.com/backend-api/codex`, which takes all of those, lists richer model metadata and offers device code; Unsloth Studio ships it as its default. Anthropic's terms forbid the equivalent for Claude plans outright. The API and both workers each resolve the chat model, and OpenAI rotates refresh tokens. As built: [ChatGPT subscription](../architecture/chatgpt-subscription.md).

## Decision

- Sign in only through OpenAI's open-source flow: `client_id=dynamic_agent_client` on first sign-in, PKCE, a loopback redirect on `127.0.0.1` at any port, and the ID token verified against OpenAI's JWKS. The Codex CLI's client id and Codex's backend are not used: they present SurfSense as the Codex CLI on endpoints OpenAI never published for other apps, put users' accounts at the risk of OpenAI enforcing that, hide SurfSense from the per-app caps users set in ChatGPT, and can break with any Codex release.
- A plan is a `provider_connections` row with `auth_kind = 'chatgpt'`. `provider` stays `openai_compatible`, read as "remote connection", and revision `0023` adds `auth_kind`, `oauth_ciphertext` and `token_version` in place, because rebuilding the table cascades into every remote selection.
- Tokens are one Fernet blob under the install secret ([ADR 0018](0018-keychain-envelope-encryption.md)), in SQLite, so the workers reach them. `NULL` is signed out.
- A refresh holds SQLite's write lock across its HTTP call, so one process refreshes and the rest use its token: a rotated refresh token sent twice signs the account out.
- What a connection can fill is one backend rule, keyed by `auth_kind` ([`serves.py`](../../surfsense_local/backend/modules/llm/connections/serves.py)): a plan fills `text_gen` only. Selection, resolution and the model tests enforce it, and every picker reads it as `serves`.
- The request body is `{model, input, store: false, stream: true}`; a caller's token cap, reasoning switch and JSON schema are dropped.

## Consequences

- No client id is shipped in the repo, and nothing has to be applied for.
- Plan models have no reasoning control, structured output, tools or known context window, and there is no device code. A desktop app's browser is on the same machine, so the loopback redirect always works.
- A Codex route can still be added as its own `auth_kind`, opt-in and labelled unofficial, if the maintainers decide its capabilities are worth its risks; nothing here has to be reworked for it.
- A Claude subscription is not offered; Claude stays reachable with an API key.

## Where the code stands

The flow is tested against a fake built from OpenAI's documentation, not yet against a real ChatGPT account ([ChatGPT subscription](../architecture/chatgpt-subscription.md#known-gaps)).
3 changes: 2 additions & 1 deletion docs/adr/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ The Date is the day the decision was first written down. Source links are permal
| 0012 | [The GPU backend is Vulkan everywhere off Apple Silicon, with no CUDA payload](0012-vulkan-only-gpu-backend.md) | Accepted, for Docker images superseded by 0035 |
| 0013 | [Model fit comes from the allocator's view of one device, and only physics refuses an install](0013-fit-from-the-allocator.md) | Accepted |
| 0014 | [The model catalog is a curated offline manifest plus Hugging Face search, and SurfSense downloads the files itself](0014-two-tier-model-catalog.md) | Accepted, in part superseded by 0026, 0027 and the model catalog proposal |
| 0015 | [Remote models come through named OpenAI-compatible connections, discovered live](0015-openai-compatible-connections.md) | Accepted, in part revised by the model catalog proposal |
| 0015 | [Remote models come through named OpenAI-compatible connections, discovered live](0015-openai-compatible-connections.md) | Accepted, in part revised by the model catalog proposal, in part superseded by 0038 |
| 0016 | [The app sends no telemetry or crash reports](0016-no-telemetry.md) | Accepted |
| 0017 | [Every outbound destination is off until the user allows it](0017-egress-off-by-default.md) | Accepted, in part superseded by 0027 |
| 0018 | [Provider keys are encrypted with a per-install secret kept in the OS keychain](0018-keychain-envelope-encryption.md) | Accepted |
Expand All @@ -72,3 +72,4 @@ The Date is the day the decision was first written down. Source links are permal
| 0035 | [The desktop app's stack also ships as one Docker container, with Caddy as its only listener](0035-docker-compose-runs-the-desktop-stack.md) | Accepted |
| 0036 | [The index records which embedder built it, and the embedder is fixed once per install](0036-the-index-records-its-embedder.md) | Accepted |
| 0037 | [Curated embedders are a fourth engine in the local catalog, and embedding is a type, never a slot](0037-embedding-is-a-type-not-a-slot.md) | Accepted |
| 0038 | [A ChatGPT plan is a connection signed in through OpenAI's open-source flow, not the Codex CLI's client](0038-chatgpt-plans-sign-in-through-openai-not-codex.md) | Accepted |
2 changes: 1 addition & 1 deletion docs/architecture/chat.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@ The list handed to the generator is `[system, *history within budget, user]`.
- A turn that ends without any answer text is deleted, both halves, with any image file no other turn of the thread points at, and the stream goes from `error` straight to `[DONE]`, even if the model had reasoned. That holds whether generation failed or closed cleanly with nothing to say; the clean case is sent as `unknown`, because discarding the turn also removes the person's own message and silence would hide that. A turn that fails midway keeps its partial text.
- A client that hangs up mid-reply still commits the partial text and releases the model's in-use mark. Both cleanup paths are shielded from Starlette cancelling the response task on `http.disconnect`.
- The trace comes from the provider's `chat_deltas()`, which marks each chunk as answer or reasoning ([`connections.md`](connections.md#runtime)). It never reaches `resolve_citations()`, so a `[n]` inside it is neither rewritten nor stored as a citation.
- Errors are sorted by exception type, HTTP status and provider, and for a 400 by the body's `error.type` ([`errors.py`](../../surfsense_local/backend/modules/chat/errors.py)), into `provider_auth`, `provider_not_found`, `provider_rate_limited`, `provider_unavailable`, `model_cannot_run`, `context_too_long`, `network`, `timeout` and `unknown`, each with a plain-language English message. The frontend shows its own translated text per kind ([`chat-error-text.ts`](../../surfsense_local/frontend/src/features/chat/chat-error-text.ts)); a pre-stream `unknown` failure keeps an informative error message inside a translated sentence, and an unrecognized future kind falls back to the backend's message.
- Errors are sorted by exception type, HTTP status and provider, and for a 400 by the body's `error.type` ([`errors.py`](../../surfsense_local/backend/modules/chat/errors.py)), into `provider_auth`, `provider_not_found`, `provider_rate_limited`, `provider_unavailable`, `model_cannot_run`, `context_too_long`, `subscription_sign_in`, `subscription_limit`, `network`, `timeout` and `unknown`, each with a plain-language English message. The two `subscription_` kinds come from a ChatGPT connection: its account has to sign in again, or its plan's usage limit is reached, which no retry fixes ([`chatgpt-subscription.md`](chatgpt-subscription.md)). The frontend shows its own translated text per kind ([`chat-error-text.ts`](../../surfsense_local/frontend/src/features/chat/chat-error-text.ts)); a pre-stream `unknown` failure keeps an informative error message inside a translated sentence, and an unrecognized future kind falls back to the backend's message.

## Citations

Expand Down
Loading
Loading