diff --git a/docs/architecture/data-model.md b/docs/architecture/data-model.md index 798b0ffaf3..0ff3184e7b 100644 --- a/docs/architecture/data-model.md +++ b/docs/architecture/data-model.md @@ -424,5 +424,5 @@ erDiagram ## Known gaps - Deleting a folder is permanent: the Trash is designed, not built ([proposal](../proposals/file-agent/01-sources-and-folders.md)). -- The snapshot needs free disk about the size of `surfsense.db`, and nothing checks for it first: when the copy fails, the API does not start and the app closes with no message ([overview](overview.md#known-gaps)). +- The snapshot needs free disk about the size of `surfsense.db`, and nothing checks for it first: when the copy fails, the API does not start and the app quits after a dialog naming the failure ([overview](overview.md)). - A release cannot open a database a later release migrated, since its migrations do not know the newer revision, so its API does not start. Going back a version means restoring the snapshot by hand, as going from 2.2.0 to 2.1.0 takes `backups/0023-0027.db`. diff --git a/docs/architecture/overview.md b/docs/architecture/overview.md index 4bbd111110..ddb9e164c5 100644 --- a/docs/architecture/overview.md +++ b/docs/architecture/overview.md @@ -41,7 +41,7 @@ worker-ingest, worker-studio, worker-plugins ── POST /internal/events ── - sd-server takes its model as startup arguments, so it cannot start at boot. Whenever its pinned build is staged, in dev too (`pnpm build:sdcpp`, which `predev` runs), `watchImageModel` asks the API every 5 seconds which files to run, and starts, restarts or stops it on a change. The API names a model only while a Studio job needs one and for up to 5 minutes after, so its weights are not held beside the chat model all session ([`studio.md`](studio.md#the-image-path)). - audiocpp_server refuses an empty model list, so it starts only once the API has written `audio/server.json` under the data directory, in dev too (`pnpm build:audiocpp`, which `predev` runs). On Windows and Linux that script compiles audio.cpp, and without a C++ toolchain the app runs without local audio ([packaging](packaging.md)). Electron checks that file every 5 seconds and restarts the server when it changes. Electron sets the machine-wide flags: the CPU backend, half the logical cores up to 8, one loaded model, and an unload after 5 idle minutes. The API writes that file whenever an audio model is installed or deleted, and at startup ([`local-models/catalog.md`](local-models/catalog.md)). The Studio worker voices podcasts there, at `SURFSENSE_LOCAL_AUDIO_BASE_URL`, and unloads the model when a podcast ends ([`studio.md`](studio.md)). - The same `audiocpp_server` runs a second time as `audiocpp-transcription`, for speech recognition, from its own `audio/transcription.json` and on its own port, so a voice install, a podcast's unload or a busy voice model never touches it ([ADR 0056](../adr/0056-speech-recognition-runs-on-a-second-audiocpp-server.md)). Electron checks that file every 5 seconds: it starts the server once the file is there, restarts it when the file changes and stops it when the file goes. It takes the voices' flags, with no eSpeak. Unlike the voices' server it is restarted after a crash, at most once every 10 seconds, and left stopped after 3 crashes in 5 minutes until the file changes. The API alone is told its address, `SURFSENSE_LOCAL_TRANSCRIPTION_BASE_URL`. The API writes that file once a speech recogniser is downloaded, and only when its bytes change ([catalog](local-models/catalog.md)), so the server runs only then. The API's transcription routes are its only caller, one request at a time through one gate ([admission](local-models/admission.md#speech-recognition)). -- Only the API gates the window. Electron waits up to 60 seconds for `/health` and gives up at once if the API exits. llama-server is best-effort; its state shows through `/llm/providers`. +- Only the API gates the window. Electron waits up to 60 seconds for `/health` and gives up at once if the API exits. Giving up shows an error dialog, in English, with the reason and the last three lines of the API's output (a failed migration snapshot is named there), then quits ([`startup-failure/`](../../surfsense_local/electron/src/main/startup-failure/)); the log, kept only in memory, goes with the app ([about](about.md#known-gaps)). llama-server is best-effort; its state shows through `/llm/providers`. - opencode, the agent's engine, starts only once the API has written `agent/opencode.json` under the data directory, in dev too (`pnpm build:opencode`, which `predev` runs, stages it unless `SURFSENSE_LOCAL_OPENCODE_ENABLED=0`: [agent](agent.md)). Electron checks for that file every 2 seconds: it starts opencode once the file is there, stops it when the file goes, and restarts it after a crash with a growing delay, stopping after three quick exits until the API writes the file again ([agent](agent.md#from-a-thread-to-a-running-opencode)). A rewrite does not restart it: opencode watches the file and loads it again ([agent](agent.md#from-a-thread-to-a-running-opencode)). Electron removes the file at boot, so each run's API writes its own. At boot Electron picks opencode's port and a random password, which it passes to the API alone as `SURFSENSE_LOCAL_OPENCODE_URL` and `SURFSENSE_LOCAL_OPENCODE_PASSWORD`. opencode's environment is built from a few system variables rather than inherited, with its home and XDG folders under `agent/opencode/`, its own fetches switched off and its proxy pointed at the discard port ([`sidecars/opencode.ts`](../../surfsense_local/sidecars/opencode.ts)). `serve` keeps running when its parent is killed, so Electron records the pid, port and password in `agent/opencode-process.json`, and the next boot stops a recorded process that accepts that password and reports that pid as its own. - Before a quit, whether from the menu or by closing the last window, Electron asks the API how many chat replies are being written; with any, it asks the person, and on Quit has the API store each reply before anything is stopped ([chat](chat.md#runs)). Cancel keeps the app open. - On macOS and Linux each child runs in its own process group. On quit Electron sends SIGTERM and, after 5 seconds, SIGKILL; the group is also killed as soon as the sidecar itself exits, because a child that ignores SIGTERM would outlive the app. On Windows it kills the process tree. A single-instance lock hands a second launch to the first window, because two sets of sidecars would fight over the SQLite file. @@ -127,7 +127,7 @@ The rules that keep the processes out of each other's way: - [`backend/shared/`](../../surfsense_local/backend/shared/): configuration, the engine and its pragmas, `upgrade_to_head()`, the three Huey queues, `retrieve()`, and the secret that encrypts stored keys. - [`backend/alembic/`](../../surfsense_local/backend/alembic/): revisions `0001` to `0033`, all hand-written. `env.py` has no `target_metadata`, so autogenerate cannot run by accident. - [`frontend/src/features/`](../../surfsense_local/frontend/src/features/): `chat`, `agent`, `sources`, `studio`, `workspaces`, `dashboard`, `models`, `onboarding`, `settings`, `egress`, `plugins`, `license`, `jobs`, `migration`, `updates` and `feedback`. -- [`electron/src/main/`](../../surfsense_local/electron/src/main/): `index.ts` (the window, IPC and the paths it hands to boot), `sidecars/` (boot, the supervisor, one spec per sidecar, and the image-model, preset, audio and agent watchers), `session-log/` (this run's output, for issue reports), `permissions/` (who may use the microphone), `microphone/` (the operating system's switch), `secret.ts`, `updater.ts` and `document-files.ts`. +- [`electron/src/main/`](../../surfsense_local/electron/src/main/): `index.ts` (the window, IPC and the paths it hands to boot), `sidecars/` (boot, the supervisor, one spec per sidecar, and the image-model, preset, audio and agent watchers), `session-log/` (this run's output, for issue reports), `startup-failure/` (what the dialog says when the API does not start), `permissions/` (who may use the microphone), `microphone/` (the operating system's switch), `secret.ts`, `updater.ts` and `document-files.ts`. ## Where to read next @@ -158,5 +158,4 @@ The rules that keep the processes out of each other's way: ## Known gaps - The sources list reloads itself on a workspace event rather than through TanStack Query: it is component state, so nothing else that shows a document is invalidated by the same event. -- When the API does not start, because it exits or `/health` does not answer within 60 seconds, Electron writes the reason to the session log and quits: no window opens, nothing on screen says why, and the log, kept only in memory, goes with the app ([about](about.md#known-gaps)). A migration snapshot that cannot be written, such as on a full disk, ends this way ([data model](data-model.md#known-gaps)). diff --git a/surfsense_local/electron/src/main/index.ts b/surfsense_local/electron/src/main/index.ts index a413e43b0d..ef39d19795 100644 --- a/surfsense_local/electron/src/main/index.ts +++ b/surfsense_local/electron/src/main/index.ts @@ -5,6 +5,7 @@ import { pathToFileURL } from "node:url" import { app, BrowserWindow, + dialog, ipcMain, Menu, nativeTheme, @@ -26,6 +27,7 @@ import { allowedExternalUrl, confirmedExternalUrl } from "./external-url.ts" import { frameMayNavigate } from "./framed-navigation.ts" import { pluginPageUrl, type PluginPage } from "./plugin-page-url.ts" import { loadSecret } from "./secret.ts" +import { describeStartupFailure } from "./startup-failure/describe-startup-failure.ts" import { bootSidecars, type BootContext } from "../../../sidecars/boot.ts" import { stopOnce } from "../../../sidecars/stop-once.ts" import { stopAll, type Sidecars } from "../../../sidecars/supervisor.ts" @@ -590,6 +592,12 @@ function main(): void { }) .catch((err: unknown) => { warnMain(sessionLog, `failed to start: ${String(err)}`) + // No window exists yet, so without this the app just vanishes. English, + // like the application menu: main's own text is not translated. + if (!shuttingDown) { + const { title, detail } = describeStartupFailure(err, sessionLog.lines()) + dialog.showErrorBox(title, detail) + } void shutdown().finally(() => app.exit(1)) }) diff --git a/surfsense_local/electron/src/main/startup-failure/describe-startup-failure.test.ts b/surfsense_local/electron/src/main/startup-failure/describe-startup-failure.test.ts new file mode 100644 index 0000000000..1790b72ffd --- /dev/null +++ b/surfsense_local/electron/src/main/startup-failure/describe-startup-failure.test.ts @@ -0,0 +1,37 @@ +import assert from "node:assert/strict" +import test from "node:test" + +import { describeStartupFailure } from "./describe-startup-failure.ts" + +test("names why startup failed", () => { + const { title, detail } = describeStartupFailure( + new Error("sidecar exited during startup"), + [] + ) + + assert.equal(title, "SurfSense could not start") + assert.match(detail, /^sidecar exited during startup/) +}) + +test("quotes the end of the API's output, where a failed migration is named", () => { + const { detail } = describeStartupFailure(new Error("sidecar exited during startup"), [ + "10:00:00 [api] Traceback (most recent call last):", + '10:00:00 [api] File "main.py", line 5, in lifespan', + '10:00:00 [api] File "migrations.py", line 30, in upgrade_to_head', + '10:00:00 [api] File "migration_snapshot.py", line 36, in snapshot_before_upgrade', + "10:00:00 [llamacpp] loading model", + "10:00:00 [api] RuntimeError: could not write a database snapshot to ~/backups/0030-0032.db before migrating: database or disk is full", + ]) + + assert.match(detail, /RuntimeError: could not write a database snapshot/) + assert.match(detail, /line 36, in snapshot_before_upgrade/) + assert.doesNotMatch(detail, /llamacpp|\[api\]/) + assert.doesNotMatch(detail, /Traceback/) +}) + +test("says nothing about the API when it wrote nothing", () => { + const { detail } = describeStartupFailure("boom", ["10:00:00 [llamacpp] hi"]) + + assert.match(detail, /^boom/) + assert.doesNotMatch(detail, /last output/) +}) diff --git a/surfsense_local/electron/src/main/startup-failure/describe-startup-failure.ts b/surfsense_local/electron/src/main/startup-failure/describe-startup-failure.ts new file mode 100644 index 0000000000..71f6b436b5 --- /dev/null +++ b/surfsense_local/electron/src/main/startup-failure/describe-startup-failure.ts @@ -0,0 +1,28 @@ +// A Python traceback ends with the error, so its last lines say what went wrong. +const API_TAIL_LINES = 3 + +export type StartupFailure = { title: string; detail: string } + +/** + * What the dialog says when the API did not start: why Electron gave up, and + * the last of the API's own output, where a refused migration or a full disk + * is named. `logLines` is the session log, oldest first. + */ +export function describeStartupFailure( + error: unknown, + logLines: readonly string[] +): StartupFailure { + const reason = error instanceof Error ? error.message : String(error) + const apiTail = logLines + .filter((line) => line.includes(" [api] ")) + .slice(-API_TAIL_LINES) + .map((line) => line.slice(line.indexOf(" [api] ") + " [api] ".length)) + const parts = [reason] + if (apiTail.length > 0) { + parts.push(`The API's last output:\n${apiTail.join("\n")}`) + } + parts.push( + "Start SurfSense again. If it keeps failing, include this message in a report." + ) + return { title: "SurfSense could not start", detail: parts.join("\n\n") } +}