CREDO23 · Sep 22, 2026 · Sep 22, 2026 · Sep 22, 2026 · Sep 22, 2026 · Sep 22, 2026
@@ -0,0 +1,55 @@
+# Plugins
+
+> A plugin is a folder in this repo. A third party adds one by pull request. The desktop app installs it on demand and runs it as its own process for one job.
+
+Shared contract: [`01-protocol.md`](01-protocol.md). Every stream below implements against that file. Do not invent a second wire format.
+
+## Workstreams
+
+Same letter can be picked up at the same time. A stream's own files are in order.
+
+| Stream | Files | Owns | Starts from |
+|---|---|---|---|
+| **SDK** | [`sdk/01-library-and-harness.md`](sdk/01-library-and-harness.md) | `plugins/sdk/`, `plugins/example/`, `plugins/README.md` | the protocol |
+| **Runtime** | [`runtime/01-process.md`](runtime/01-process.md) then [`runtime/02-calls-and-results.md`](runtime/02-calls-and-results.md) | `surfsense_local/backend/modules/plugins/` runner and result import | the protocol. `02` reads the file `01`'s process leaves behind |
+| **Catalog** | [`catalog/01-manifest-and-ci.md`](catalog/01-manifest-and-ci.md) then [`catalog/02-install-and-publish.md`](catalog/02-install-and-publish.md) | manifest checker, PR workflow, install, publish workflow | the protocol. `02` needs the manifest rules in `01` |
+| **App** | [`app/01-api.md`](app/01-api.md) and [`app/02-screen.md`](app/02-screen.md) | plugin routes, the plugin screen | the route list in `01`. The screen can be built against that list before the routes exist |
+| **Python** | [`python/01-interpreter.md`](python/01-interpreter.md) | the interpreter the packaged app spawns | nothing. Other streams use `uv`'s Python until this lands |
+
+**Demo:** SDK harness runs `plugins/example` on a laptop, and the API runs the same plugin as a subprocess and a `document` result shows up as a note. No GitHub release required for that demo.
+
+**Ship:** catalog publish workflow uploads a real tarball, the app installs it from the bundled catalog, a paid plugin stays locked without a license.
+
+## Locked decisions
+
+| Decision | Choice |
+|---|---|
+| Where source lives | `plugins/<id>/` in this repo. A third party contributes by pull request. |
+| Id | Folder name. `^[a-z][a-z0-9-]{0,63}$`. Unique. Immutable once a version is published. `plugins/RESERVED` plus the prefix `surfsense-` are ours. `author` is a field, not part of the id. |
+| Process | One subprocess per run. Not loaded into the API or the worker. |
+| Talks to the app | Arguments, environment variables, and a results file. No socket and no request protocol. Stdout is logs. |
+| Network | The plugin makes its own HTTP calls. `hosts` in `plugin.json` is copied into the catalog and shown. The app does not intercept those calls and Settings → Network does not apply to them. |
+| Dependencies | `requirements.txt` with `==` pins. CI runs `pip install` into the tarball. The user's machine never runs `pip`. |
+| SDK | `plugins/sdk/` in this repo. Not published to PyPI. The app ships that folder next to the plugin interpreter. A plugin tarball does not contain it. |
+| License | `free` or `paid`. Community plugins are `free`. `paid` means one of ours, and it installs and runs only with an unexpired `individual` or `team` license. A trial does not unlock it. No price field. |
+| Catalog | `catalog.json` is bundled in the installer, so the list is visible with no network. Refreshing it is a separate egress destination, off by default. The plugins themselves are not bundled. |
+| Results | The plugin appends them to a file. After the process exits, `kind: "document"` becomes a note in the run's workspace. Any other kind is stored on the run and shown there. The platform does not require documents, and it does not define resume or dedup. |
+| Secrets | Names in the manifest. Values go through `shared/secrets.py`. The app puts each one in an environment variable before spawn. |
+| What we do not build | A sandbox, a process-count cap, WASM, `host/generate`, hot reload, pip or npm on the user's machine, inferring hosts from imports, plugin-authored HTML, payments. The host places an entry in the existing sidebar and draws its form. |
+
+## Caught while specifying
+
+These were not in the design conversation. They are decided here so a developer is not blocked on them.
+
+| Gap | Decision |
+|---|---|
+| The packaged app has no general Python. API and worker are PyInstaller binaries. | [`python/01-interpreter.md`](python/01-interpreter.md) ships a real interpreter. Plugins run on that, never inside the frozen binaries. |
+| A result needs a shape the app can import. | One JSON object per line in the results file. `document` becomes a note. Anything else is kept on the run. |
+| A document has to belong to a workspace. | A run is started in a workspace. Document results are notes in that workspace, then `ingest_document`. |
+| The installer still downloads from GitHub. | That path may follow only `github.com` → `objects.githubusercontent.com`. The plugin's own HTTP is not this path. |
+| A fetched `catalog.json` is the trust anchor for every sha256 in it. | The refresh URL is compiled into the app. A catalog entry whose tarball host is not `github.com` or `objects.githubusercontent.com` is rejected. |
+| `documents.dedup_key` already exists. | Plugin results do not set it. A plugin that wants to skip work it already did uses its data directory. |
+
+## Out of scope
+
+Sandboxes, memory and process caps, charging by a community author, bundling plugin code in the installer, running a plugin inside the API process, Amazon or Reddit themselves. Those are plugins someone writes after this lands. The example plugin in the SDK stream is the one this work has to ship.
@@ -0,0 +1,139 @@
+# Protocol
+
+The contract the SDK, the runtime, the catalog, and the app all implement. A change here is a change to every stream.
+
+Version is `1`. The handshake and `sdk` range are how a later version shows up. Do not add a field to a message without bumping it.
+
+## On disk
+
+```
+plugins/<id>/
+  plugin.json
+  main.py
+  requirements.txt    # omitted when there are no dependencies
+  README.md
+  LICENSE
+  tests/              # optional
+  fixtures/           # optional
+```
+
+`site-packages/` is not in git. CI creates it inside the tarball.
+
+### `plugin.json`
+
+| Field | Rule |
+|---|---|
+| `id` | Equals the folder name. Pattern above. |
+| `name` | Display string, 1–80 characters. |
+| `author` | Display string. Not unique. |
+| `version` | Semver `MAJOR.MINOR.PATCH`. |
+| `sdk` | A PEP 440 specifier. The app's SDK version must satisfy it or the run does not start. |
+| `license` | `free` or `paid`. |
+| `hosts` | Array of hostnames, no scheme, no path, no port. Shown in the catalog. The app does not intercept the plugin's own connections. |
+| `secrets` | Array of names. Values are never in this file. |
+| `entries` | At least one. |
+
+An entry:
+
+| Field | Rule |
+|---|---|
+| `name` | `^[a-z][a-z0-9-]{0,63}$`, unique within the plugin. |
+| `title` | Display string. |
+| `inputs` | Array, may be empty. Each is `{ "name", "kind", "required" }`. `kind` is `string`, `number`, or `boolean`. |
+
+Unknown fields are ignored, so a future field does not break an older app. A missing required field, a bad id, or a second entry with the same name fails the manifest check.
+
+### Tarball
+
+`<id>-<version>.tar.gz`. One top-level directory `<id>-<version>/` containing the folder above plus `site-packages/` when CI installed dependencies. The `plugin.json` inside the tarball must deep-equal the catalog entry's manifest fields (`id`, `name`, `author`, `version`, `sdk`, `license`, `hosts`, `secrets`, `entries`). A mismatch rejects the install. The sha256 is of the gzip bytes.
+
+Extraction rejects any member whose path is absolute, contains `..`, or is a link that resolves outside the destination. The destination is `<data>/plugins/<id>/<version>.tmp-<pid>/`, then one rename to `<data>/plugins/<id>/<version>/`.
+
+### Catalog
+
+One JSON file, `catalog.json`:
+
+```json
+{
+  "plugins": [
+    {
+      "id": "example",
+      "name": "Example",
+      "author": "SurfSense",
+      "version": "1.0.0",
+      "sdk": ">=1,<2",
+      "license": "free",
+      "hosts": [],
+      "secrets": [],
+      "entries": [],
+      "url": "https://github.com/MODSetter/SurfSense/releases/download/plugin-example-1.0.0/example-1.0.0.tar.gz",
+      "sha256": "<64 hex>",
+      "yanked": false
+    }
+  ]
+}
+```
+
+The app ships a copy taken at desktop build time. A refresh downloads this same file from a URL compiled into the app, and only from that URL. `yanked: true` stays in the file so an already-installed copy can be refused with a reason.
+
+## How a run starts
+
+There is no socket and no request protocol. The plugin does its own network calls, reads secrets from the environment, and appends results to a file. The app spawns it and reads that file after the process exits.
+
+```
+<python> -m surfsense_plugin <plugin-dir> <entry> --inputs <file> --results <file> --data <dir>
+```
+
+| Argument | Meaning |
+|---|---|
+| `<entry>` | An entry name from the manifest. The app checks this before spawn. |
+| `--inputs` | A JSON object. Keys are the entry's input names, values match the declared kinds. The app writes this file. |
+| `--results` | An empty file the app creates. The plugin appends one JSON object per line. |
+| `--data` | `<data>/plugins/<id>/data`. Exists from install, survives upgrades, deleted on uninstall. |
+
+For each secret name in the manifest, the app sets `SURFSENSE_PLUGIN_SECRET_<NAME>` to the stored value. Secret names match `^[a-z][a-z0-9_]{0,63}$` so they are environment variables. The app refuses to spawn when a declared secret has no stored value.
+
+Stdout and stderr are logs. The app keeps the last 16 KiB on the run.
+
+Exit 0 is success. Any other exit is `failed`. The app still imports every well-formed results line already in the file.
+
+Cancel: `SIGTERM`, then `SIGKILL` five seconds later. The run is `cancelled`. Lines already flushed stay.
+
+### Results file
+
+Each line is one JSON object, flushed before the next one:
+
+```json
+{"kind":"document","body":{"title":"Show HN","content":"..."}}
+```
+
+| `kind` | `body` | What the app does after the process exits |
+|---|---|---|
+| `document` | `{ "title": str, "content": str }` | Insert a `NOTE` in the run's workspace, set `document_metadata` to `{ "pluginId", "entry", "runId" }`, enqueue `ingest_document`. Do not set `dedup_key`. |
+| anything else | a JSON object | Insert a row on the run: `kind` + `body`. The screen shows it. |
+
+A line over 1 MiB, or a line that is not JSON, is skipped. The valid lines before and after it are still imported. A run that writes no lines is a success that produced nothing. `document` is how a source plugin puts something in the library. It is not required.
+
+The plugin's own HTTP is ordinary Python. The app does not see those calls, does not apply the `hosts` list to them, and does not apply Settings → Network to them. `hosts` is what the catalog shows.
+
+## SDK surface
+
+```python
+import urllib.request
+from surfsense_plugin import entry, result, secret
+
+@entry("search")
+def search(query: str) -> None:
+    token = secret("token")
+    request = urllib.request.Request(
+        "https://hacker-news.firebaseio.com/v0/item/1.json",
+        headers={"Authorization": f"Bearer {token}"},
+    )
+    with urllib.request.urlopen(request) as response:
+        body = response.read().decode()
+    result("document", {"title": "Show HN", "content": body})
+```
+
+`secret` reads the environment variable. A missing one raises. `result` appends one line to the results file and flushes it. There is no `fetch` in the SDK.
+
+The harness is `python -m surfsense_plugin.harness <plugin-dir> <entry> --input query=plugins`. It passes secrets from the process environment, runs the entry, and prints the results file. It does not intercept HTTP.
@@ -0,0 +1,37 @@
+# App — API
+
+> Owns: `surfsense_local/backend/modules/plugins/router.py`, `schemas.py`.
+> Routes the screen in [`02-screen.md`](02-screen.md) calls. Runtime: [`../runtime/01-process.md`](../runtime/01-process.md). Install: [`../catalog/02-install-and-publish.md`](../catalog/02-install-and-publish.md).
+
+## Goal
+
+The screen can list plugins, install one, set a secret, and start a run, using the modules the other streams built.
+
+## Work
+
+| Method | Path | Behavior |
+|---|---|---|
+| `GET` | `/plugins` | The bundled catalog, or the refreshed one when `plugin_catalog` has been fetched. Each item includes `installed` version or null, `license` and whether it is unlocked, `hosts`, `yanked`. |
+| `POST` | `/plugins/catalog/refresh` | Runs the refresh. Egress off → 403 with the destination name, same shape as the other egress denials. |
+| `POST` | `/plugins/{id}/install` | The install function. 403 for egress, 402 for `license_required`, 409 for a bad hash or manifest. |
+| `DELETE` | `/plugins/{id}` | Uninstall. |
+| `PUT` | `/plugins/{id}/secrets/{name}` | Body `{ "value": str }`. Encrypts via `shared/secrets.py` under `plugin:<id>:<name>`. `name` must be in the manifest. Never returns the value. |
+| `GET` | `/plugins/{id}/secrets` | Names and a boolean `set`. Never values. |
+| `POST` | `/workspaces/{id}/plugins/{plugin}/entries/{entry}/runs` | Body is the inputs object. 422 when a name or kind does not match the entry. 422 when a declared secret is unset, listing the names. 402 when locked. Inserts `plugin_runs` and enqueues `run_plugin`. |
+| `GET` | `/workspaces/{id}/plugin-runs/{run}` | Status, error, log tail, and `plugin_results` rows. |
+| `POST` | `/workspaces/{id}/plugin-runs/{run}/cancel` | Sets the cancel flag the task watches. |
+
+Emit `plugin.run.updated` on the existing events broker when a run status changes, carrying the run id, so the screen can invalidate.
+
+## Acceptance
+
+- List returns the example plugin from a fixture catalog with `installed: null`.
+- Install, then list shows the version.
+- Run with a missing required input returns 422 and does not insert a run.
+- Run of a `paid` plugin with no license returns 402.
+- A successful run of `plugins/example` returns a run id; polling the run reaches `succeeded`; a note exists in the workspace.
+- `GET` of a secret returns `set: true` after `PUT` and does not contain the value.
+
+## Needs from
+
+Install, the runner task, license, egress, events. The route functions can be written and the 422 tests can land before the runner does, with the enqueue mocked. The success test waits for the runtime stream.
@@ -0,0 +1,33 @@
+# App — screen
+
+> Owns: `surfsense_local/frontend/src/features/plugins/`.
+> Calls: [`01-api.md`](01-api.md).
+
+## Goal
+
+Maya installs a plugin from a list, then uses it where she already works. An entry that returns documents is a sidebar action next to New chat, not a job she runs from an admin page.
+
+## Work
+
+- Installed entries are `SidebarNavAction` rows on the existing left sidebar, below Chats. The label is the entry title. Click opens a dialog built from the app's own inputs: text, number, checkbox. Submit starts the run in the current workspace. The dialog shows running, then succeeded, failed, or cancelled, with the error and the log tail, and a cancel button.
+- A `document` result is not drawn in that dialog. It arrives in the sources list, which already lists notes. Other kinds are listed in the dialog as the kind and the JSON.
+- Settings holds the catalog, not the run. Each row: name, author, version, the `hosts` list or the word "none", `free` or `paid`, and installed version when present.
+- A `paid` row that is locked explains that a SurfSense license is required and links to the existing license settings. The install button is disabled.
+- Install calls `POST /plugins/{id}/install`. Egress 403 shows the same Settings → Network explanation the other destinations use, naming `plugin_install`.
+- Refresh calls `POST /plugins/catalog/refresh`. The same 403 names `plugin_catalog`. The list is already on screen from the bundled catalog before either destination is enabled.
+- An installed plugin with a secret shows a field per name in Settings. Saving calls `PUT`. The field does not redisplay the value. The sidebar action stays disabled until every declared secret is set.
+- Uninstall asks once, then `DELETE`. The sidebar row disappears. Notes already created stay in Sources.
+- Freshness for the dialog is the existing SSE invalidate on `plugin.run.updated`, with a refetch interval while `running` as the fallback.
+
+Follow the frontend workflow. The dialog and the settings list use the components the app already has. A plugin does not ship a layout.
+
+## Acceptance
+
+- With both egress destinations off, the screen lists the bundled catalog and no request has left the machine.
+- Install of `example` from a local fixture server adds a sidebar row. Running it with text `hi` from that row reaches `succeeded`, and Sources contains a note with content `hi`.
+- A `paid` plugin with no license on disk cannot be installed from the screen.
+- A failed run shows the error and the log tail.
+
+## Needs from
+
+[`01-api.md`](01-api.md). The screen can be built against those paths before the handlers exist.
@@ -0,0 +1,33 @@
+# Catalog — manifest and pull-request checks
+
+> Owns: `surfsense_local/backend/modules/plugins/manifest.py`, `plugins/RESERVED`, `.github/workflows/plugin-check.yml`.
+> Contract: [`../01-protocol.md`](../01-protocol.md).
+
+## Goal
+
+A bad plugin folder fails in CI before anyone reviews it. The checker is a library the install path calls too, so the rules exist once.
+
+## Work
+
+- `load_manifest(path) -> Manifest` and `check_tree(folder, reserved: set[str]) -> list[str]`. Empty list means the folder may be merged. The strings are the review comment.
+- Rules, all of them, and no others:
+  - `plugin.json` matches the protocol table.
+  - `id` equals the folder name and is not in `reserved` and does not start with `surfsense-`.
+  - `version` is semver. When `origin/main` has the same id, the version is strictly greater.
+  - `license` is `free` unless the id is reserved. A reserved id may be `free` or `paid`.
+  - `requirements.txt`, if present, has every requirement pinned with `==` and no URL, no editable, no `--index-url`.
+  - `main.py` exists. `LICENSE` exists. No `site-packages/` directory in the tree.
+- `plugins/RESERVED` is one id per line. `example` is not reserved. Add a line when we ship a paid plugin, in the same pull request as that plugin.
+- Workflow `plugin-check.yml` on pull requests that touch `plugins/**`. It runs the checker on each changed `plugins/<id>/` and fails the job with the checker's lines. It does not install requirements and it does not start Electron.
+- The checker does not read imports and does not fail on a socket call.
+
+## Acceptance
+
+- A fixture folder that is valid produces no errors.
+- One test per rule above, each with the smallest folder that breaks that rule and the error string that names it.
+- A second copy of `example`'s id in another folder is an error.
+- A `requirements.txt` of `requests==2.32.3` is valid. `requests>=2` is not.
+
+## Needs from
+
+The protocol's manifest table. Nothing from the app.