Repository navigation
docs(plugins): redesign plugins as MCP servers behind one tool gateway #2206
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,31 @@ | ||
| # ADR 0047: SurfSense's paid plugins are source-available under the Business Source License and unlocked by the SurfSense license | ||
|
|
||
| - **Status:** Accepted | ||
| - **Date:** 2026-10-08 | ||
| - **Supersedes:** [ADR 0025](0025-scraper-client-as-paid-plugin.md) in part: paid plugin source living in `plugins/` under Apache-2.0 | ||
| - **Source:** [file-agent README L252](https://github.com/MODSetter/SurfSense/blob/2327494854e590f484bd61e3b5ff9d1fc4db70ce/docs/proposals/file-agent/README.md#L252), [file-agent strategy L171–187](https://github.com/MODSetter/SurfSense/blob/2327494854e590f484bd61e3b5ff9d1fc4db70ce/plans/community-local/file-agent-strategy.md#L171-L187), maintainer decision, 8 Oct 2026 (no permalink) | ||
|
|
||
| ## Context | ||
|
|
||
| ADR 0025 kept paid plugin source in `plugins/` under Apache-2.0. A fork could remove any license check in the app ([ADR 0019](0019-offline-licenses.md)). Nothing in `surfsense_local/` is gated, so the code the license sells has to sit outside the Apache-2.0 tree. | ||
|
|
||
| Plugins became MCP servers, remote ones first, hosted by their publishers ([ADR 0051](0051-plugins-are-mcp-servers-behind-one-tool-gateway.md)). For SurfSense's own paid plugins the publisher is SurfSense, and what they sell, such as the scrapers behind the scraper API, already runs on SurfSense's servers. | ||
|
|
||
| ## Decision | ||
|
|
||
| - SurfSense's paid plugins live in `plugins/proprietary/` under the Business Source License 1.1, each a container of its own that SurfSense deploys, holding everything it sells. Everything under `plugins/proprietary/` is under the Business Source License, including helpers that are Apache-2.0 in the backend today, such as the proxy, captcha and crawl code. All of it was written by SurfSense's maintainers, so SurfSense holds the rights to relicense it. The scrapers move there from `surfsense_backend/app/proprietary/`: copied now, and deleted from the backend once its scraper API no longer serves outside clients, never before the purge on 18 Oct 2026. | ||
|
There was a problem hiding this comment. 📐 Maintainability & Code Quality | 🟠 Major | 🏗️ Heavy lift 🔎 Supported by static analysis🏁 Script executed: set -eu
printf '%s\n' '--- changed-doc excerpts ---'
for f in docs/adr/0047-premium-plugins-are-source-available.md docs/proposals/plugins/03-remote-plugins.md; do
if test -f "$f"; then
echo "--- $f ---"
nl -ba "$f" | sed -n '1,130p'
else
echo "MISSING: $f"
fi
done
printf '%s\n' '--- license and notice files ---'
find . -type f \( -iname 'LICENSE*' -o -iname 'NOTICE*' -o -iname '*COPYING*' \) -not -path './.git/*' -not -path './node_modules/*' | sort
printf '%s\n' '--- referenced proprietary/backend paths ---'
find plugins surfsense_backend -type f \( -path '*proprietary*' -o -path '*proxy*' -o -path '*captcha*' -o -path '*crawl*' -o -path '*scrap*' \) 2>/dev/null | sort | sed -n '1,240p'
printf '%s\n' '--- Apache/license references in relevant source and docs ---'
rg -n -i --glob '!node_modules/**' --glob '!**/.git/**' 'Apache-2\.0|Apache License|NOTICE|Business Source License|BSL' plugins surfsense_backend docs 2>/dev/null | sed -n '1,260p'
printf '%s\n' '--- PR diff for the two docs ---'
git diff --no-ext-diff --unified=20 37baf907a82d43095c81bb588d520a4df34d37d8 2983412b96f9ab5e8198add5a07777f77ecd62a7 -- docs/adr/0047-premium-plugins-are-source-available.md docs/proposals/plugins/03-remote-plugins.md | sed -n '1,320p'Repository: MODSetter/SurfSense Length of output: 42616 🏁 Script executed: set -eu
for f in LICENSE surfsense_backend/app/proprietary/LICENSE \
surfsense_backend/app/utils/crawl/__init__.py \
surfsense_backend/app/utils/captcha/__init__.py \
surfsense_backend/app/utils/captcha/solvers.py \
surfsense_backend/app/proprietary/web_crawler/site_crawler.py \
surfsense_backend/app/proprietary/web_crawler/testbench/__init__.py; do
echo "--- $f ---"
if test -f "$f"; then nl -ba "$f" | sed -n '1,45p'; else echo "MISSING"; fi
doneRepository: MODSetter/SurfSense Length of output: 12605 Verify Apache notice obligations before adopting BSL-only terms.
Check the copied files and their license metadata. Retain required notices, or document why they do not apply, before distributing the code under BSL-only terms. 🤖 Prompt for AI Agents |
||
| - A paid plugin is unlocked by the SurfSense license key, a trial included, checked by the plugin's server on every call and never by the app ([plugins proposal](../proposals/plugins/core/05-paid.md)). | ||
| - The first kind of paid plugin is remote: SurfSense hosts its server, and the user connects to it. A paid plugin that runs on the user's machine comes later, delivered by the license server only to a valid license, never in the installer or the public plugin list. | ||
| - Other publishers' paid plugins never use the SurfSense license. They charge on their own service and check access themselves. | ||
| - Everything else stays Apache-2.0: SurfSense's free plugins in `plugins/`, the app, its engines, the format skills, all job content, and job packs. | ||
|
|
||
| ## Consequences | ||
|
|
||
| - The root [`LICENSE`](../../LICENSE) names only `surfsense_backend/app/proprietary/` as Business Source License today. It needs a line for `plugins/proprietary/`, with a copy of that license in the folder, before code lands there. | ||
| - Anyone can read a paid plugin's source, but production use outside the license's grant is not allowed. The server's license check is the only gate. | ||
| - Until the backend's copy is deleted, the scrapers exist twice, and a fix goes to both. | ||
|
|
||
| ## Where the code stands | ||
|
|
||
| - No paid plugin, `plugins/proprietary/` folder or license line for it exists. The scrapers still live only in `surfsense_backend/app/proprietary/`. | ||
| - `surfsense_mcp` exposes the scrapers to outside clients with personal keys; it is not the app's plugin. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,32 @@ | ||
| # ADR 0051: A plugin is an MCP server, and every caller reaches plugin tools through one Tool Gateway, remote servers first | ||
|
|
||
| - **Status:** Proposed, with the [plugins proposal](../proposals/plugins/README.md) | ||
| - **Date:** 2026-10-08 | ||
| - **Source:** maintainer decision, 8 Oct 2026 (no permalink); [Pi's built-in MCP extension](https://github.com/earendil-works/pi/blob/ce950d78f424dcaf9f5d6a03ce80ab141130eb1d/packages/coding-agent/src/extensions/mcp/index.ts), [Pi's MCP client](https://github.com/earendil-works/pi/blob/ce950d78f424dcaf9f5d6a03ce80ab141130eb1d/packages/mcp/README.md) | ||
|
|
||
| ## Context | ||
|
|
||
| The first plugin design made a plugin a sidebar action with a form. Its only output was notes in Sources, which the chat and the agent reached by searching ([earlier design](../proposals/plugins/bundles/README.md)). No engine could call a plugin, a note went stale once written, and every plugin was code in this repository published with app releases, so only SurfSense could grow the set. | ||
|
|
||
| The agent already calls SurfSense's own tools over MCP ([agent](../architecture/agent.md#surfsenses-tools)). Many companies already run MCP servers that every MCP client connects to. Pi keeps one tool pipeline of its own and adapts MCP servers into it as one source among others, through its own small client. Claude's connectors list remote MCP servers and package local ones as MCP Bundles. | ||
|
|
||
| ## Decision | ||
|
|
||
| - A plugin is an MCP server. SurfSense speaks MCP unmodified and defines no protocol of its own. | ||
| - One Tool Gateway in the API is the only way any caller reaches a plugin tool: opencode, the chat engine's router step, and `@` mentions. Approval, policy, logging, trimming and storing results live there. | ||
| - Behind the gateway, sources implement one interface. The first is a remote MCP server over HTTPS, hosted by its publisher, which may be SurfSense, a company or a community developer. Plugins that run on the user's machine come later as bundles behind the same interface. | ||
| - A tool's result returns into the turn and is stored with the call. Nothing reaches Sources unless the user saves it. | ||
| - The API carries a small MCP client of its own rather than the official SDK. | ||
| - No plugin code runs inside the app, and a plugin cannot change SurfSense's own behaviour. | ||
|
|
||
| ## Consequences | ||
|
|
||
| - An existing MCP server becomes a SurfSense plugin with a registry entry and no new code. | ||
| - Both engines and the user get one set of tools; adding a kind of source changes none of them. | ||
| - What a tool is sent leaves the machine for its publisher's host. Plugins are optional, and the app keeps working offline without them; offline plugins wait for bundles. | ||
| - SurfSense cannot see or check a remote server's code. Review of the listing, egress consent, approval and delisting are the controls. | ||
| - The local runner, SDK and CLI built for the earlier design are kept for bundles, not wired in. | ||
|
|
||
| ## Where the code stands | ||
|
|
||
| Nothing of the gateway, the client or the plugin tables exists. `modules/plugins/` holds the earlier design's runner, and `plugins/core/` its SDK, CLI and manifest rules. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,31 @@ | ||
| # ADR 0052: The chat engine's model never calls tools; SurfSense chooses a plugin tool through structured output and calls it | ||
|
|
||
| - **Status:** Proposed, with the [plugins proposal](../proposals/plugins/README.md) | ||
| - **Date:** 2026-10-08 | ||
| - **Source:** maintainer decision, 8 Oct 2026 (no permalink); [agent: which engine a model gets](../proposals/agent/01-which-engine.md) | ||
|
|
||
| ## Context | ||
|
|
||
| The chat engine answers every model with one call and no tools; a thread that needs tools is the agent's ([chat](../architecture/chat.md)). It serves models as small as Qwen3-0.6B, which scores 3.62% on multi-turn function calling against 41.75% for Qwen3-8B on the Berkeley leaderboard, and models whose template has no tool support at all, such as Gemma 3 4B. Plugins have to reach chat threads too. | ||
|
|
||
| Every text route the chat engine uses already takes a JSON schema, and the remote catalog records which remote models support structured output ([agent/01](../proposals/agent/01-which-engine.md)). A schema is not a guarantee: llama-server rejects one for some templates, and some endpoints ignore it. | ||
|
|
||
| ## Decision | ||
|
|
||
| - The chat engine's model never receives tools and never makes a tool call. | ||
| - When ready plugin tools exist and the router is on for the model, a router step runs before the answer: one call held to a schema choosing a tool or `none`, then one call held to the chosen tool's flat input schema filling its inputs. SurfSense makes the call through the Tool Gateway, and the result joins the answer's prompt. | ||
| - The router is on by default for a model that passes the chat eval's router test, a remote model the catalog says supports structured output, and a ChatGPT plan's model. For a local model not yet measured, a model the catalog says nothing about, or a custom connection's model, it is off until the user turns it on. A model the catalog says lacks structured output never runs it. | ||
| - Every router reply is validated against its schema; one that does not match counts as `none`, and a failed router call leaves the turn to answer as it does today. | ||
| - An `@` mention names the tool, so the model only fills its inputs, and where it cannot, the user fills a small form. It works on every model. | ||
|
|
||
| ## Consequences | ||
|
|
||
| - Plugins reach every model in a chat thread, with no tool-calling support needed. | ||
| - A turn with the router costs one or two extra model calls; a turn without ready tools costs nothing more. | ||
| - A small model may choose badly. Measuring it, gating the router per model, approval showing the exact arguments, and `@` mentions bound the cost. | ||
| - Destructive tools and tools without a flat schema are never chosen by the router. | ||
| - On a ChatGPT plan, each router call counts against the user's plan. | ||
|
|
||
| ## Where the code stands | ||
|
|
||
| The chat engine has no router step, `@` mentions, or plugin steps, and the chat eval has no router test. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,31 @@ | ||
| # ADR 0053: Plugins are listed in a registry in this repository, and every plugin not published by SurfSense starts off | ||
|
|
||
| - **Status:** Proposed, with the [plugins proposal](../proposals/plugins/README.md) | ||
| - **Date:** 2026-10-08 | ||
| - **Source:** maintainer decision, 8 Oct 2026 (no permalink); [obsidianmd/obsidian-releases](https://github.com/obsidianmd/obsidian-releases), [Obsidian plugin security](https://obsidian.md/help/Extending+Obsidian/Plugin+security) | ||
|
There was a problem hiding this comment. 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win Pin the external ADR sources.
🤖 Prompt for AI Agents |
||
|
|
||
| ## Context | ||
|
|
||
| Remote plugins are servers their publishers run ([ADR 0051](0051-plugins-are-mcp-servers-behind-one-tool-gateway.md)). The app needs a list of them it can show with no network, a way for companies and community developers to join it, and a way to keep third-party plugins away from users who never chose them. SurfSense's users are mostly not developers. | ||
|
|
||
| Obsidian keeps its community plugins as one list, served from its own server with a mirror in a public repository, each entry pointing at the author's repository, whose GitHub releases hold the files. It ships with community plugins off until the user turns them on. | ||
|
|
||
| ## Decision | ||
|
|
||
| - One file, `plugins/registry/plugins.json` in this repository, lists every plugin the app shows, SurfSense's own included; nothing is maintained beside it. An entry is added by pull request and reviewed once by a maintainer. A change to the server behind it needs no review. | ||
| - The registry points at plugins and never holds them: a plugin's code, and later its bundle files, stay with its publisher, in its own host or repository. | ||
| - Each entry has a `kind`, `remote` now, with `bundle` reserved. Its publisher, `surfsense`, `partner` or `community`, is set in the registry, and `CODEOWNERS` requires a maintainer's approval for every change to it. | ||
| - Every plugin not published by SurfSense, a custom one added by URL included, is off until the user turns third-party plugins on once. | ||
| - A tool a server adds after the user connected, or one that changes as the proposal's rule says, starts off until the user turns it on ([tools added later](../proposals/plugins/core/02-registry.md#tools-added-later)). | ||
| - A maintainer delists a plugin by moving its entry to `removed` with a reason; apps stop offering its tools. | ||
| - On each merge to `main`, CI signs the registry and publishes it to SurfSense's own plugin host. The app fetches it from there after consent and checks the signature against a key compiled in, so the registry grows without app updates. The installer carries a copy, used until the user allows the host and then whenever it is the newer. | ||
|
|
||
| ## Consequences | ||
|
|
||
| - A company or developer joins with one pull request and no build pipeline. | ||
| - Review sees the listing and the tools at the time; it does not see later deploys, which is why later tools start off. | ||
| - SurfSense needs a host for the signed list, and a signing key kept as a CI secret. | ||
|
|
||
| ## Where the code stands | ||
|
|
||
| `plugins/registry/` does not exist, and the app has no copy of the registry, Restricted mode or plugin screen. The only plugin interface is a "Plugins / Coming soon" button in [`dashboard-page.tsx`](../../surfsense_local/frontend/src/features/dashboard/dashboard-page.tsx). | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Match the link label to the new path.
The link target points to
proposals/plugins/bundles/python/01-interpreter.md, but the visible label still saysproposals/plugins/python/01-interpreter.md. Update the label so readers see the correct path.Proposed correction
📝 Committable suggestion
🤖 Prompt for AI Agents