[CREDO23]

Summary

• The worker runs a plugin's action as its own process and records how it ended. run_plugin(run_id) on a new plugins Huey queue starts python -S -m surfsense_plugin_sdk.run from the installed folder, and the run's plugin_runs row goes from queued to running to succeeded, or to failed with exit <code>. It keeps the last 16 KiB of the plugin's output as the log tail. The plugin writes through the app's API while it runs, so nothing is imported afterwards.
• modules/plugins/: models.py (PluginRun), tasks.py, plugin_interpreter.py, and runner/ (job.py for the row, plugin_process.py for the process, plugin_environment.py for the environment). Migration 0019 adds plugin_runs.
• The plugin's environment is built from scratch, as the protocol lists it. No SURFSENSE_LOCAL_ variable reaches it, and the API address comes from the worker's own SURFSENSE_LOCAL_HOST and SURFSENSE_LOCAL_PORT.
• worker.py plugins drains the queue with four threads; a fifth run waits as queued. Electron does not start this worker yet.
• -S hides the packages installed beside the plugin's Python. In development that is the backend's venv, or the CLI's under surfsense-plugins invoke. The runner, invoke and the SDK's own tests all use it, and the protocol, SDK and CLI docs say so.
• The runner takes the transaction's commit before the plugin starts, so the plugin's own writes through the API never wait on the worker's lock.
• Tests start real plugins through run_plugin(run_id):
  • a clean exit, a failing exit, and a crash that keeps what the plugin already wrote;
  • a note written through the real API, with its document_metadata;
  • nothing of the worker's environment reaching the plugin, and no API address;
  • a plugin's own json.py, the app's packages, and its own pid;
  • the inputs file, and the log tail's size;
  • a real worker.py plugins process running four at a time;
  • plugins/example end to end. That test skips unless invoke has installed the example's dependency.
• Backend tests now also run when plugins/core/sdk/** changes. data-model.md lists plugin_runs and the third queue.
• Not in this PR: stopping a run (timeout, cancel, runs left over at worker start, deleting a workspace), secrets, the routes, the Electron sidecar, and run events.

Follows #2081

High-level PR Summary

This PR implements the plugin runtime system that executes plugins as isolated processes. The worker spawns a plugin as python -S -m surfsense_plugin_sdk.run in its own environment, tracks its lifecycle through a new plugin_runs table (status from queued to running to succeeded/failed), and captures the last 16 KiB of output. Plugins communicate with the app through the existing API during execution, with the worker committing transactions before spawn to avoid deadlocks. A new plugins Huey queue with 4 worker threads handles concurrent runs, and -S isolates plugins from the backend's packages in development.

⏱️ Estimated Review Time: 30-90 minutes

💡 Review Order Suggestion

Order	File Path
1	docs/proposals/plugins/runtime/01-process.md
2	surfsense_local/backend/modules/plugins/models.py
3	surfsense_local/backend/alembic/versions/0019_plugin_runs.py
4	docs/architecture/data-model.md
5	surfsense_local/backend/shared/queue.py
6	surfsense_local/backend/modules/plugins/plugin_interpreter.py
7	surfsense_local/backend/modules/plugins/runner/plugin_environment.py
8	surfsense_local/backend/modules/plugins/runner/plugin_process.py
9	surfsense_local/backend/modules/plugins/runner/job.py
10	surfsense_local/backend/modules/plugins/tasks.py
11	surfsense_local/backend/worker/consumer.py
12	surfsense_local/backend/worker.py
13	surfsense_local/backend/shared/db.py
14	surfsense_local/backend/shared/config.py
15	surfsense_local/backend/modules/plugins/__init__.py
16	surfsense_local/backend/modules/plugins/runner/__init__.py
17	surfsense_local/backend/tests/integration/conftest.py
18	surfsense_local/backend/tests/integration/plugins/conftest.py
19	surfsense_local/backend/tests/integration/plugins/test_run_plugin.py
20	surfsense_local/backend/tests/integration/plugins/test_plugins_worker.py
21	surfsense_local/backend/tests/integration/plugins/test_run_the_example.py
22	surfsense_local/backend/tests/conftest.py
23	surfsense_local/backend/tests/integration/events/test_events.py
24	plugins/core/sdk/surfsense_plugin_sdk/run/__main__.py
25	plugins/core/sdk/tests/conftest.py
26	plugins/core/cli/surfsense_plugin_cli/invoke/plugin/run_plugin.py
27	plugins/core/cli/tests/invoke/test_running.py
28	plugins/README.md
29	docs/proposals/plugins/01-protocol.md
30	docs/proposals/plugins/02-extending.md
31	docs/proposals/plugins/cli/01-author-commands.md
32	docs/proposals/plugins/python/01-interpreter.md
33	docs/proposals/plugins/sdk/01-library.md
34	.github/workflows/desktop-tests.yml

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
surf-sense-frontend	Ready	Preview	Sep 30, 2026 5:49pm UTC

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: cc35af59-3054-46f1-8e65-26f81c8a4c43

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

• Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Autopilot is currently an internal CodeRabbit preview.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}