Repository navigation
fix(plugins): get the plugin host ready for plugins.surfsense.com - #2249
Merged
Merged
Conversation
…s the backend does keygen.py held Keygen Cloud's address and an empty account id, so the host could not start with a paid plugin, and could not reach the self-hosted Keygen CE that holds production's licenses. It now reads KEYGEN_API_URL (the accounts endpoint, Keygen Cloud's when unset), KEYGEN_ACCOUNT_ID and KEYGEN_HOST, with the backend's names and meanings, so a deployment gives both the same values. With KEYGEN_HOST set it sends Host and X-Forwarded-Proto: https, as CE needs. A 4xx or a redirect from Keygen refuses the key as invalid, with a warning in the log, instead of failing the request with a 500. A reply with no code is no answer, so the 24-hour grace still applies.
…t makes Scrapling logs a failed proxy with its user and password, so a deployed host would write the proxy vendor's credentials into its logs. python -m host now redacts each record as it is made, its message, arguments and traceback: a URL's user:password@, the captcha solvers' user:password@host:port form, and a password or a solver's key as a field.
…y list Run against the live sites on 2026-10-13, Reddit refused every anonymous .json request, even from inside the warmed browser, and Instagram's profile endpoint asked for a login on every IP. Both plugins stay on the host; the app lists the other eight. The tests that used Reddit as their sample SurfSense plugin use Amazon now.
The entry was verified with no domain, so registry_check static failed, which broke the registry's pull request check and stopped the sign job before it signed. No ownership proof exists: the MCP Registry holds no DeepWiki listing and Cognition publishes no SurfSense file or DNS record, so the registry's rule makes it Community.
…eight listed plugins plugin-host.md describes the Keygen settings and answers, log redaction and the new tests, and replaces the empty-account-id gaps with Reddit's and Instagram's. .env.example gains the KEYGEN_* settings and names the eight plugins in PLUGINS.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Note Currently processing new changes in this PR. This may take a few minutes, please wait... ⚙️ Run configuration
📒 Files selected for processing: 27
✨ Finishing Touches
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
This was referenced Oct 14, 2026
This was referenced Oct 14, 2026
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Gets the plugin host ready to deploy at
plugins.surfsense.comfor 2.2.1.What changes
The license check reaches production's Keygen.
keygen.pyhad Keygen Cloud's address and an empty account id hard-coded, so the host could not start with any paid plugin, and it could not reach the self-hosted Keygen CE that holds the licenses. It now reads the backend's own settings, with the same names and meanings, so the deployment gives both the same values:KEYGEN_API_URLhttp://keygen-web:3000/v1/accounts(Keygen Cloud's when unset)KEYGEN_ACCOUNT_IDKEYGEN_HOSTkeygen.internal, sent asHostwithX-Forwarded-Proto: httpsA 4xx or a redirect from Keygen now refuses the key as
403 invalid, with a warning in the log, instead of failing the request with a 500. A reply with no code counts as Keygen being unreachable, so the 24-hour grace still applies.Credentials never reach the logs. Scrapling logs a failed proxy as
Proxy 'http://user:password@gw.dataimpulse.com:823' failed.python -m hostnow redacts every log record as it is made: message, arguments and traceback. That covers URL userinfo, the captcha solvers'user:password@host:portform, andpassword/ solverkeyfields. A local run against the live sites wrote 8 such lines before this change.Reddit and Instagram are left out of the app's first-party list. Testing the plugins against the live sites on 2026-10-13 found:
.jsonrequest, even from inside the warmed browser.The code stays. The list offers the other eight,
.env.example'sPLUGINSnames those eight for the deployment, and the backend tests that used Reddit as their sample SurfSense plugin use Amazon now.DeepWiki is listed as Community. The entry was
verifiedwithout adomain, soregistry_check staticfailed. That broke the registry PR check and stopped the sign job before it signed anything. No ownership proof exists: the MCP Registry has no DeepWiki listing, and Cognition publishes no SurfSense file or DNS record. So by the registry's rule it iscommunity.Tests
Run locally on Windows:
tests/plus everytest_<plugin>_plugin.pygave 114 passed and 2 skipped (Redis). License check and redaction: 30 passed.ruff check ..andruff format --check ..are clean.static check: passed, and 76 passed. 2test_icon_checkcases error on Windows only, because their test id is longer than an environment variable can hold. They run on Linux CI.tests/packaging/test_spec_data_files.pygave 7 passed, and ruff is clean.Not in this PR
plugins.surfsense.comDNS record (GoDaddy), and theREGISTRY_SIGNING_KEYsecret.user_search.High-level PR Summary
This PR prepares the plugin host for production deployment at
plugins.surfsense.comfor version 2.2.1. It fixes the license check to reach the production Keygen instance by reading settings from the environment (matching the backend's configuration), adds credential redaction to prevent passwords and API keys from appearing in logs, removes Reddit and Instagram from the first-party plugin list due to anonymous access issues, and corrects the DeepWiki registry entry fromverifiedtocommunitystatus.⏱️ Estimated Review Time: 30-90 minutes
💡 Review Order Suggestion
plugins/remote/host/.env.exampleplugins/remote/proprietary/license_check/keygen.pyplugins/remote/host/redacted_logs.pyplugins/remote/host/__main__.pyplugins/remote/proprietary/license_check/tests/stand_in_keygen.pyplugins/remote/proprietary/license_check/tests/test_license_access.pyplugins/remote/host/tests/test_redacted_logs.pyplugins/remote/proprietary/license_check/tests/sample_plugins/proprietary/paid/server.pysurfsense_local/backend/modules/plugins/lists/first_party/plugins.jsonplugins/registry/plugins.jsondocs/architecture/plugin-host.mddocs/architecture/plugins.mdplugins/remote/proprietary/reddit/tests/test_reddit_plugin.pyplugins/remote/proprietary/instagram/tests/test_instagram_plugin.pyplugins/remote/proprietary/youtube/tests/test_youtube_plugin.pyplugins/remote/proprietary/tiktok/tests/test_tiktok_plugin.pyplugins/remote/proprietary/google_maps/tests/test_google_maps_plugin.pyplugins/remote/proprietary/google_search/tests/test_google_search_plugin.pyplugins/remote/proprietary/amazon/tests/test_amazon_plugin.pyplugins/remote/proprietary/walmart/tests/test_walmart_plugin.pyplugins/remote/proprietary/indeed/tests/test_indeed_plugin.pyplugins/remote/proprietary/web_crawl/tests/test_web_crawl_plugin.pysurfsense_local/backend/tests/integration/agent/test_plugin_tools.pysurfsense_local/backend/tests/integration/plugins/routes/test_first_party_plugins.pysurfsense_local/backend/tests/integration/plugins/routes/test_registry.pysurfsense_local/backend/tests/mcp_test_server.pysurfsense_local/backend/tests/unit/plugins/lists/test_first_party_list.pySummary by CodeRabbit