Skip to content

feat(web): update feed and usage endpoints for the desktop app - #2318

Merged
MODSetter merged 7 commits into
devfrom
feat/desktop-telemetry-endpoints
Oct 22, 2026
Merged

MODSetter merged 7 commits into
devfrom
feat/desktop-telemetry-endpoints

Conversation

@MODSetter

@MODSetter MODSetter commented Oct 22, 2026 •

Copy link
Copy Markdown
Owner

What

Adds the two website routes the desktop app's opt-in telemetry calls. They are safe to deploy on their own, before any app release uses them.

  • GET /api/updates/<file> (served at updates.surfsense.com):
    • Only stable.yml, stable-mac.yml, stable-linux.yml and stable-linux-arm64.yml are served; anything else, latest*.yml included, is a 404. Legacy 0.0.x clients stay entirely on GitHub.
    • It fetches the manifest from GitHub's latest release (cached 900 s) and checks it line by line. Every url:/path: is rewritten to the absolute versioned github.com/MODSetter/SurfSense/releases/download/v<version>/<name>, and sha512 is passed through untouched, so installer bytes always come from GitHub.
    • Anything unexpected answers 502, and the app then falls back to GitHub.
    • It records one personless desktop_update_check event: file, app version, platform, OS and country.
  • POST /api/usage (served at usage.surfsense.com):
    • Strict zod validation of schema 1: enums only, 4 KB cap, 415, 413 or 400 on anything off.
    • 204 on success, and one personless desktop_usage_report with the fields flattened.
  • Shared code in lib/desktop-telemetry/. Both routes never read the IP address, x-forwarded-for, User-Agent or cookies; country comes from x-vercel-ip-country only. Capture runs in after(), so PostHog never delays a response, and is a no-op until DESKTOP_TELEMETRY_POSTHOG_KEY is set.

The contract is docs/contracts/05-desktop-telemetry.md, which lands with the desktop telemetry PR.

Why

These routes let the founders count update-enabled installs per day, version adoption and platform mix. With usage statistics on, they also get bucketed feature, hardware and onboarding data. The app sends no identifier, and an update never depends on SurfSense's service being up.

Fixes

No linked issue.

How to test

  • cd surfsense_web && pnpm test:unit (includes 93 new tests: rewriting, hostile manifests, the allowlist, usage validation, no IP in captured properties)
  • After deploy: curl -i https://updates.surfsense.com/api/updates/stable.yml gives 200 with versioned github.com URLs; .../latest.yml gives 404; POSTing the contract's example body to https://usage.surfsense.com/api/usage gives 204.

Before it does anything in production

  1. Add updates.surfsense.com and usage.surfsense.com as domains on the Vercel project.
  2. Add GoDaddy CNAMEs for updates and usage pointing at Vercel's target (usually cname.vercel-dns.com).
  3. Set DESKTOP_TELEMETRY_POSTHOG_KEY in Vercel and redeploy. Recommended: turn on PostHog's "Discard client IP data" for that project.

Known limits: no rate limiting, so unauthenticated requests can inflate counts. The manifest allowlist is strict, so a new electron-builder key in a future manifest returns a 502 and apps fall back to GitHub until it is added.

High-level PR Summary

This PR adds two new API endpoints for opt-in desktop app telemetry: an update feed endpoint that serves versioned manifests from GitHub releases while recording update checks, and a usage reporting endpoint that accepts and validates schema-based usage statistics. Both endpoints are designed to be personless (no IP tracking, no user identification) and capture telemetry data via PostHog only after sending responses to avoid delays. The implementation includes strict validation, comprehensive error handling with GitHub fallback for updates, and 93 new unit tests covering manifest rewriting, security scenarios, and data privacy guarantees.

⏱️ Estimated Review Time: 30-90 minutes

💡 Review Order Suggestion
Order File Path
1 surfsense_web/lib/desktop-telemetry/desktop-dimensions.ts
2 surfsense_web/lib/desktop-telemetry/request-country.ts
3 surfsense_web/lib/desktop-telemetry/personless-capture.ts
4 surfsense_web/lib/desktop-telemetry/record-after-response.ts
5 surfsense_web/lib/desktop-telemetry/update-feed/channel-files.ts
6 surfsense_web/lib/desktop-telemetry/update-feed/latest-github-manifest.ts
7 surfsense_web/lib/desktop-telemetry/update-feed/pin-manifest-to-release.ts
8 surfsense_web/lib/desktop-telemetry/update-feed/serve-update-manifest.ts
9 surfsense_web/lib/desktop-telemetry/usage-report/usage-report-schema.ts
10 surfsense_web/lib/desktop-telemetry/usage-report/read-text-within.ts
11 surfsense_web/lib/desktop-telemetry/usage-report/usage-event-properties.ts
12 surfsense_web/lib/desktop-telemetry/usage-report/receive-usage-report.ts
13 surfsense_web/app/api/updates/[file]/route.ts
14 surfsense_web/app/api/usage/route.ts
15 surfsense_web/tests/unit/desktop-telemetry/fixtures/stable.yml
16 surfsense_web/tests/unit/desktop-telemetry/fixtures/stable-mac.yml
17 surfsense_web/tests/unit/desktop-telemetry/fixtures/stable-linux.yml
18 surfsense_web/tests/unit/desktop-telemetry/personless-capture.test.ts
19 surfsense_web/tests/unit/desktop-telemetry/sunset-passthrough.test.ts
20 surfsense_web/tests/unit/desktop-telemetry/update-feed.test.ts
21 surfsense_web/tests/unit/desktop-telemetry/usage-report.test.ts

Need help? Join our Discord

Summary by CodeRabbit

  • New Features
    • Desktop apps can retrieve update manifests for supported release channels. Update file links are tied to the release version, and temporary upstream issues return a clear error.
    • Desktop usage reports can be submitted when users opt in. Reports are validated before recording, and include limited usage details and country information without capturing IP addresses, user agents, or cookies.
    • Telemetry is configured without GeoIP enrichment or person-profile creation.

Sends one event per call with GeoIP off, no person profile and a fresh distinct id, after the response has gone. Does nothing until DESKTOP_TELEMETRY_POSTHOG_KEY is set.
Fetches the latest release's channel file from GitHub, pins every file to that release's download URL and answers 502 on anything unexpected, so the app falls back to GitHub. latest*.yml stays a 404. Each check records a personless desktop_update_check.
Validates schema 1 strictly (400 on an unknown key or value, 413 over 4 KB, 415 for anything but JSON) and records a flattened, personless desktop_usage_report.
Next answers HEAD with the GET handler, so uptime monitors and scanners were recorded as update checks. The app only GETs the channel file.
The old oversize body ended in a line the parser refuses anyway, so it passed without the cap. Repeated file entries straddling 64 KB now get 200 under the cap and 502 over it.
…etry key must live

The 15-minute cache limits our GitHub downloads to about 96 a day per channel file rather than removing them. The PostHog key must belong to the instance NEXT_PUBLIC_POSTHOG_HOST names, and pre-release builds count as invalid.
@vercel

vercel Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
surf-sense-frontend Ready Ready Preview Oct 22, 2026 7:30am UTC

Request Review

@coderabbitai

coderabbitai Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
📥 Commits

Reviewing files that changed from the base of the PR and between 6956302 and 8a12f3a.

📒 Files selected for processing: 21
  • surfsense_web/app/api/updates/[file]/route.ts
  • surfsense_web/app/api/usage/route.ts
  • surfsense_web/lib/desktop-telemetry/desktop-dimensions.ts
  • surfsense_web/lib/desktop-telemetry/personless-capture.ts
  • surfsense_web/lib/desktop-telemetry/record-after-response.ts
  • surfsense_web/lib/desktop-telemetry/request-country.ts
  • surfsense_web/lib/desktop-telemetry/update-feed/channel-files.ts
  • surfsense_web/lib/desktop-telemetry/update-feed/latest-github-manifest.ts
  • surfsense_web/lib/desktop-telemetry/update-feed/pin-manifest-to-release.ts
  • surfsense_web/lib/desktop-telemetry/update-feed/serve-update-manifest.ts
  • surfsense_web/lib/desktop-telemetry/usage-report/read-text-within.ts
  • surfsense_web/lib/desktop-telemetry/usage-report/receive-usage-report.ts
  • surfsense_web/lib/desktop-telemetry/usage-report/usage-event-properties.ts
  • surfsense_web/lib/desktop-telemetry/usage-report/usage-report-schema.ts
  • surfsense_web/tests/unit/desktop-telemetry/fixtures/stable-linux.yml
  • surfsense_web/tests/unit/desktop-telemetry/fixtures/stable-mac.yml
  • surfsense_web/tests/unit/desktop-telemetry/fixtures/stable.yml
  • surfsense_web/tests/unit/desktop-telemetry/personless-capture.test.ts
  • surfsense_web/tests/unit/desktop-telemetry/sunset-passthrough.test.ts
  • surfsense_web/tests/unit/desktop-telemetry/update-feed.test.ts
  • surfsense_web/tests/unit/desktop-telemetry/usage-report.test.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 8 remain after this review.

📝 Walkthrough

The change adds desktop update-manifest and usage-report endpoints. The endpoints validate requests, return responses, and schedule personless telemetry capture. Update manifests are fetched from GitHub and pinned to release URLs.

Changes: Desktop telemetry

Layer / File(s) Summary
Telemetry dimensions and capture
surfsense_web/lib/desktop-telemetry/desktop-dimensions.ts, surfsense_web/lib/desktop-telemetry/personless-capture.ts, surfsense_web/lib/desktop-telemetry/request-country.ts, surfsense_web/lib/desktop-telemetry/record-after-response.ts, surfsense_web/tests/unit/desktop-telemetry/personless-capture.test.ts, surfsense_web/tests/unit/desktop-telemetry/sunset-passthrough.test.ts
Adds platform, OS, app-version, and country parsing. Adds personless event capture and response-deferred recording. Tests cover capture properties and paths excluded from the configured matcher.
Update-manifest serving
surfsense_web/lib/desktop-telemetry/update-feed/*, surfsense_web/app/api/updates/[file]/route.ts, surfsense_web/tests/unit/desktop-telemetry/fixtures/stable*.yml, surfsense_web/tests/unit/desktop-telemetry/update-feed.test.ts
Adds channel filename checks, GitHub manifest retrieval, manifest validation and release URL pinning, and the update route. Tests cover supported and rejected files, responses, manifest contents, and telemetry fields.
Usage-report ingestion
surfsense_web/lib/desktop-telemetry/usage-report/*, surfsense_web/app/api/usage/route.ts, surfsense_web/tests/unit/desktop-telemetry/usage-report.test.ts
Adds a strict report schema, bounded body reading, flattened event properties, and the usage-report route. Tests cover valid reports, rejected input, body limits, and recorded properties.

Priority: ➖ Normal

Estimated code review effort: 4 (Complex) | ~40 minutes

Change: Feature

Sequence Diagram(s)

sequenceDiagram
  participant Updater as Desktop updater
  participant UpdateRoute as GET /api/updates/{file}
  participant Manifest as serveUpdateManifest
  participant GitHub as GitHub Releases
  participant UsageRoute as POST /api/usage
  participant Usage as receiveUsageReport
  participant PostHog as PostHog capture

  Updater->>UpdateRoute: Request update manifest
  UpdateRoute->>Manifest: Pass request and dependencies
  Manifest->>GitHub: Fetch latest-release manifest
  GitHub-->>Manifest: Return manifest text
  Manifest->>Manifest: Validate and pin release URLs
  Manifest-->>Updater: Return manifest or error response
  Manifest->>PostHog: Capture update-check event after response

  Updater->>UsageRoute: Submit JSON usage report
  UsageRoute->>Usage: Pass request and record callback
  Usage->>Usage: Read body and validate report
  Usage-->>Updater: Return response
  Usage->>PostHog: Capture usage event after response
Loading
Merge Risk: ⚪ Minimal · up to 8a12f

No identified issue blocks merging after normal checks.

Pre-merge checks: Passed 4 · Failed 1

Failed Failed checks: 1 warning

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 59.26% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 27 functions across 18 files. (3 skipped:… Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed Passed checks: 4 passed
Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title clearly and concisely identifies the two main changes: update-feed and usage endpoints for the desktop app.
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage

Explanation

Docstring coverage is 59.26% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 27 functions across 18 files. (3 skipped: 3 unsupported.)

Resolution

Write docstrings for the functions missing them to satisfy the coverage threshold.

✨ Finishing Touches: 💡 2
📝 Generate docstrings: 💡
  • Commit to this branch
  • Create a new PR
🛠️ Fix failing CI checks: 💡
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@MODSetter
MODSetter merged commit 647d268 into dev Oct 22, 2026
28 of 31 checks passed

This branch was successfully deployed

1 active deployment
Preview — 8a12f3ad Deployed Oct 22, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant