Skip to content

feat(privacy): disclose the dependency scan and let users decline sharing - #1099

Merged
MattBro merged 9 commits into
mainfrom
matt/wizard-scan-disclosure
Aug 31, 2026
Merged

MattBro merged 9 commits into
mainfrom
matt/wizard-scan-disclosure

Conversation

@MattBro

@MattBro MattBro commented Aug 18, 2026 •

Copy link
Copy Markdown
Contributor

Problem

The wizard scans dependency manifests and .env variable names to find third-party tools in a project. It reports what it finds to PostHog. The report includes competitors such as Sentry, Mixpanel and Amplitude.

This creates two problems. The wizard does not tell the user that it does this. The user cannot stop it.

The scan also runs first, before the intro screen and before login. No existing consent surface covers it. The OAuth screen is the login itself. The organization AI toggle covers source code sent to Anthropic. Neither surface can move earlier: one is the login, and the other reads a field from an organization the user has not logged into yet.

Changes

The intro screen now states what the wizard reads and why. A new menu option, Continue, don't share tools, sits directly under Continue, so the first press of the down arrow selects it.

A decline stops the report only. The scan still runs on the machine. A user who declines receives the same outro suggestion, the same setup report line, and the same warehouse task as any other user. Only the telemetry stops.

Consent has three states instead of two: undecided, granted, and declined. Undecided consent sends nothing. A future code path that reports before the wizard asks will therefore send nothing instead of everything.

  • A CI run resolves to granted when the session is built. No screen can ask a question there, and a scripted run is the user's own automation.
  • A --signup run does not resolve to granted. That flag creates a new account with no screens, and that user saw no disclosure.
  • A --ci --signup run resolves to granted, because --ci covers it.

One function converts scan results into telemetry. It runs when consent resolves, and a second call makes no further change.

When consent is granted and the scan finds an AI SDK, the wizard also sets wizard_ai_sdk_detected: true on the organization group. An AI SDK is an entry in the registry's LLM or AI section, or the LLM feature from dependency discovery. The value is a boolean. It never carries the list of kinds, and it never names a tool that is not an AI SDK. The wizard does not write it on a decline, and it never writes false, so an absent value means the wizard reported nothing. The wizard derives the AI kind set from the registry instead of holding a copy. Only wizard versions that carry this disclosure write the stamp, so anything downstream that scores on it is post-consent by construction. Scan results reported before this change stay unused.

The privacy panel held a wrong statement. It said that .env* files stay on your machine. The values do stay on the machine, but a variable name alone can produce a detection that the wizard sends. MIXPANEL_TOKEN in a .env file reports Mixpanel even when the project has no Mixpanel dependency. The panel now states this.

A decline sends no wizard: warehouse sources detected event. That event means "the scan ran, and here is what it found". Six saved insights depend on that meaning, and two of them apply no property filter. A row for a decline would give both a wrong result. wizard: intro menu selected reports the decline rate instead.

Test plan

Run the wizard against a project that holds a recognised tool, for example a package.json that lists stripe. Select Continue, don't share tools. The outro still offers to connect the detected source. No wizard: warehouse sources detected event arrives. setup wizard finished carries no warehouse_source_kinds tag and no discovered_features property.

New tests cover these cases:

  • Undecided consent reports nothing.
  • Granted consent reports the original property shape.
  • Declined consent reports nothing, and still fills the detected sources for local use.
  • A --ci session builds as granted. A --signup session without --ci builds as undecided.
  • A second call to the report function makes no further change.

The group stamp has its own tests. It is written for an AI kind, and for the LLM feature alone. It is not written on a decline, on undecided consent, on non-AI kinds only, or without an organization id.

All twelve intro screens share one centered menu box of 24 columns, and that width broke the new label inside a word. IntroScreenLayout now accepts an optional width. Only this screen passes one, so the other eleven do not change. A test measures every label on this menu against that width, so a later addition fails in the test suite instead of in a screenshot.

Follow-ups, not in this change

  • The self-driving program runs the same scan through detectConnectedTools. It reports the result with no disclosure and no way to decline. It detected Sentry in 292 runs over the last 30 days. It needs the same treatment, and it has its own intro screen.
  • The standalone wizard warehouse command also reports detected sources with no consent check.
  • Four call sites apply the consent check by hand today. A test that fails when a new reporting site skips the check would hold this in place.

…ring

The wizard scans dependency manifests and .env variable names to identify
third-party tools in a project, then reports what it finds to PostHog. That
happened with no disclosure and no way to opt out.

The intro screen now says what is read and why, and offers "Continue without
sharing what you use". Declining suppresses reporting only: the scan still
runs locally, so the outro suggestion, the setup report line, and the
warehouse task behave identically for decliners.

Consent is tri-state rather than a boolean. An unresolved consent sends
nothing, so a future collection path that reports before asking fails quiet.
CI and signup runs resolve to granted at session build, since no screen can
ask and a scripted run is the user's own automation.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016EXMTphgZGARMRdRw1yeFk
@github-actions

Copy link
Copy Markdown

🧙 Wizard CI

Run the Wizard CI and test your changes against wizard-workbench example apps by replying with a GitHub comment using one of the following commands:

Test all apps:

  • /wizard-ci all

Test all apps in a directory:

  • /wizard-ci ai-observability
  • /wizard-ci basic-integration
  • /wizard-ci mcp-analytics
  • /wizard-ci revenue
  • /wizard-ci self-driving

Test an individual app:

  • /wizard-ci ai-observability/anthropic
  • /wizard-ci ai-observability/groq
  • /wizard-ci ai-observability/manual-capture
Show more apps
  • /wizard-ci ai-observability/openai
  • /wizard-ci ai-observability/openai-agents
  • /wizard-ci ai-observability/vercel-ai
  • /wizard-ci basic-integration/android
  • /wizard-ci basic-integration/angular
  • /wizard-ci basic-integration/astro
  • /wizard-ci basic-integration/django
  • /wizard-ci basic-integration/fastapi
  • /wizard-ci basic-integration/flask
  • /wizard-ci basic-integration/javascript-node
  • /wizard-ci basic-integration/javascript-web
  • /wizard-ci basic-integration/laravel
  • /wizard-ci basic-integration/next-js
  • /wizard-ci basic-integration/nuxt
  • /wizard-ci basic-integration/python
  • /wizard-ci basic-integration/rails
  • /wizard-ci basic-integration/react-native
  • /wizard-ci basic-integration/react-router
  • /wizard-ci basic-integration/sveltekit
  • /wizard-ci basic-integration/swift
  • /wizard-ci basic-integration/tanstack-router
  • /wizard-ci basic-integration/tanstack-start
  • /wizard-ci basic-integration/vue
  • /wizard-ci mcp-analytics/custom-dispatcher
  • /wizard-ci mcp-analytics/typescript-sdk
  • /wizard-ci revenue/stripe
  • /wizard-ci self-driving/astro
  • /wizard-ci self-driving/fastapi
  • /wizard-ci self-driving/nuxt
  • /wizard-ci self-driving/react-router
  • /wizard-ci self-driving/sveltekit

Results will be posted here when complete.

…yout

Review follow-ups on the disclosure change.

The consent check was inlined at three call sites that disagreed about
whether 'undecided' and 'declined' behave the same. It is now one predicate,
following the aiApproved() pattern. sessionProperties() no longer knows what
consent means: it calls an accessor that lives beside the field it governs,
so the branch stops silently affecting six unrelated capture sites.

A scan failure was reported as a clean zero result. Splitting the scan from
the reporting meant a thrown error left the same empty state as a project
with no detectable tools, so crashes landed in the denominator as successful
scans. Failures are now distinguishable and skip the capture, which is what
happened before the split.

The opt-out label wrapped mid-word because the centered menu box is a fixed
24 columns shared by every intro screen. IntroScreenLayout takes an optional
width so this screen can opt into a wider menu without changing the other
eleven. The disclosure paragraph was unbounded and spanned the terminal
while everything around it was centered.

The subtitle claimed .env* contents stay local while the paragraph said .env
variable names are read and shared. Both were true and together they read as
a contradiction, so this screen now states the distinction once.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016EXMTphgZGARMRdRw1yeFk
@MattBro
MattBro marked this pull request as ready for review August 18, 2026 17:23
@MattBro
MattBro requested a review from a team as a code owner August 18, 2026 17:23
MattBro and others added 4 commits August 18, 2026 13:24
Half the added lines in two files were prose. Trimmed to the reasons a
reader cannot get from the code, and dropped the rest.

Merges main, which had moved five commits ahead, including changes to
warehouse source handover and .env reading.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016EXMTphgZGARMRdRw1yeFk
A --signup run creates a brand-new account headlessly, and that user
has never seen the disclosure screen. Only --ci still auto-grants: a
headless --ci --signup run stays covered by the ci branch.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NMkv2CLet2YqY6XxRqfSiy
…d scans

Adds Analytics#groupIdentify, mirroring capture()'s structure. Note:
capture()/wizardCapture() carry no telemetry enabled/disabled guard
today (--no-telemetry only gates the headless task stream), so there
is nothing to mirror there.

Splits the warehouse-source registry's LLM/AI section into its own
array so AI_SOURCE_KINDS derives from the same entries instead of a
hand-copied list, then stamps the org group with a boolean only once
consent is granted and an AI kind (or a discovered LLM feature) shows
up. Never the list of kinds, never other tools, never false or absent
on decline/undecided.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NMkv2CLet2YqY6XxRqfSiy

ctx.setFrameworkContext(DETECTED_WAREHOUSE_SOURCES_KEY, sources);
} catch (error) {
ctx.setFrameworkContext(WAREHOUSE_SCAN_FAILED_KEY, true);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Low: Scanner errors bypass consent

Detection runs while consent is undecided, but the scanner directly calls analytics.captureException for malformed package.json files and unexpected filesystem errors before this wrapper returns. A malicious repository can supply a malformed or symlinked manifest and cause repository-derived parser text to be transmitted before the user can decline; Node's JSON parse errors include an excerpt of the rejected input. Return a sanitized failure state or log locally during detection, and only capture it after the consent check.

@veria-ai

veria-ai Bot commented Aug 27, 2026

Copy link
Copy Markdown

PR overview

This PR adds disclosure of dependency scanning and lets users decline sharing scan-related information through the PostHog integration.

One security issue remains open: dependency detection can report malformed manifest or filesystem errors before consent is resolved. A crafted repository could therefore cause repository-derived parser text to be sent despite the user later declining sharing.

Open issues (1)

Fixed/addressed: 0 · PR risk: 5/10

@edwinyjlim edwinyjlim left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MattBro i trust you to take it the rest of the way

@MattBro
MattBro enabled auto-merge (squash) August 31, 2026 13:40
@MattBro
MattBro merged commit dc65104 into main Aug 31, 2026
17 checks passed
@MattBro
MattBro deleted the matt/wizard-scan-disclosure branch August 31, 2026 13:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants