Skip to content

Report anonymous automation-created and split-view-opened events - #5316

Open
brsbl wants to merge 3 commits into
bb/tipsfrom
bb/telemetry-automation-split-view-thr_gqshddt6w2
Open

brsbl wants to merge 3 commits into
bb/tipsfrom
bb/telemetry-automation-split-view-thr_gqshddt6w2

Conversation

@brsbl

@brsbl brsbl commented Oct 10, 2026 •

Copy link
Copy Markdown
Collaborator

Human comments

What was wrong

bb couldn't tell whether people create automations or use split view, so neither could be compared with retention, and the Tips feed couldn't rank tips for them. Neither action sent a usage event.

What changed

  • Allow-list: automation_created and split_view_opened are added to recordTelemetryEventRequestSchema (packages/server-contract/src/api/system.ts). Their properties are strict enums and a bounded count:
    • automation_created: trigger (schedule or once), mode (agent or script), origin (human, app, or agent).
    • split_view_opened: panes, an integer from 2 to 8 (MAX_PANES).
  • Automations: the bundled plugin's server create path (plugins/automations/src/service.ts, helper in src/telemetry.ts) reports the event through bb.sdk.system.experimental_recordTelemetryEvent, the same SDK path Tips uses. This is the only place that writes a new automation, so the RPC and the CLI report once each; legacy imports don't report. Telemetry errors are swallowed and never affect creation.
  • Split view: a watcher on splitLayoutAtom (useSplitViewTelemetry, mounted in AppLayout) reports when the pane count grows. Every surface that adds a pane (menu, palette, drag, nav rail, server open-in-split) is covered once. Closing panes and restoring a saved layout don't report.
  • Opt-out: unchanged. The server's telemetry service still drops every event when Share anonymous usage data is off or BB_TELEMETRY=false.
  • Docs:
    • The privacy page lists both events.
    • The docs/configuration.md telemetry list includes them.
    • The docs/api_to_audit.md entry for experimental_recordTelemetryEvent now lists every core and bundled-plugin caller.
    • The Automations overview has a "Usage data" section.
  • Versions: Plugin SDK 0.6.40, because the public RecordTelemetryEventRequest type grew. Automations now requires bbPluginSdk >=0.6.40.

Stack: this PR depends on #5222's telemetry route and SDK method, and sits on top of #4991 because both extend the same allow-list, privacy copy and docs.

How you verified

Privacy page, telemetry paragraph (Chrome for Testing, local apps/web dev server, 2×):

Before (#4991 head) After (this PR)
Privacy telemetry paragraph before Privacy telemetry paragraph after
  • Added a server route test: both events are accepted; extra properties, unknown enums, and out-of-range pane counts get 400, and nothing is captured for them.
  • Added an Automations harness assertion: one automation_created event per successful create, and none for rejected creates.
  • Added a split-view hook test: one event per added pane (2, then 3), and none for closing a pane or restoring a saved split layout.
  • Ran the branch web app in Chrome for Testing:
    • Cmd-clicking the Automations nav item opened a split and sent exactly one split_view_opened with { panes: 2 }, which the server accepted (200).
    • Reloading with the split restored sent nothing.
  • Slop Cop review at 8854feb found one P1: the Automations telemetry call didn't typecheck against the service's narrow SDK type. It's fixed in 83db9dd.
  • Two P2 follow-ups are left out of this PR:
    • Watch a derived pane-count atom instead of the whole layout, so AppLayout doesn't re-render on resize or focus.
    • Add a test tying the schema's pane cap to MAX_PANES.
  • Remote CI is green.

BB-Thread: Retention measurement

🤖 Generated with Claude Code

AGENT GENERATED

Adds automation_created (trigger, mode, origin) and split_view_opened
(pane count after opening, 2-8) to the client telemetry allow-list.
Automations emits from its server create path through the plugin SDK's
system.experimental_recordTelemetryEvent, so the RPC and CLI paths report
once each and failures never affect creation. The app reports split view
from one watcher on the split layout, so every surface that adds a pane
reports it and restoring a saved layout does not. The server still drops
both when usage data sharing is off. Privacy copy, configuration docs, the
API audit entry, and the Automations overview list the new events.
Plugin SDK 0.6.40.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

@brsbl
brsbl added this pull request to stack #5232 October 10, 2026 02:40
brsbl and others added 2 commits October 9, 2026 19:50
Automations' service receives a narrow ServiceApi, so the telemetry helper
and that type now name system.experimental_recordTelemetryEvent explicitly,
and the hand-built test SDKs stub it. Formats the PR's files with oxfmt.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant