Repository navigation
feat(telemetry): stamp wizard_ask outcome events with the ask subject - #1162
Merged
Merged
Conversation
The `wizard_ask answered` / `wizard_ask cancelled` events carried the run `source` but not the per-call `subject` (the thing the questions collect for, e.g. a data-warehouse source kind). The warehouse task asks one `wizard_ask` call per detected source, so without the subject a cancellation can be seen at the run level but not attributed to which source's credentials the user declined — the single largest structured signal in this flow cannot be broken down by the very dimension the funnel is analysed on. Thread the subject (already computed for the cap accounting, and already on the `wizard_ask capped` event) through the ask bridge onto both outcome events, normalised via the same `normaliseAskSubject` so the values join across the three events. Purely additive: no behaviour change, both harness facades (MCP + pi) updated symmetrically. Generated-By: PostHog Desktop Task-Id: c1dc9e8e-929c-49eb-988c-054298a47462
🧙 Wizard CIRun the Wizard CI and test your changes against wizard-workbench example apps by replying with a GitHub comment using one of the following commands: Test all apps:
Test all apps in a directory:
Test an individual app:
Show more apps
Test against a Context Mill branch:
Add Results will be posted here when complete. |
edwinyjlim
approved these changes
Aug 28, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The
wizard_ask answered/wizard_ask cancelledevents — the primary signalfor how users respond to the in-CLI credential prompt — carried the run
sourcebut not the per-callsubject(the thing the questions collect for,e.g. a data-warehouse source kind). The seeded warehouse task asks one
wizard_askcall per detected source, so a cancellation was visible at therun level but could not be attributed to which source's credentials the user
actually declined.
That is the gap that makes the flow's dominant failure mode hard to act on: the
largest structured signal (users cancelling the credential prompt) cannot be
broken down by the per-source dimension the flow is analysed on, even though the
subject is already known at the call site and already stamped on the sibling
wizard_ask cappedevent.Changes
subjecttoWizardAskRequestand stamp it onto bothwizard_ask answeredandwizard_ask cancelledin the ask bridge.wizard_askand the pinative mirror), normalised through the same
normaliseAskSubjectthe capaccounting uses, so the value joins cleanly to
wizard_ask capped.Purely additive — no behaviour change, and the two facades are updated
symmetrically so they can't drift.
Why
The warehouse-connect telemetry breaks outcomes down per provider, but the
cancellation event that drives most of the loss was provider-blind, so the two
couldn't be joined. This closes that gap in the same spirit as the earlier
reason-on-skip change.Test plan
wizard-ask-bridgeunit tests to assertsubjectis stamped on boththe answered and cancelled events.
pnpm build && pnpm test(ask-bridge + wizard-tools + pi harness suites) andpnpm lintpass (0 errors).Created with PostHog Desktop