Skip to content

feat(transactional-email): add a command that sets up the first email send - #1507

Open
Silthus wants to merge 18 commits into
mainfrom
feat/transactional-email-program
Open

Silthus wants to merge 18 commits into
mainfrom
feat/transactional-email-program

Conversation

@Silthus

@Silthus Silthus commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

Moved from Silthus#1.

Problem

A team that has never sent email from code has no guided way to start with PostHog. Wizard can add PostHog to a project, but the setup agent has no path to send a transactional email, such as a welcome email after signup.

With this PR, npx @posthog/wizard transactional-email checks for a verified sender, writes the email in the app's code (or as a PostHog template, when the person picks that), writes the project secret API key to the env file, and adds the first send to the app's server code. A plain run that finds PostHog already installed lists the command in its intro spell book.

Part of Silthus/posthog#544.

Merge order

Done up to the last step:

  1. ✅ PostHog/posthog#127123 merged.
  2. ✅ The SDK releases of PostHog/posthog#128870's merge order, such as posthog-js#5509 and posthog-python#1171.
  3. ✅ workflows-email-from-code is at 100%.
  4. ✅ PostHog/posthog#128870 merged on 2026-11-13.
  5. PostHog/context-mill#456 merges and is released.
  6. This PR merges.

Warning

🟡 Merge risk: medium · one-way door once a Wizard release publishes the transactional-email command name, a public command that changes people's apps and sends email

  • Every linear Pi program with abort cases now ends on its [ABORT] line, as the Anthropic binding does: audit, migration, revenue-analytics, warehouse-source, warehouse-destination, self-driving, mcp-analytics, web-analytics-doctor and error-tracking-upload-source-maps. Today Pi ignores that line, and such a run carries on to a nudge, then the success outro or open tasks. Prod shape: every linear Pi abort of these programs in 30 days was a user cancel, and the Anthropic binding of the same programs has not ended a run on a matched [ABORT] since September, so few runs should move. The failure path is a run that prints an [ABORT] line by mistake and now ends on the error outro. A reply to the closing remark question never ends a run.
  • After login, an agent's [ABORT] now shows its own outro (message, body, docs link) instead of "The Wizard's a little busy" handoff, on every program and harness. A security stop already did. The run still ends with exit code 1 and analytics error. Prod shape: in 30 days the only busy handoff after a program's [ABORT] was this PR's own walk; every other busy handoff keeps its screen. The same routing also moves two uncovered rows that never reach an agent: a stop before the run, such as "AI processing approval declined.", and the unsupported-platform stop of error-tracking, feature-flags and replay-vision. They now show their own outro, which names the manual setup docs, instead of the busy handoff, still with exit code 1. Residual incident: cosmetic, a wrong message on a stop that already happened.
  • Released before PostHog/context-mill#456 is released, the command fails when it downloads its skill. Hence the merge order.
  • New command, covered at this head by the live walks: an Express app gets its email code, the key in .env through set_env_values, and a first welcome email that a signup sends, delivered. A frontend-only app ends on "No server code found" before any question. Prod shape: about a third of Wizard runs with a known framework have no server code and take that outro.
  • New command, uncovered: a project with no verified sender yet, the common case in prod shape; the template path; a missing PostHog tool or a failing wizard_ask. Residual incident: the run stops or waits on the person, or the first send answers an error, visible in the run and its report. The key is written only to the app's env file, and a send goes only to an address the person gives after a yes: no row sends silently.
  • The transactional email API flags are at 100% in US, so the [ABORT] transactional email api not enabled outro reaches no US project today.
  • migrated_from is a new property on the events of a run whose skill ID is migrate-email-<provider>, setup wizard finished included. Other runs send the same events as before.
  • Switch-off: none, the change has no flag. Fix forward with a Wizard release; a revert removes the command, the spell book row, the property and the Pi [ABORT] handling.

Tier: critical (c, f: a Wizard release publishes the command name)

Impact

Moves weekly orgs outside PostHog's own org that send email from code, on the Code-sending orgs by client insight: the Wizard's runs show up as their own client after a release. It rolls up to Workflows revenue through billed workflows_emails.

Changes

  • New program transactional-email (src/programs/transactional-email/), built with createSkillProgram. It names its binding: Pi, the linear sequence, GPT-5.6 Sol at medium thinking, the same as the default today.
  • New command transactional-email (nativeCommandFactory), registered in runCli.
  • The intro spell book of a plain run lists it last: transactional-email Send your first transactional email from code.
  • Two abort cases match the skill's "Inside the PostHog Wizard" section: the API is not enabled, and no server code was found. Each error outro links https://posthog.com/docs/workflows/transactional-emails.
  • Pi [ABORT] handling (src/agent/runner/harness/pi/index.ts, 287887f4, 226aad62, 904543aa): on a program with abort cases, a reply line that starts with [ABORT] ends the linear Pi run with its reason, with no nudge, and the linear sequence shows the matching outro. A reply that only mentions the marker mid-line keeps running, and a reason on the line after a bare marker still counts. The gateway refusal marker keeps its own path and its gateway_policy source. A reply to the closing remark question is ignored.
  • Abort outro after login (src/tui/mint-failure.ts, 60445c76): an error outro with PHW_AGENT_ABORT or a matched case's PHW_DETECT_UNSUPPORTED_PLATFORM is a decision, like a security stop, so the TUI shows it instead of the busy handoff.
  • No extra OAuth scopes. The PostHog MCP hides tools whose scopes the login lacks: the integrations-* sender tools and the workflows-* email tools. The skill's Wizard section treats a missing tool as inconclusive, sends the person to the PostHog UI to verify a sender, and takes only a send's 404 as "API not enabled".
  • The program runs sending-transactional-emails. Its prompt asks for a verified sender, the email, the project secret API key, and the first send. The skill owns where the email lives: in the app's code by default, or a PostHog template when the person picks that.
  • README: a "Transactional email" section. The email goes in your code by default, and the send uses PostHog's SDK for the backend's language, or the HTTP API.
  • migrated_from on setup wizard finished (src/services/session/session-store.ts): when a run's skill ID is migrate-email-<provider>, such as migrate-email-resend, the session sets the analytics tag migrated_from to the provider. Every event after the run picks its skill carries it. A later skill, or none, clears it. No new event.

The skill's Wizard section is in PostHog/context-mill#456. A test there checks that the built skill emits only the two [ABORT] reasons this program matches.

This revision (2026-11-16, 7c9de7e4): 04b79150 merges main (the support command landed next to this one), 287887f4 makes Pi end on a program's [ABORT], b09af2e4 replaces tests that read the program's config with tests through the harness and the analytics client, and 226aad62 and 904543aa take only a line that starts with the marker, from round 2 and the final read, 60445c76 shows an agent's abort on its outro, from the live walk, and 7c9de7e4 makes the abort tests assert the run's outcome instead of prompt counts.

Test plan

  • Pi [ABORT] (src/agent/runner/harness/pi/__tests__/structured-run.test.ts): a linear Pi run with an open task whose reply holds the line [ABORT] no server code found returns an abort with that reason. So does the reason in its own text block, or on the line after a bare [ABORT]. A mid-line mention, or the same line in the remark reply, leaves the run a success. A program without abort cases runs on and returns its open-tasks failure. Each fails without the code it guards.
  • migrated_from (src/services/session/__tests__/migrated-from.test.ts): the test fakes posthog-node and reads the setup wizard finished event the analytics client sends. A migrate-email-resend run carries migrated_from: resend. A migrate-statsig run, and a run that moves on from migrate-email-resend to migrate-statsig or to no skill, carry none.
  • Abort outro (src/tui/__tests__/router.test.ts): after login, an error outro with PHW_AGENT_ABORT or PHW_DETECT_UNSUPPORTED_PLATFORM resolves to the outro, then the exit once dismissed. Both fail on 904543aa.
  • Intro spell book (intro-menu.test.ts): the list ends with transactional-email, and the row fits 80 columns.
  • Bindings and flow: program-registry.test.ts pins the linear Pi binding. The flow-traces snapshot records intro, health check, auth, run, and outro for the program.
  • CI runs the full gate on this head.

Proofs:

  • Live run (2026-11-16, on 7c9de7e4, Wizard TUI host walks in project 2 through the CI gateway, skill served by the dev server of PostHog/context-mill#456 at b58090a2; no key value in any output):
    • Code path, first send. Fixture: an Express app on posthog-node with a TODO for a welcome email at signup.
      • The first question batched which email to send with "Keep the email in your code (recommended) or design it in PostHog?". Then the verified sender, branding, the project secret key (sensitive, with the "Send transactional email" preset), and "Send the first welcome email now?".
      • The run completed every task: analyze, sender, email, API key, engagement, test, verify, report. It wrote src/emails/welcome.js and src/posthog.js, and a send at signup through posthog.emails.send with secretKey, the template key welcome, the idempotency key welcome:<user id> and the user's distinct ID.
      • The harness answers free text with a placeholder, so the skill's own test send was skipped. A signup on the fixture app then sent the first email through the generated code: posthog.emails.get answered delivered 10 s later, and "Welcome to Notes" is in the PR author's inbox.
    • No server code (frontend-only Vite and React fixture): the run ends on "✘ No server code found", its body and the docs link, with exit code 1, before any question. The fixture is unchanged. On 904543aa this run showed "The Wizard's a little busy"; 60445c76 fixed that.
    • The 2026-11-05 walks on f95c37b1 also covered the template path.
  • Prod shape (2026-11-14 and 2026-11-16, read only, in the Wizard analytics project and US prod; the numbers stay in the private proof store): every linear Pi abort of a program with abort cases in 30 days was a user cancel; the only busy handoff after a program's [ABORT] was this PR's walk; about a third of Wizard runs with a known framework have no server code; most projects have no email sender, no engagement events and no transactional template yet; both API flags are at 100% in US.
  • Cross-version (gap): migrated_from is one added string property on existing events, which ingestion stores as is.
  • Post-deploy watch: owed after merge and a Wizard release: agent aborted with a matched reason on Pi, by program.

Decisions

Question Answer Who Where
Where the email lives by default In the app's code; a PostHog template when the person picks it, asked once by the skill 👤 Human Map conductor, 2026-11-04
When this merges After workflows-email-from-code reaches 100% and the skill is released 👤 Human Map conductor, 2026-11-04
Which skill transactional-email runs sending-transactional-emails from PostHog/posthog, published to the menu by PostHog/context-mill#456 👤 Human Silthus/posthog#461
Offer transactional email setup in Wizard Yes, in the core run where it fits 👤 Human Silthus/posthog#544
Where the entry hooks into the core run A native command plus a spell book row. An install-run offer needs a signal for "sends no email yet" 🤖 Agent Lane #318
OAuth scopes for sender setup None now. The skill treats hidden tools as inconclusive and uses the UI path 🤖 Agent Lane #318
How the Wizard records a migration migrated_from on the existing setup wizard finished event, from the skill ID, set in SessionStore.setSkillId 🤖 Agent Silthus/posthog#461
Fix Pi's [ABORT] gap here or in its own PR Here: the command's abort outros need it, and a second PR would need its own person review. Rejected: a separate PR, or the deprecated Anthropic binding 🤖 Agent Lane #318
Keep [ABORT] handling to programs with abort cases Yes, so a program without cases keeps today's Pi behaviour 🤖 Agent Lane #318
Which lines end a Pi run Only a line that starts with [ABORT], in the Pi harness; the shared parser stays as the Anthropic harness uses it. Rejected: changing the shared parser 🤖 Agent Lane #318
Check the Pi test's analytics through a posthog-node fake No: the file mocks analytics for all its tests; the test checks the returned abort and the prompts instead. Rejected: a second file with the harness setup copied 🤖 Agent Lane #318

Follow-ups:

  • The "API not enabled" outro says to ask PostHog to enable the beta; it names the exact way once the docs page names one.
  • The linear sequence shows a matched abort's own docs link and never falls back to the program's; a fallback in linear.ts would let this program drop its per-case links.

Involvement: no grilling rounds, no prototypes. Fork reviews: GPT-6.1 Sol rounds and a two-family QA swarm (9 lenses). On 04b79150: ReviewHog and a fleet proof panel (GPT-6.1 Sol correctness, risk, design, systems, sweep, anti-overengineering; Opus 5.5 fresh eyes). It found the Pi [ABORT] gap and tests that read config; both fixed in 287887f4 and b09af2e4. Round 2 on the fixes (GPT-6.1 Sol correctness, risk, systems; Opus 5.5 fresh eyes): three minor findings and one test rewrite, fixed in 226aad62. Two GPT-6.1 Sol final reads: one high finding, fixed in 904543aa, then none.

LLM context

Written by Claude Code, Claude Opus 5.5 (claude-opus-5-5), as a build lane of a planning map. Reviews: GPT-6.1 Sol (high) and Claude Opus 5.5 (high) in fleet proof panels, plus ReviewHog.

🤖 Generated with Claude Code

Silthus and others added 7 commits October 31, 2026 09:54
… send

`npx @posthog/wizard transactional-email` runs the context-mill skill
omnibus-send-transactional-email: the setup agent checks for a verified
sender, writes the project secret API key to the env file, and adds the
first send with posthog-node or the HTTP API. The intro spell book of a
plain run lists it, and the skill's two [ABORT] reasons end the run with
a next step.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…eanup

Import `ProgramConfig` from `@programs/program-contract`, where the services
refactor moved it, and take the outro's `exit` step from the skill cleanup
in the flow trace.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A run of a migrate-email-<provider> skill sets migrated_from, so the
setup wizard finished event names the provider.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…migration

The tag stayed on the process after the skill changed, so a later
skill's events still said the run migrated from the email provider.
Every skill change now sets or clears it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Both abort cases carry the transactional emails docs link, so the error
outro of a matched abort shows where to go next. The test reads the cases
through the program's run config, so dropping them from the config fails
it. The program sets its binding explicitly, and the added comments go.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The program names Pi, the linear sequence and its model, so a change to the
default binding no longer moves it off the sequence its abort cases need.
The abort test drops a check the type already makes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Nov 2, 2026

Copy link
Copy Markdown

🧙 Wizard CI

Run the Wizard CI and test your changes against wizard-workbench example apps by replying with a GitHub comment using one of the following commands:

Test all apps:

  • /wizard-ci all

Test all apps in a directory:

  • /wizard-ci ai-observability
  • /wizard-ci basic-integration
  • /wizard-ci feature-flags
  • /wizard-ci mcp-analytics
  • /wizard-ci replay-vision
  • /wizard-ci revenue
  • /wizard-ci self-driving
  • /wizard-ci warehouse
  • /wizard-ci warehouse-seeded

Test an individual app:

  • /wizard-ci ai-observability/anthropic
  • /wizard-ci ai-observability/google-adk
  • /wizard-ci ai-observability/groq
Show more apps
  • /wizard-ci ai-observability/manual-capture
  • /wizard-ci ai-observability/openai
  • /wizard-ci ai-observability/openai-agents
  • /wizard-ci ai-observability/opentelemetry
  • /wizard-ci ai-observability/vercel-ai
  • /wizard-ci basic-integration/android
  • /wizard-ci basic-integration/angular
  • /wizard-ci basic-integration/astro
  • /wizard-ci basic-integration/django
  • /wizard-ci basic-integration/fastapi
  • /wizard-ci basic-integration/flask
  • /wizard-ci basic-integration/flutter
  • /wizard-ci basic-integration/javascript-node
  • /wizard-ci basic-integration/javascript-web
  • /wizard-ci basic-integration/laravel
  • /wizard-ci basic-integration/next-js
  • /wizard-ci basic-integration/nuxt
  • /wizard-ci basic-integration/python
  • /wizard-ci basic-integration/rails
  • /wizard-ci basic-integration/react-native
  • /wizard-ci basic-integration/react-router
  • /wizard-ci basic-integration/sveltekit
  • /wizard-ci basic-integration/swift
  • /wizard-ci basic-integration/tanstack-router
  • /wizard-ci basic-integration/tanstack-start
  • /wizard-ci basic-integration/vue
  • /wizard-ci feature-flags/django
  • /wizard-ci feature-flags/next-js
  • /wizard-ci mcp-analytics/custom-dispatcher
  • /wizard-ci mcp-analytics/typescript-sdk
  • /wizard-ci replay-vision/javascript-node
  • /wizard-ci replay-vision/next-js
  • /wizard-ci replay-vision/react-vite
  • /wizard-ci revenue/stripe
  • /wizard-ci self-driving/astro
  • /wizard-ci self-driving/fastapi
  • /wizard-ci self-driving/nuxt
  • /wizard-ci self-driving/react-router
  • /wizard-ci self-driving/sveltekit
  • /wizard-ci warehouse/monorepo-env
  • /wizard-ci warehouse/multi-source-next
  • /wizard-ci warehouse/stripe-node
  • /wizard-ci warehouse/zero-source
  • /wizard-ci warehouse-seeded/next-stripe
  • /wizard-ci warehouse-seeded/next-stripe-declined

Test against a Context Mill branch:

  • /wizard-ci all context-mill:my-branch

Add context-mill:<branch> to any command above to pin the Context Mill branch. It defaults to main.

Results will be posted here when complete.

The program ran omnibus-send-transactional-email, which no menu
publishes. Run sending-transactional-emails, the PostHog/posthog skill
that PostHog/context-mill now publishes, and ask for what it sets up: a
transactional template in PostHog and its template ID in the report.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Silthus and others added 2 commits November 4, 2026 07:19
The setup prompt asked the agent for a transactional template. The
sending-transactional-emails skill now keeps the email in the app's
code by default and uses a PostHog template only when the person picks
it, so the prompt and README say the same. The report names where the
email's content lives and its template key or ID.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…he skill

The prompt restated the skill's choice between code and a PostHog
template. It now lists only what to set up, so the skill alone owns
that choice and its default.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Silthus and others added 2 commits November 5, 2026 09:43
The skill sends with PostHog's SDK for the backend's language, eight in
all, not only Node.js and Python.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ail-program

# Conflicts:
#	src/cli/index.ts
#	src/programs/program-runner/__tests__/program-registry.test.ts
#	src/programs/program-runner/program-registry.ts
#	src/programs/tsconfig.json
#	src/tui/__tests__/__snapshots__/flow-traces.test.ts.snap

@posthog posthog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PostHog Review · Standard · reviewed 04b7915: 1 Must fix.

See this review and its findings in PostHog Review. Comment @posthog review --deep for a Deep review. A Deep review takes longer and looks deeper than a Standard review.

Silthus and others added 2 commits November 14, 2026 19:25
A program with abort cases tells its skill to emit '[ABORT] <reason>'.
The Pi harness only stopped on the gateway's [posthog:abort], so such a
run went on to the success outro. It now ends on the reason, as the
Anthropic path does, and the linear sequence shows the matching case.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Capture setup wizard finished at a fake posthog-node client instead of
reading a mocked setTag. Drop the tests that asserted a mocked runner's
arguments and re-ran the abort regexes over the config.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@posthog posthog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PostHog Review · Standard · reviewed b09af2e: no findings.

Silthus and others added 2 commits November 14, 2026 19:50
A reply that mentions the marker mid-line no longer ends the run, and
each text block is read on its own line, so a block after the [ABORT]
line can't run into its reason. Tests pin both, and that a program
without abort cases keeps its nudge.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@posthog posthog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PostHog Review · Standard · reviewed 904543a: no findings.

After login, every error outro but a security stop went to "The Wizard's
a little busy". An agent that stops on [ABORT] made a decision, so its
outro, such as "No server code found", now shows with its body and docs
link. Found on a live walk of transactional-email.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@posthog posthog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PostHog Review · Standard · reviewed 60445c7: no findings.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@posthog posthog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PostHog Review · Standard · reviewed 7c9de7e: no findings.

@Silthus
Silthus marked this pull request as ready for review November 16, 2026 08:31
@Silthus
Silthus requested a review from a team as a code owner November 16, 2026 08:31
@Silthus
Silthus requested a review from mayteio November 16, 2026 08:31
@veria-ai

veria-ai Bot commented Nov 16, 2026 •

Copy link
Copy Markdown

PR overview

All previously flagged issues have been addressed. No open security concerns remain on this pull request.

Fixed/addressed: 1 · PR risk: 0/10

Reviews

@posthog-security-review-bot posthog-security-review-bot Bot added the security-review Request a security review label Nov 16, 2026
@posthog-security-review-bot

posthog-security-review-bot Bot commented Nov 16, 2026 •

Copy link
Copy Markdown

✅ Security review complete — 7c9de7e4591e

No findings above the confidence threshold, in 7 min.

Add the security-review label to run again on the latest commit.

@posthog-security-review-bot posthog-security-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agent-driven security review - no findings.

@posthog-security-review-bot posthog-security-review-bot Bot removed the security-review Request a security review label Nov 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants