diff --git a/context/skills/migrate-email/config.yaml b/context/skills/migrate-email/config.yaml index 2accfcd5..8becd4c8 100644 --- a/context/skills/migrate-email/config.yaml +++ b/context/skills/migrate-email/config.yaml @@ -2,7 +2,7 @@ type: skill template: description.md category: migrate -description: Migrate transactional email from {display_name} to PostHog. Use when asked to migrate from {display_name}, replace {display_name} with PostHog, or switch email providers. It finds every {display_name} send in the code, sets up the sender and DNS next to {display_name}'s records, imports {display_name}'s suppression list, hands over the PostHog secret key, and writes a migration report. It changes no send call site yet. +description: Migrate transactional email from {display_name} to PostHog. Use when asked to migrate from {display_name}, replace {display_name} with PostHog, or switch email providers. It finds every {display_name} send in the code, sets up the sender and DNS next to {display_name}'s records, imports {display_name}'s suppression list, moves each send PostHog can take to posthog-node, and writes a migration report that lists the sends that stay. tags: [migration, workflows] cli: role: command diff --git a/context/skills/migrate-email/description.md b/context/skills/migrate-email/description.md index 7bb85a8d..b36e4c43 100644 --- a/context/skills/migrate-email/description.md +++ b/context/skills/migrate-email/description.md @@ -2,7 +2,7 @@ Use this skill to move an app's transactional email, such as welcome emails, password resets, and receipts, from {display_name} to PostHog. -This version prepares the move and changes no send call site. It finds every {display_name} send, sets up the PostHog sender and its DNS records next to {display_name}'s, imports {display_name}'s suppression list, hands over the PostHog secret key, and writes a report. Both providers can send during the cutover. +It finds every {display_name} send, sets up the PostHog sender and its DNS records next to {display_name}'s, imports {display_name}'s suppression list, and hands over the PostHog secret key. Then it moves every send PostHog can take to `posthog-node`, checks one email on one person, and writes a report. Sends PostHog can't take stay on {display_name} with a reason, so both providers send during the cutover. The transactional email API is in beta. PostHog enables it per project. @@ -17,8 +17,8 @@ Read the step files in order, one at a time. Each step names the next one. Do no The folder `references//` holds everything specific to {display_name}. The steps tell you when to read each file: - `sdk-reference.md`: how {display_name} looks in code, so you can find every send. -- `mapping.md`: how each send field maps to PostHog, and what keeps a call site on {display_name}. -- `account.md`: {display_name}'s DNS records and how to export its suppression list. +- `mapping.md`: how each send field maps to PostHog, what keeps a call site on {display_name}, and how to rewrite a site that moves. +- `account.md`: {display_name}'s DNS records, and how to get its templates and its suppression list. Everything you need is on disk. Do not fetch {display_name}'s docs. @@ -28,10 +28,10 @@ The plan file `.posthog-email-migration-plan.md` lives at the project root. Step ## Rules -- Change no send call site. This version only prepares the move. +- Change send code only in Step 7, and fix it only in Step 8: the sites that move, the helpers, callers, and tests they need, and other `posthog-node` code that the upgrade breaks. Leave every site that stays as it is. - Never touch DNS or the {display_name} account. Give the person the records and the links, and let them make the change. -- Never remove the {display_name} package, its client, or its API key. -- Never create, upload, or edit a template in PostHog. +- Never remove the {display_name} package or its API key. The report tells the person when to remove them. +- Never create, upload, or edit a template in PostHog. The person makes each template that moves to PostHog. - Never write a recipient's email address into a file in the repository, the plan, or the report. Count addresses instead. - Keep keys in environment variables. Never hardcode a key, and never ask the person to paste one into the chat. - Do not commit. The person reviews the diff and commits. @@ -54,4 +54,4 @@ Outside the Wizard, skip the `[STATUS]` lines, and tell the person why you stopp `transactional-emails.md` is the transactional email overview. It covers how to create a transactional email and its API key, idempotency, and the error codes. `api.md` covers the send route, including content from your code, and the checks of a send. -When the `sending-transactional-emails` skill is available, read it too. Its `moving-from-other-providers.md` moves any provider's sends to a PostHog template. This skill adds what is specific to {display_name}: it finds every send, sets up the sender next to {display_name}'s DNS records, imports {display_name}'s suppression list, and hands over the key. +When the `sending-transactional-emails` skill is available, read it for the PostHog side of a send. Its `moving-from-other-providers.md` moves sends to a template stored in PostHog. This skill moves {display_name}'s sends to `posthog-node`, with the content in the app's code, or to a template the person makes in PostHog. Where the two disagree, this skill's steps win. diff --git a/context/skills/migrate-email/references/3-inventory.md b/context/skills/migrate-email/references/3-inventory.md index 87086bac..85f2e7c2 100644 --- a/context/skills/migrate-email/references/3-inventory.md +++ b/context/skills/migrate-email/references/3-inventory.md @@ -5,7 +5,7 @@ next_step: 4-sender.md # Step 3, list and classify every send -List every place the code sends email through the provider, and decide for each one whether PostHog can send it. Change no source file. The only file you write is the plan file. +List every place the code sends email through the provider, and decide for each one whether PostHog can send it. Change no source file. The only files you write are the plan file and the template files below. ## Status @@ -33,6 +33,8 @@ For each site, read the surrounding code and record: Also list every other call to the provider that is not a send, such as reading an email's status, contacts, or domains. These calls stay on the provider. Note what each one does, because a site whose code reads the status back stays on the provider too. +When `sdk-reference.md` has a "Webhooks" section, also list each webhook handler: a route of the app that receives the provider's delivery events. Note the events it handles, what it does with them, and how it finds the send an event belongs to, such as by the provider's message id, by a value the send set, or by the recipient's address. `mapping.md` says under "Webhooks" which sites a handler keeps on the provider. + When you find no site, stop before you write the plan file: ``` @@ -47,10 +49,20 @@ Then go through `mapping.md` field by field. A site **moves** when PostHog can s When the code passes a value you can't read, such as a list of recipients built at run time, follow it to its callers. When you still can't tell, the site stays, and the reason says what you could not check. -A `template` site moves only to a template made in PostHog, and the person makes that template. Check the site's other fields first: a site that another field keeps on the provider stays with that reason, and needs no question about its template. Ask the person once, for every other template the sites send, whether to move it to PostHog or keep it on the provider for now. A template they keep stays, and the reason is `kept on for now`. For each template they move, ask them to copy it from the provider's dashboard into a file outside the repository, and to tell you where they saved it. Read those files to classify the site, and never write the content into the plan. When the person can't give the content, the site stays, and the reason is `template content not checked`. +A `template` site moves only to a template made in PostHog, and the person makes that template. Check the site's other fields first: a site that another field keeps on the provider stays with that reason, and needs no question about its template. Ask the person once, for every other template the sites send, whether to move it to PostHog or keep it on the provider for now. A template they keep stays, and the reason is `kept on for now`. + +A template they move needs its content to be classified. Read the provider's `account.md`, and ask the person once for every template they move: copies from the dashboard that they save in files outside the repository and tell you where, or a read through the API. Read the person's copies where they saved them. Write an API answer to a file in the system's temporary folder, never in the repository or the plan, and delete that file as soon as its site is classified. When the person can't give the content, the site stays, and the reason is `template content not checked`. A template whose content passes the checks moves only once the person has made it in PostHog. Ask them once, for every such template, to make it in PostHog from the provider's template and give you its template ID and the address it sends from, usually the provider template's sender. PostHog makes a template transactional only once a verified sender covers its address: ask them to make it transactional now when the project has one, and otherwise once Step 4 verifies the sender. When the email carries a login code, a reset link, or another secret, ask them to mark the template sensitive, as `mapping.md` says. The report lists both before cutover either way. Tell them how its placeholders change, as `mapping.md` says under "Content". Record each template ID with its site under the sites table, and write the address in the site's `from` column, so Step 4 sets up a sender for its domain. A template without an ID stays, and the reason is `PostHog template not made yet`. +### Opens the provider didn't track + +PostHog adds its open pixel to every email it sends, in the code form and from a template made in PostHog alike. Check this for each site that nothing else keeps on the provider, as the provider's `account.md` says: whether the provider tracked the opens of its emails, with an open pixel in each one. + +- When a site's emails had no open pixel, because the provider never tracks opens, or open tracking was off or never set up, ask the person once, for all such sites together, whether PostHog may add its open pixel to emails that never had one. Yes: those sites move, and the report says that PostHog now counts their opens. No, or no answer: they stay, and the reason is `open tracking off`. +- When nobody can tell whether the provider tracked a site's opens, the site stays, and the reason is `open tracking not checked`. +- For each site that moves with the person's yes, write `open pixel added: the person agreed` in its note under the sites table, so the report names it. + ## Write the plan file Write `.posthog-email-migration-plan.md` at the project root with three sections. Never write an email address of a recipient into it. A `from` address is fine. @@ -63,8 +75,12 @@ A phase checklist, in this order. Tick `preflight` with what Step 2 recorded, an - [ ] sender - [ ] suppressions - [ ] key +- [ ] rewrite +- [ ] verify ``` +When the run created a full-access provider key, as `account.md` says, add a line under the checklist: `full-access key — created: `. Step 5 adds whether the person deleted it. + A sites table, one row per site, in source tree order: ``` @@ -73,4 +89,4 @@ A sites table, one row per site, in source tree order: `moves` is `yes` or `no`. `reason` is empty when the site moves. -A list of the other provider calls, each with its file, line, and what it does. +A list of the other provider calls, each with its file, line, and what it does, and a list of the webhook handlers, each with its file, line, the events it handles, and how it finds the send. diff --git a/context/skills/migrate-email/references/5-suppressions.md b/context/skills/migrate-email/references/5-suppressions.md index 5dfeb60d..e5dff94e 100644 --- a/context/skills/migrate-email/references/5-suppressions.md +++ b/context/skills/migrate-email/references/5-suppressions.md @@ -21,7 +21,7 @@ Read the provider's `account.md` first, whether or not the import runs. Its "For Look for the PostHog MCP tool `email-suppressions-add`, and read its schema with `info`. -When it is missing, the import is not available to this person yet. Do not try other tools, and do not use the opt-out tools: opt-outs are marketing preferences, and transactional email ignores them. Tick the phase as `- [ ] suppressions — not imported: no import tool`, and continue to the next step. The report tells the person to add the addresses in PostHog (Workflows, then Messaging, then Suppression list) right before cutover, so the list holds the entries the provider adds until then. +When it is missing, the import is not available to this person yet. Do not try other tools, and do not use the opt-out tools: opt-outs are marketing preferences, and transactional email ignores them. Tick the phase as `- [ ] suppressions — not imported: no import tool`, ask about the full-access key as the last section says, and continue to the next step. The report tells the person to add the addresses in PostHog (Workflows, then Messaging, then Suppression list) right before cutover, so the list holds the entries the provider adds until then. ## Get the list @@ -37,8 +37,12 @@ The provider's `account.md` says how to export the list, which entries to take, - Repeating an import is safe: an address already on the list stays on it. - The provider keeps adding to its list while it sends, so the report asks the person for one last import right before cutover. - Never print the addresses, and never write them into the plan or the report. Count them. -- When a call fails, read its error and fix the call. When it fails again, stop the import and record how many entries you imported. +- When a call fails, read its error and fix the call. When it fails again, stop the import, record how many entries you imported, and ask the person once whether to repeat it now. Repeat it when they say yes. ## Write down Tick the phase in the plan file only when every entry is imported: `- [x] suppressions — / imported`. When the list is empty, write `- [x] suppressions — empty list`. When the import stopped partway, write `- [ ] suppressions — / imported: `. When the person has no export and no key, write `- [ ] suppressions — not imported: `. + +## The full-access key + +Every way out of this step ends here. When the plan says the run created a full-access provider key, ask the person now to delete it at the provider and remove its variable from the env file, as `account.md` says. After a partial import, ask once the person said no to a repeat, or the repeat ended. Note in the plan whether they did. diff --git a/context/skills/migrate-email/references/6-key.md b/context/skills/migrate-email/references/6-key.md index 2366c5ba..cbad5fa5 100644 --- a/context/skills/migrate-email/references/6-key.md +++ b/context/skills/migrate-email/references/6-key.md @@ -1,6 +1,6 @@ --- title: Hand over the PostHog secret key -next_step: 7-report.md +next_step: 7-rewrite.md --- # Step 6, hand over the PostHog secret key diff --git a/context/skills/migrate-email/references/7-report.md b/context/skills/migrate-email/references/7-report.md deleted file mode 100644 index 47a27361..00000000 --- a/context/skills/migrate-email/references/7-report.md +++ /dev/null @@ -1,83 +0,0 @@ ---- -title: Write the migration report -next_step: null ---- - -# Step 7, write the migration report - -The report comes from the plan file. Invent nothing. - -## Status - -``` -[STATUS] Writing the migration report -``` - -## Write the report - -Read `.posthog-email-migration-plan.md`. Write `posthog-migrate-email--report.md` at the project root, where `` is the name of the provider folder under `references/`. Use the structure below, then delete the plan file. Never write a recipient's email address into the report. - -```markdown -# Email migration report: to PostHog - - - -## Send call sites - -| File | Line | Content | Status | -|------|------|---------|--------| - -: ".> - -## Other calls - -.> - -## Sender and DNS - - - -## Suppression list - - - -## PostHog secret key - - - -## Sending tier - - - -## Before cutover - - - -## Summary -``` - -End the report with the summary block, as the last thing in the file. The Wizard parses it, so keep the keys, their order, and bare values: a whole number, `true`, `false`, or the provider folder name. Never add a comment or a reason to a value. The reasons are in the tables above. For example: - -```yaml -from: resend -sites_found: 3 -sites_rewritten: 0 -sites_left: 1 -dns_ready: false -suppressions_ready: true -``` - -- `from`: the provider folder name. -- `sites_found`: every row of the sites table. -- `sites_rewritten`: the sites this run changed to send with PostHog. This version changes none, so it is `0`. -- `sites_left`: the sites that stay on the provider. -- `dns_ready`: `true` when every sending domain has a verified sender. -- `suppressions_ready`: `true` when every entry of the suppression list was imported, or the list was empty. - -Sites that are ready to move count in `sites_found` only. - -After the report is written, print its path: - -``` -Created migration report: -``` diff --git a/context/skills/migrate-email/references/7-rewrite.md b/context/skills/migrate-email/references/7-rewrite.md new file mode 100644 index 00000000..924ead98 --- /dev/null +++ b/context/skills/migrate-email/references/7-rewrite.md @@ -0,0 +1,84 @@ +--- +title: Move every site that can move +next_step: 8-verify.md +--- + +# Step 7, move every site that can move + +Rewrite each site whose `moves` is `yes` in the plan to send with `posthog-node`. The provider's `mapping.md` has the rewrite of each field, call, and content form under "Rewrite". + +## Status + +``` +[STATUS] Setting up posthog-node for email +[STATUS] Moving the send call sites +``` + +## Before you change code + +- Change only the sites whose `moves` is `yes`. Never change a site that stays, and never change a line that a site that stays still needs. +- When no site moves, tick the phase in the plan with `none moved` and go to the next step. +- Run the project's typecheck, such as its `typecheck` script or `tsc --noEmit`, or its build when it has no typecheck, and write each error's file, code, and message, such as `TS2345` and its text, with how many times it shows, under the rewrite phase in the plan: Step 8 compares with them. Leave out the line: the rewrite moves it. + +## Set up the client + +Install `posthog-node` with the project's package manager when the app doesn't have it. `posthog.emails` alone is not enough: early releases send only a template made in PostHog, by `templateId`. Check the installed package's types for what the moving sites need, and upgrade it to its latest release only when they lack one of these: + +- a site with `html`, `text`, or a component: `posthog.emails.send` takes `templateKey` with `subject` and `html` or `text`; +- a site that sets `replyTo`: the send takes `replyTo`; +- a site with rendered content whose email carries a login code, a reset link, or another secret: the send takes `sensitive`; +- the client: the client options take `migratedFrom`. + +Before you install or upgrade `posthog-node`, check that the app runs on a Node.js version the release you install supports: compare the `engines` field the registry lists for it with the version the app runs in production, from its `engines`, `.nvmrc`, `.node-version`, Dockerfile base image, or hosting config. When you can't tell that version, ask the person. When the release doesn't support it, change no site: an older Node lacks what the client needs, such as `crypto.randomUUID()`, and every send fails. Mark each moving row `moves: no — the app runs Node , which posthog-node doesn't support`, tell the person, tick the phase with `none moved`, and go to the next step. + +When the latest release lacks one of these, change no site that needs it. Mark each such row `moves: no — posthog-node has no yet`, tell the person, and go on with the rest. When the client options lack `migratedFrom`, leave it out of the client. When no site is left, undo the install or upgrade of `posthog-node`, tick the phase with `none moved`, and go to the next step. + +Configure a server-side client of its own for email, also when the app already has a `posthog-node` client: + +```js +import { PostHog } from 'posthog-node' + +export const posthog = new PostHog(process.env.POSTHOG_PROJECT_TOKEN, { + host: process.env.POSTHOG_HOST, + secretKey: process.env.POSTHOG_PROJECT_SECRET_API_KEY, + enableLocalEvaluation: false, + migratedFrom: '', // only when the client options take it +}) +``` + +- When the client options take `migratedFrom`, set it to the value that the provider's `mapping.md` names under "The PostHog client". posthog-node sends it with every email request, so PostHog knows that the sends moved from that provider. +- Never change the app's own `posthog-node` client for email. posthog-node also uses `secretKey` for its flags and remote config, so the email key, or `enableLocalEvaluation: false`, would change how the app's flags evaluate, and a client the app disables answers every send with `client_disabled`. +- When the project token or the host is not configured yet, get the token from the PostHog MCP `project-get` tool, called without an ID (`api_token`). The host is `https://us.i.posthog.com` for US Cloud and `https://eu.i.posthog.com` for EU Cloud. Ask the person which one when you can't tell. +- When `host` is a reverse proxy or a self-hosted instance, also set `appHost` to the PostHog app URL, such as `https://us.posthog.com`. posthog-node never sends the secret key to a proxy, and without `appHost` the send fails with `missing_app_host`. +- Put the client in its own server-only module, next to the provider's client, and import it where the sites send. +- A site whose request had its own deadline keeps it: the provider's `mapping.md` says how, under "HTTP". Each distinct deadline gets its own client like the one above, with that timeout, in the same module. +- Use the env variable names the app already uses for the PostHog token and host. Add each new name, without a value, to the env example file. + +## Link emails to your users + +PostHog records each send, and its sent, opened and clicked events when engagement capture is on, against the person the send names with `distinctId`. Pass the ID the app already passes to `identify()`: + +- Find the app's identify call, such as `posthog.identify(user.id)` in posthog-js, or the `distinctId` of its `posthog-node` captures. +- At each site, pass the same ID for the recipient, such as `distinctId: user.id`. `distinctId` takes a string, and posthog-js turns a number into its string, so a numeric ID becomes `distinctId: String(user.id)`. Follow the recipient back through the callers when the site only has the address. +- Without `distinctId`, PostHog records the email against a person whose distinct ID is the recipient's address. Unless the app identifies people by their address, that is a separate person from the user. +- When the recipient isn't a user yet, such as an invite or an address check before the account exists, isn't a user at all, such as a team inbox, or the site has no way to know the user, leave `distinctId` out and note it in the plan row. +- When the app identifies no one in PostHog, use the app's own user ID, and note it for the report. + +## Rewrite each site + +For each site that moves, in plan order: + +- Rewrite the call as the provider's `mapping.md` says, for the site's content form: `html` and `text` content, a component the app renders, or a template made in PostHog. +- Give each send with rendered content a `templateKey` that names the kind of email, such as `welcome` or `password-reset`, with letters, digits, `-`, and `_` only. Two sites that send the same email share a key. A send whose key a template saved in PostHog already has goes out through that template's settings, and fails when that template isn't transactional. The PostHog MCP doesn't list template keys, so ask the person once, for all the keys together, whether a template in the project already has any of them, and give each such send another key. +- A send with rendered content whose email carries a login code, a reset link, or another secret gets `sensitive: true`. Without it, a send whose key matches a template saved in PostHog takes that template's setting, which is off by default, and PostHog then sends the email's links through its click tracking and puts its subject on engagement events. A send by `templateId` takes no `sensitive`: PostHog refuses it, and the template made in PostHog holds that setting, as Step 3 asks. +- A site that moves to a template made in PostHog sends the template ID the person gave in Step 3, never one you guess. +- Install `@react-email/render` with the project's package manager when a site renders a component and the app doesn't have it. +- Keep the site's idempotency key, error branches, logging, and return value working as before. Keep the code around the send unchanged where you can. +- `posthog.emails.send` never throws on an API error: it returns `{ data, error }`. When the old call threw on an error, or the code checked an HTTP status, check `error` so the site fails the same way. +- Never add a field that the old call didn't set, apart from `templateKey`, `templateId`, `distinctId`, `sensitive: true` on a send with rendered content that carries a secret, and a field the provider's `mapping.md` says to add, such as `maxRetries: 0` on a request that kept its own deadline. `templateData` takes the place of the provider's template variables. + +When a site turns out to need something `mapping.md` keeps on the provider, undo its change, set `moves` to `no` with the reason, and go on. + +## Write down + +Tick the phase in the plan: `- [x] rewrite — sites moved`. Add a `key` column to the sites table with each moved site's `templateKey`, or its `templateId` for a template made in PostHog. Under the table, note each site without a `distinctId`, each fallback that an empty string now replaces, each site that got a branch for a suppressed recipient, each idempotency key a retry across the deploy could send twice, each new dependency, each upgraded one with its old and new version, and each new env variable name. diff --git a/context/skills/migrate-email/references/8-verify.md b/context/skills/migrate-email/references/8-verify.md new file mode 100644 index 00000000..aa53a9fe --- /dev/null +++ b/context/skills/migrate-email/references/8-verify.md @@ -0,0 +1,52 @@ +--- +title: Check the moved sites +next_step: 9-report.md +--- + +# Step 8, check the moved sites + +Check that the app still builds, its tests still pass, and one moved email reaches one person in PostHog. When no site moved in Step 7, tick the phase with `nothing to check` and go to the next step. + +## Status + +``` +[STATUS] Checking the moved sites +[STATUS] Sending a test email +``` + +## Build and tests + +- Run the project's typecheck, such as its `typecheck` script or `tsc --noEmit`, or its build when it has no typecheck. Fix each error whose file, code, and message are not in the list Step 7 wrote into the plan, in any file: the rewrite or the posthog-node upgrade caused it. An error in a line the rewrite changed is new too, even when the list has it, unless the line's old version in `git diff` had it, such as in a value the rewrite carried over as it was: fix it. For each error in the list, take the number of times the list says, less the copies that changed lines had in their old version and no longer have. When the error still shows more times than that, and the changed lines don't explain the extra ones, look for them in the unchanged lines that use what the rewrite changed, such as the send's result, its error, or the client, and fix them there. Note each file you fix outside the moved sites for the report. Every other error in the list was there before: note it, and leave it. +- Run the tests that cover the files you changed. Never delete or skip a test, and never turn a PostHog send back into the provider's request shape to keep an old assertion. + - A test of a moved site runs the real `posthog-node` client and intercepts PostHog's send request at the HTTP boundary, as `api.md` describes the request. It asserts on the request PostHog gets and on what the app does, with the same expected values the old test had. A test that intercepted the provider's HTTP requests now intercepts PostHog's. A test that mocked the provider's client, such as `jest.mock('resend')`, drops that mock and becomes such a test too. A test that mocks the app's own email module stays as it is. + - The test sets the env variables Step 7's email client reads: any project token and secret key, and the PostHog host the app uses, such as `https://us.i.posthog.com`. Without them the send fails before it makes a request. It intercepts the app host the client sends to, such as `https://us.posthog.com` for that host. A test that answers with a 429 or a 503 answers each of the client's two retries too, which wait about one second, then two. + - Intercept with the tool the test suite already uses for HTTP. posthog-node sends with the global `fetch`, which `msw`, `nock` 14 or later, and undici's `MockAgent` intercept; `nock` before 14 doesn't see `fetch`. When the suite has none of them, the test replaces `globalThis.fetch` with a fake that records the request and answers it, and puts the real one back after, so no dependency is added. +- Search the changed files for the provider's calls: a moved site has none left, and each moved site has one `posthog.emails.send`. + +## One email on one person + +The chain to prove: PostHog sends the email, records its engagement on the person, and that person's product events show next to it. + +Skip this part and record `not checked` with the reason when the API is not confirmed on, no sender is verified, or `POSTHOG_PROJECT_SECRET_API_KEY` is not set. + +1. Check that the app sends to the MCP's project: compare the project token the app reads with `api_token` from the PostHog MCP `project-get` tool, called without an ID. Read only the token's line of the env file, by searching for the token variable's name: the file also holds secret keys, so never open all of it. When they differ, record `not checked: the app uses another project`, tell the person which project the app uses, and skip the rest of this part. +2. Ask the person whether to send one test email. Say that it sends a real email. When they say no, record `not checked: declined`, and skip the rest of this part. +3. Run `SELECT toUnixTimestamp(now()), ''` through the PostHog MCP `execute-sql` tool, so the answer can't come from a cached run, and keep the number as the start of the test; skip this when the MCP has no SQL tool. Ask them to trigger one moved email in their local app that sends it to an address they own, from a site whose plan row shows it passes `distinctId`, such as a password reset or a welcome email after signup. When no moved site passes `distinctId`, record `not checked: no moved site names a person`, and skip the rest of this part. Ask them to open the email, then to sign in as the user it went to and use the app for a moment, so that the app's product events show on the same distinct ID. Ask them to tell you when they are done. +4. When the send fails, find its code in `transactional-emails.md` or `api.md`, tell the person the fix, and record it. Never retry a send yourself. +5. Find the person's distinct ID: the value the moved site passed as `distinctId`. Run a SQL query through the PostHog MCP `execute-sql` tool for that distinct ID since the start of the test, with a new random string each time you run it: + + ```sql + SELECT event, timestamp, properties.$workflow_invocation_id AS send, properties.$email_template_key AS template_key, properties.$email_template_id AS template_id, '' AS run FROM events + WHERE distinct_id = '' AND toUnixTimestamp(timestamp) >= + ORDER BY timestamp + ``` + +6. The chain holds when the result has a `$workflows_email_sent` whose `template_key` or `template_id` is the moved site's, an engagement event such as `$workflows_email_delivered` or `$workflows_email_opened` with the same `send`, and at least one product event of the app. A send of another email to the same person proves nothing about this one. An event from before the start of the test belongs to another email. Delivery events arrive after the provider reports back: when only the send shows, wait a minute and run the query once more. + +When the MCP has no SQL tool, ask the person to open their person in PostHog and tell you which of the three they see since they triggered the test email. + +Never write the address or the distinct ID into a file. + +## Write down + +Tick the phase in the plan: `- [x] verify — typecheck: , tests: , chain: `. diff --git a/context/skills/migrate-email/references/9-report.md b/context/skills/migrate-email/references/9-report.md new file mode 100644 index 00000000..47bb4a7a --- /dev/null +++ b/context/skills/migrate-email/references/9-report.md @@ -0,0 +1,93 @@ +--- +title: Write the migration report +next_step: null +--- + +# Step 9, write the migration report + +The report comes from the plan file. Invent nothing. + +## Status + +``` +[STATUS] Writing the migration report +``` + +## Write the report + +Read `.posthog-email-migration-plan.md`. Write `posthog-migrate-email--report.md` at the project root, where `` is the name of the provider folder under `references/`. Use the structure below. Then delete the plan file. Never write a recipient's email address into the report. + +```markdown +# Email migration report: to PostHog + + + +## Send call sites + +| File | Line | Content | Key | Status | +|------|------|---------|-----|--------| + +", or "Stays on : ".> + + + +## Other calls + +.> + +## Sender and DNS + + + +## Suppression list + + + +## PostHog secret key + + + +## Checks + + + +## Sending tier + + + +## Before cutover + +The moved sites send through PostHog as soon as this code is deployed. Everything in this list has to be done before that deploy. + + + +## After cutover + + + +## Summary +``` + +End the report with the summary block, as the last thing in the file. The Wizard parses it, so keep the keys, their order, and bare values: a whole number, `true`, `false`, or the provider folder name. Never add a comment or a reason to a value. The reasons are in the tables above. For example: + +```yaml +from: resend +sites_found: 3 +sites_rewritten: 2 +sites_left: 1 +dns_ready: false +suppressions_ready: true +``` + +- `from`: the provider folder name. +- `sites_found`: every row of the sites table. +- `sites_rewritten`: the sites this run moved to send with PostHog. +- `sites_left`: the sites that stay on the provider. +- `dns_ready`: `true` when every sending domain has a verified sender. +- `suppressions_ready`: `true` when every entry of the suppression list was imported, or the list was empty. + +After the report is written, print its path: + +``` +Created migration report: +``` diff --git a/context/skills/migrate-email/references/resend/account.md b/context/skills/migrate-email/references/resend/account.md index dfd42d4b..92eca841 100644 --- a/context/skills/migrate-email/references/resend/account.md +++ b/context/skills/migrate-email/references/resend/account.md @@ -18,18 +18,36 @@ A domain verified at Resend carries these records. Keep all of them until the cu - PostHog's DKIM records have their own names, so they sit next to `resend._domainkey`. - Keep the one `_dmarc` record. Never add a second one. +## Full-access key + +The app's own Resend key is usually a sending-only key, and it can't read templates or the suppression list. When the person chooses the API over the dashboard, ask them to create a full-access key in Resend, store it in `RESEND_FULL_ACCESS_API_KEY` in the git-ignored env file, and write it into the plan as Step 3's plan layout says. Its last use is the suppression import in Step 5. At the end of Step 5, on every way out of it, ask the person to delete the key in Resend and remove its variable from the env file, and note in the plan whether they did. The report reminds them when they didn't. Never print the key, and never write it into another file. + +## Template content + +A site whose template the person moves to PostHog needs the template's content, to check it before they make the PostHog template. + +**Copy from the dashboard.** In Resend: Templates, then the template. Ask the person for its alias or name, its HTML, its plain text when it has one, its subject, its `from` and reply-to when they are set, and each variable's fallback. + +**Read from the API.** `GET https://api.resend.com/templates/{id or alias}` with `Authorization: Bearer $RESEND_FULL_ACCESS_API_KEY`. The answer holds `alias`, `name`, `subject`, `from`, `reply_to`, `html`, `text`, and `variables`, a list of `{ key, type, fallback_value }`. + +Read the person's copies where they saved them. An API answer goes only into the temporary file Step 3 writes, which Step 3 deletes once the site is classified. Never write the content into the plan or the report. + +## Open tracking + +Resend tracks opens per sending domain, only when open tracking is on for it: Domains, then the domain, then Configuration. Ask the person once, for each sending domain of the sites that move, whether open tracking is on. Off means the site's emails had no open pixel, and no answer means nobody knows whether they had one, as Step 3 says under "Opens the provider didn't track". + ## Suppression list Resend suppresses an address after a hard bounce or a spam complaint, and keeps addresses the team added by hand. **Export from the dashboard (preferred).** In Resend: Emails, then Suppressions, then **Download CSV**. Read the header row to find the address column and the origin column. -**Read from the API.** `GET https://api.resend.com/suppressions?limit=100` with `Authorization: Bearer $RESEND_SUPPRESSIONS_API_KEY`. +**Read from the API.** `GET https://api.resend.com/suppressions?limit=100` with `Authorization: Bearer $RESEND_FULL_ACCESS_API_KEY`. - The answer is `{ "object": "list", "has_more": , "data": [{ "id", "email", "origin", "source_id", "created_at" }] }`. - Page with `after=` until `has_more` is `false`. - Resend allows 10 requests per second per team. Wait and retry on `429`. -- A sending-only key can't read the list. Ask the person to create a full-access key, store it in `RESEND_SUPPRESSIONS_API_KEY` for the import, and delete the key in Resend afterwards. +- A sending-only key can't read the list. Use the full-access key above. **Map each entry.** diff --git a/context/skills/migrate-email/references/resend/mapping.md b/context/skills/migrate-email/references/resend/mapping.md index 7dbd3587..49404222 100644 --- a/context/skills/migrate-email/references/resend/mapping.md +++ b/context/skills/migrate-email/references/resend/mapping.md @@ -13,14 +13,14 @@ This file decides, field by field, whether PostHog can send a Resend call site. | `text` | `text` | Never | | `react` | `html`, rendered in the app | Never. The app renders the component | | `template` | a template made in PostHog, sent by its `templateId` with `templateData` | The person keeps it on Resend for now, its content can't be read, it uses syntax beyond variables and fallbacks, the app fills a variable with HTML, or the site sets `from`, `replyTo` / `reply_to`, or `subject` next to it | -| `cc`, `bcc` | none | Always | -| `replyTo` / `reply_to` | `replyTo` / `reply_to`, as a list | An entry carries a name, such as `Jane `, there are more than 10, or the code builds the value at run time and you cannot tell that it holds bare addresses only. A single address moves as a list of one | -| `headers` | none | Always | -| `tags` | none | Always | -| `attachments` | none | Always | -| `idempotencyKey` / `Idempotency-Key` | `idempotencyKey` / `Idempotency-Key` | The key can hold anything but visible ASCII characters, such as a space, or be longer than 255 characters. PostHog takes 1 to 255 visible ASCII characters. Otherwise the key keeps its value, and PostHog keeps it for 24 hours as Resend does. The two providers don't share keys: a retry that runs after the deploy, of a send Resend already took, sends a second email. Note it in the report | -| `scheduledAt` / `scheduled_at` | none | Always | -| `topicId` / `topic_id` | none | Always | +| `cc`, `bcc` | none | Always. PostHog's send doesn't take it yet | +| `replyTo` / `reply_to` | `replyTo`, one address or a list | An entry carries a name, such as `Jane `, there are more than 10, or the code builds the value at run time and you cannot tell that it holds bare addresses only | +| `headers` | none | Always. PostHog's send doesn't take it yet | +| `tags` | none | Always. PostHog's send doesn't take it yet | +| `attachments` | none | Always. PostHog's send doesn't take it yet | +| `idempotencyKey` / `Idempotency-Key` | `idempotencyKey` / `Idempotency-Key` | The key can hold anything but visible ASCII characters, such as a space, or be longer than 255 characters. PostHog takes 1 to 255 visible ASCII characters. A key built from a value whose length or characters the code doesn't bound, such as a URL or text from a caller, counts as one that can. The report names that value, so the person can bound the key, such as by hashing that part, and move the site. Otherwise the key keeps its value, and PostHog keeps it for 24 hours as Resend does. The two providers don't share keys: a retry that runs after the deploy, of a send Resend already took, sends a second email. Note it in the report | +| `scheduledAt` / `scheduled_at` | none | Always. PostHog's send doesn't take it yet | +| `topicId` / `topic_id` | none | Always. PostHog's send doesn't take it yet | ## Calls @@ -39,15 +39,50 @@ This file decides, field by field, whether PostHog can send a Resend call site. | `if (error.name === '...')` | The name is not one of posthog-node's, which every site that moves sends through, an HTTP site too: PostHog's HTTP API answers with `type` and `code`, and only posthog-node gives these names: `validation_error`, `missing_api_key`, `invalid_permission`, `not_found`, `concurrent_idempotent_requests`, `invalid_idempotent_request`, `rate_limit_exceeded`, `monthly_quota_exceeded`, `service_unavailable`, `application_error` | | `emails.get(id)` and a branch on `last_event` | Always: the send and its status read stay together. PostHog's status is not `last_event` | | `data.id` stored for a webhook or a status read | Always, when Resend webhooks or status reads use the id | +| A `signal` the code can abort, such as one a caller passes in or an `AbortController` it aborts, on a `fetch` or axios request or in the options of `resend.emails.send(payload, { signal })`, which the SDK passes to its request | Always: `posthog.emails.send` takes no signal, so an email the old code cancelled would go out. A fixed `AbortSignal.timeout(...)` is a deadline, and moves as "HTTP" below says, in an SDK send too | ## Content - `html`, `text`, and `react` content is rendered for one recipient before the send. PostHog sends it as it is. - A `template` site moves only to a template made in PostHog. This skill never makes one: the person makes it from the Resend template and gives you its template ID, and makes it transactional once a verified sender covers its address, as Step 3 says. Until they give the ID, the site stays on Resend. - An email that carries a login code, a reset link, or another secret needs its PostHog template marked sensitive: PostHog then keeps no copy of each sent email's body or filled-in subject, and a clicked link keeps only the site's address. Templates are not sensitive by default. -- The check needs the template's content from the Resend dashboard, because the code has only its id. -- In the PostHog template, `{{{NAME}}}` becomes `{{ variables.NAME }}`, and the name keeps its case. `{{{NAME|there}}}` becomes `{{ variables.NAME | default: 'there' }}`, and a variable with a `fallback_value` in the template's definition gets the same `default`. Quote a fallback that holds a `'` with `"` instead. Liquid has no escape character in a quoted value. +- The check needs the template's content, because the code has only its id. `account.md` says how to get it. +- In the PostHog template, `{{{NAME}}}` becomes `{{ variables.NAME }}`, and the name keeps its case. `{{{NAME|there}}}` becomes `{{ variables.NAME | default: 'there' }}`, and a variable with a `fallback_value` in the template's definition gets the same `default`. Quote every fallback with `'`: a `"` inside the template's JSON body raises `TokenizationError`. Liquid has no escape character in a quoted value, so a fallback that holds a `'` keeps the site on Resend. - Liquid's `default` also replaces an empty string, which Resend inserts as it is. When the code can pass an empty string for a variable with a fallback, the email shows the fallback instead. Note it in the report. - Resend inserts a variable's value as it is. PostHog's Liquid escapes HTML in a value, so `` shows as text. A variable is HTML when the code fills it with markup or with a renderer's output, and such a variable keeps the site on Resend. A variable the app fills with plain values, such as names, numbers, and URLs, renders the same. -- A template that holds anything other than `{{{VAR}}}` and `{{{VAR|fallback}}}` placeholders, such as a section, a loop, or a condition, stays on Resend. So does a fallback that holds both `'` and `"`, and a fallback that holds HTML, such as a link: Liquid escapes the fallback too, so the markup shows as text. +- A template that holds anything other than `{{{VAR}}}` and `{{{VAR|fallback}}}` placeholders, such as a section, a loop, or a condition, stays on Resend. So does a fallback that holds a `'`, and a fallback that holds HTML, such as a link: Liquid escapes the fallback too, so the markup shows as text. - The PostHog template carries its own sender, reply-to address, and subject. A site that sets `from`, `replyTo` / `reply_to`, or `subject` next to `template` stays on Resend. + +## Webhooks + +Resend stops sending email events for a site that moves, and PostHog's events carry no Resend `email_id` and no `tags`. A site whose emails a webhook handler acts on stays, however the handler finds the send an event belongs to: by `email_id`, by a tag, or by anything else, such as the recipient's address. The reason is `its events feed a webhook handler`. When you can't tell which sites' emails a handler acts on, such as one that finds the send by the recipient's address, every site whose emails can reach it stays. The handler itself keeps working for the sites that stay. Never change it. + +## Rewrite + +How a site that moves is written with `posthog-node`. Step 7 says when to use each part. + +**The PostHog client.** Set `migratedFrom: 'resend'` on the `posthog-node` client that Step 7 configures, when its client options take it. + +**The call.** `resend.emails.send(payload, { idempotencyKey })` and `resend.emails.create(...)` become one `posthog.emails.send({ ... })`. The options object goes away: `idempotencyKey` moves into the payload. A `signal` in it is code around the send, as the table above says. + +**The fields.** + +- `to: ['jane@example.com']` becomes `to: 'jane@example.com'`. A value that is a list with one entry becomes that entry. +- When a helper takes a list of recipients and every caller passes one address, change the parameter to one address and update the callers, so the type keeps a second address out. Never send one email per address in a loop, and never drop the addresses after the first. +- `from`, `subject`, `html`, `text`, and `replyTo` keep their names and values. +- A send with rendered content gets a `templateKey`, as Step 7 says. + +**React.** `react: ` becomes `html: await render()`, with `import { render } from '@react-email/render'`. Keep a `text` the site already passes. + +**HTTP.** A `fetch` or `axios` call to `https://api.resend.com/emails` becomes `posthog.emails.send` with the camelCase fields: `reply_to` becomes `replyTo`, and the `Idempotency-Key` header becomes `idempotencyKey`. Code that read the response status reads `error` instead. A request with its own deadline, such as `signal: AbortSignal.timeout(5000)` or axios `timeout: 5000`, keeps it: set `emailRequestTimeoutMs` to that value on the client the sends use, and pass `maxRetries: 0` on the send, since the request never retried. posthog-node otherwise waits up to 15 seconds and retries twice, which can run past the site's deadline. The timeout belongs to the client, so each distinct deadline gets its own client, set up as Step 7 says with that `emailRequestTimeoutMs`, and its sites send through it. Sites without a deadline keep the shared client, and with it the SDK's limit: up to 15 seconds per request and two retries with the same idempotency key, so a retry never sends twice, and a send slower than 15 seconds on every try now ends in `network_error` where it used to wait. Never give a site a deadline it didn't have. + +**Templates made in PostHog.** The site sends `templateId` with the template ID the person gave, and `templateData` with the site's `variables` object. It sends no `templateKey`, `subject`, `html`, `text`, `from`, or `replyTo`: the template holds them. + +**The answer.** posthog-node answers `{ data, error }` like Resend and never throws on an API error. + +- `data.id` is PostHog's message id. Code that logs it or stores it for the app's own use keeps working. +- `error.name` uses Resend's names for the API's answers, so `if (error.name === 'rate_limit_exceeded')` keeps working. `error.message` and `error.statusCode` keep their names. +- A network failure is the exception. Resend answers it as an `error` with the name `application_error` and a `statusCode` of `null`. posthog-node answers it, once its retries are spent, with the code `network_error`, the name `service_unavailable`, and no `statusCode`. So a branch on `error.name === 'application_error'` also takes `error.code === 'network_error'`, and a branch on `error.name === 'service_unavailable'`, which took only Resend's `503` answers, also checks `error.code !== 'network_error'`, wherever it stands: a network failure then takes the path the old code gave it. A check of `error.statusCode === null` becomes `error.statusCode == null`, which a local argument error matches too, as Resend's `null` did. +- Resend takes a send to a suppressed address and drops it later. PostHog refuses it with `error.code` `recipient_suppressed` (name `validation_error`). When the site throws, retries, or reports a failure on any error, add `if (error?.code === 'recipient_suppressed')` before that branch and handle it the way the site handled a send that went out: log it and go on. A suppressed send has no `data.id`: when the code after the send uses the id for more than a log line, such as returning it to its callers or storing it, the site stays, and the reason is `uses the message id of a suppressed send`, unless its callers already handle a send without an id. Note each site that got this branch in the report. + +**The Resend client.** Keep `new Resend(...)` and its import while any site or other call still uses it. When nothing uses it, remove the import and the client, and keep the `resend` package and its key: the person removes them after the cutover. diff --git a/context/skills/migrate-email/references/resend/sdk-reference.md b/context/skills/migrate-email/references/resend/sdk-reference.md index fa3cf5f0..cb0b72b6 100644 --- a/context/skills/migrate-email/references/resend/sdk-reference.md +++ b/context/skills/migrate-email/references/resend/sdk-reference.md @@ -79,7 +79,18 @@ const { data, error } = await resend.emails.send({ ## Templates stored at Resend -A send with `template: { id: 'tmpl_...', variables: { FIRST_NAME: 'Jane' } }` renders a template from the Resend dashboard. Its content is not in the code. The template reads variables as `{{{FIRST_NAME}}}`, with an optional fallback: `{{{FIRST_NAME|there}}}`. +A send with `template: { id: '...', variables: { PRODUCT: 'Laptop' } }` renders a template from the Resend dashboard. `id` is the template's id or its alias. Its content is not in the code. + +- The template reads variables as `{{{PRODUCT}}}`, with an optional fallback: `{{{PRODUCT|a laptop}}}`. A variable's definition in the template can also carry a `fallback_value`. +- A variable's type is `string` or `number`. +- The template can hold a default `from`, `subject`, and `reply_to`. A value the send passes wins over the default. +- A send with `template` passes no `html`, `text`, or `react`. + +## Webhooks + +Resend posts each event to an endpoint of the app, signed through Svix. An event has `type`, `created_at`, and `data`, which holds the send's `email_id`, `from`, `to`, `subject`, and `tags`. An email event's `type` is such as `email.sent`, `email.delivered`, `email.delivery_delayed`, `email.bounced`, `email.complained`, `email.opened`, or `email.clicked`. Resend also sends events on contacts and domains. + +Look for routes that read `email_id` or an `email.` event type, `resend.webhooks.verify`, `Webhook` from `svix`, and the `svix-id`, `svix-timestamp`, and `svix-signature` headers. ## Status reads diff --git a/scripts/lib/tests/migrate-email-skill.test.js b/scripts/lib/tests/migrate-email-skill.test.js index e55985fd..9685afc9 100644 --- a/scripts/lib/tests/migrate-email-skill.test.js +++ b/scripts/lib/tests/migrate-email-skill.test.js @@ -92,6 +92,30 @@ describe('migrate-email skill', () => { expect(markdownFiles(skillDir).map((file) => readFileSync(file, 'utf8')).join('\n')).not.toMatch(/\{\{>|\{display_name\}|\{references\}/); }); + it('chains the steps from detection through rewrite and verify to the report', async () => { + outputDir = mkdtempSync(join(tmpdir(), 'migrate-skill-')); + + const skillDir = await buildSkill('migrate-email-resend', outputDir); + const visited = []; + for (let step = '1-detect.md'; step && visited.length < 20; ) { + visited.push(step); + const text = readFileSync(join(skillDir, 'references', step), 'utf8'); + step = text.match(/\*\*Upon completion, continue with:\*\* \[[^\]]+\]\(([^)]+)\)/)?.[1]; + } + + expect(visited).toEqual([ + '1-detect.md', + '2-preflight.md', + '3-inventory.md', + '4-sender.md', + '5-suppressions.md', + '6-key.md', + '7-rewrite.md', + '8-verify.md', + '9-report.md', + ]); + }); + describe('the key check in step 6', () => { const keyStep = readFileSync(join(CONFIG_DIR, 'skills', 'migrate-email', 'references', '6-key.md'), 'utf8'); const command = keyStep.match(/```bash\n(grep .+)\n```/)?.[1];