Skip to content

fix(messaging): sync the transactional workflow on asset url rewrites - #127295

Closed
Silthus wants to merge 16 commits into
masterfrom
fix/rewrite-email-asset-url-sync
Closed

Silthus wants to merge 16 commits into
masterfrom
fix/rewrite-email-asset-url-sync

Conversation

@Silthus

@Silthus Silthus commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

Moved from Silthus#571. It first opened as #118965.

Problem

  • When an operator moves an email asset with rewrite_email_asset_url, a transactional template's sends can keep the old asset URL.
  • Sends use the workflow's copy of the template content. The command rewrote the template row and never rebuilt that copy, so the template and its workflow disagreed.
  • This bites when a run is narrowed to --targets message_templates or --id <template id>. A default run rewrites hog_flows in place, so sends get fixed there, but the workflow gets no new version.
  • Found in the ship review of Silthus/posthog#547–Silthus/posthog#549 (Silthus/posthog#498). The gap predates that PR.

Impact

Keeps transactional sends on the asset URL their template shows, after an operator moves an asset. Not measured; the command runs by hand. Expected move: none on a metric. Run cost: n/a.

Merge Danger

Caution

🔴 Merge risk: high · two-way door (merge-risk rubric)

  • The API template update and the design patch now save through the new shared path. That path holds the template row lock and runs the workflow-editor check, the access check for transactional workflows. A mistake there affects every template save.
  • Limited by: the existing transactional email, template API and design operation tests all pass on the new path, and a revert restores the old inline blocks. The rewrite command writes workflow versions only when an operator runs it.

Changes

  • An asset URL rewrite of a transactional template now writes a new version of its workflow with the rewritten content, in the same transaction as the template save.
  • All three template content writers share one path, services/template_saves.py: the API update, the design patch, and the rewrite command. It locks the template row, checks the workflow-editor permission, checks the template can still send, saves, and syncs the workflow.
  • The rewrite command has no user, so it passes no workflow-editor check. A template that can no longer send (a marketing category, no subject or body, no sender at all) now fails its row and keeps the old URL, instead of drifting from its workflow. A workflow keeps its existing sender, as an API save does. The command already reports a failed row and exits non-zero.
  • The design patch now runs the same "can it still send" check as the update. Before, it skipped that check.
  • The rewrite now also covers a template's unpublished draft (draft_content, added in feat(workflows): draft and publish edits to transactional email templates #118664). Before, a draft saved before a rewrite kept the old URL, and publishing it brought that URL back.
  • A draft-only rewrite saves the draft alone and writes no workflow version, because sends never read the draft.
  • The command rewrites message templates first, then workflows, template workflows and functions. A transactional template's workflow gets its new version through the template sync. Each workflow row locks its live template first, the same order a template save uses. That lookup filters on the workflow's team, so it stays on the team_id index on an --all-teams run. When that template's published content still holds the old URL, the pass rewrites only the draft. Every other workflow gets its live actions and draft rewritten in place. --list-rows and the summaries follow that order.
  • --dry-run skips the check that a transactional template can still send: that check runs while the workflow version is built, which a dry run never writes.
  • The API update writes master's template tags through the shared path.
  • Mechanical: the view's inline lock-save-sync blocks, lock_transactional_workflow_id, check_transactional_update, update_template and edit_content_locked are gone. Nothing else called them.

Rebase note

services/template_saves.py uses the file, names and signatures Silthus/posthog#548 planned for the same orchestration. Silthus/posthog#548 is closed, so a later PR that adds a write intent or a content hash extends these functions. #118664's draft save keeps its own path, and a content save through the shared path still clears the draft.

How did you test this code?

Lane gate on a12f65d0cd, 2026-11-03: the rewrite command, template API and transactional email tests on a private database (133 passed), ruff, mypy on the changed files, tach and product:lint for messaging and workflows. Not rerun on this head: repo-wide mypy and the frontend typecheck, which this backend-only change does not reach.

CI report: comment density sits at the 3% warn line (16 of 486). Ten of the added comment lines are master's comments on the moved save code. The others say why a deleted workflow link stops restricting a template, why messaging rewrites its own rows, and why the workflow pass decides ownership per row under the template lock, within the run's --id.

Test rationale: one case in TestTransactionalEmail (products/messaging/backend/tests/api/test_transactional_email.py) catches a rewrite that saves the template without syncing its workflow. The closest existing test, test_saving_a_transactional_template_writes_a_new_workflow_version, covers the API update only. The rewrite command tests in products/workflows/backend/test/test_rewrite_email_asset_url.py never make a template transactional.

  • Test seams: rewrite_template_content, the messaging facade function the command calls for message templates, and the command itself for drafts.
  • A second case in TestRewriteEmailAssetUrl (products/workflows/backend/test/test_rewrite_email_asset_url.py) catches a rewrite that skips a template whose draft alone holds the URL. Red before the fix: the command matched 0 rows.
  • test_a_run_writes_one_new_version_of_a_transactional_templates_workflow in test_rewrite_email_asset_url.py runs the real command twice, with the default targets and with --targets hog_flows,message_templates, and expects version 2 with the new URL. Red before cc06f9d913 (assert 1 == 2).
  • test_a_failed_template_leaves_its_workflow_for_a_rerun_to_version fails a template on an unverified sender, then reruns: version 1 with the old URL after the failure, version 2 with the new URL after the rerun. Red before 9ad607cdf3: version 1 with the new URL.
  • test_rewrites_a_transactional_workflow_in_place_when_its_template_sync_does_not covers a default run, a --id <workflow id> run, and a template whose draft alone holds the old URL, each with a workflow whose live actions still hold it. Red before 6b4248a7e8 and ea7ba66dfa: the workflow kept the old URL. Its fourth case runs --id <workflow id> while the template also holds the old URL. Red before 99b51d88a6: only the draft changed.
  • test_keeps_a_workflow_on_its_template_when_a_save_puts_the_old_url_back_mid_run saves the old URL back into the template after the template pass. Red before e220b6dd53: the workflow pass rewrote the save's new version in place.
  • design_edit_after_the_category_became_marketing in test_rejects_an_update_that_would_stop_a_transactional_template_sending: a design edit of a template that can no longer send returns 400 and changes nothing. With the check stubbed out it returns 200.
  • test_rewrites_the_workflow_of_a_template_deleted_during_the_run deletes the template in place of its rewrite. Red before a62991ca21: the workflow kept the old URL.
  • Red then green: on the test commit with base code, the workflow stayed at version 1 (assert 1 == 2).
  • Lane gate on 87d1b6befe: the rewrite command, template API and transactional email tests (128 passed), ruff, and the commit hook's ruff, format and ty checks.
  • Live run: the real management command against a local database, on rows written by base code and by HEAD (below).

Live run

At a12f65d0cd, 2026-11-03: the real command through call_command on a migrated local database, with three templates made transactional first. The URLs are https://assets.example.com/live/{old,new}.png.

Run Template Workflow
before old URL version 1, old URL
--dry-run unchanged unchanged; reports 2 rows and 2 occurrences, the same counts as the real run
default run A and B new URL A and B version 2, new URL, written by the template sync; the workflow pass matched 0 rows
same run again unchanged, 0 rows matched still version 2
--id <workflow C>, template C still old old URL, outside the run's scope version 1, live actions rewritten in place

An --all-teams run over templates in two projects gives the same results: the dry run counts only the templates, each workflow gets version 2 through its template sync, and --id <workflow> rewrites in place.

Earlier heads, on a local database, each template made transactional first. URLs are https://assets.example.com/{old,new}.png.

Code Run Template Workflow
base --id A new URL version 1, old URL
HEAD --id B, row written by base code new URL version 2, new URL
HEAD --targets message_templates, row written by HEAD new URL version 2, new URL
HEAD same run again unchanged still version 2 (no-op)

A second run covers a transactional template whose draft alone holds https://assets.example.com/draft-old.png.

Code Run Draft Workflow
before the draft fix --id A 0 rows matched, old URL version 1
HEAD --id B new URL version 1
HEAD same run again 0 rows matched (no-op) version 1

Not run: the plugin server, so the worker reload after commit was not observed. It is scheduled with on_commit, as the API save already does.

Proofs (per qa-swarm PROOF.md)

Proof Status
Live run Done at a12f65d0cd (first table above). The two earlier tables come from earlier heads.
Prod shape Done, 2026-11-03: a read-only aggregate over stored transactional templates counted the ones the rewrite now fails (no subject, no body, a marketing category) and the ones with a draft. The counts stay in the private proof record.
Cross-version Not owed. No stored or queued format changes.
Shadow diff Gap, accepted by the author. Owed at high risk; a base-versus-head comparison needs production template and workflow rows, which this box lacks. The command tests run both target orders on the same rows, a failure followed by a rerun, and a drifted workflow.

Follow-ups

  • A rerun is the recovery when an editor writes the old URL back into a template during a run; a post-run count of templates that still hold --from-url would make that visible.

Release status

  • No feature flag controls this change

Docs update

None. The command's behavior is documented in its own help text, which is unchanged.

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

Agent: Claude Code, Claude Opus 5.5 (claude-opus-5-5)

Decisions

Question Answer Who Where
Route the rewrite through the update's save path, or add a sync call to the rewrite? Extract the shared save path, so there is only one sync path 👤 Human Side-fix brief from the conductor thread
Which shape for the extracted function? Silthus/posthog#548's template_saves.py names and signatures, so the rebase is a straight conflict resolution 🤖 Agent This PR
Rewrite template drafts here or in a separate PR? Here 👤 Human This thread, after the rebase onto #118664
What does the rewrite do when a transactional template can't send any more? Fail that row and keep the old URL, rather than save a template its workflow can't match 🤖 Agent This PR
In which order does the command rewrite its targets? Message templates first, whatever --targets says, so the template sync writes the workflow version 🤖 Agent pending approval
What does the workflow pass do with a workflow its template rewrite owns? Decides per row under the template's lock: rewrites only the draft when the template's published content holds the old URL, so a failed template keeps the workflow for a versioned rerun; every other workflow is rewritten in place 🤖 Agent pending approval
Does a dry run check that a transactional template can still send? No, and the help text says so; the check needs the workflow version a dry run never writes 🤖 Agent pending approval

Involvement: no grilling rounds, no prototypes. Review: one GPT-6.1 Sol review (high effort) on c90e0ca, no findings; one note on sender reuse corrected this description. Final GPT-6.1 Sol reads (high effort) of the rebase onto #118664 and of the draft rewrite found nothing. After the restack: GPT-6.1 Sol round 1 (one finding on the target order, fixed test-first) and a clean read of the fix. QA swarm on 2026-11-01: full panels in rounds 2 and 3 and a fixes panel in round 4 (GPT-6.1 Sol review lanes, Claude Opus 5.5 fresh eyes), every finding fixed test-first or answered, then final reads. After the move: a judge found the mid-run save race and the untested design-edit rejection, both fixed; a final read found the --id scope gap, fixed; PostHog Review found an unindexed per-row lookup, fixed; the last final read found nothing. Proof: a live run at a12f65d0cd, a production template read, and the shadow diff gap accepted.

More context for AI agents

Revisions on the fork, before the move:

  • Restacked on master 43333f81da9 on 2026-10-31. Master added template tags in the same view and service; the API update now writes them through the shared save path, and 6b455d0d27 tests a save with tags and other fields.
  • 044401fec2 names the draft-only save of a rewrite.
  • Review fix cc06f9d913: the command now rewrites message_templates before hog_flows, whatever order --targets gives. Before, a default run rewrote the workflow in place first, so the template's sync found nothing to change and the workflow kept its version.
  • Review fixes 9ad607cdf3, 6b4248a7e8, ea7ba66dfa and a62991ca21, QA swarm 2026-11-01:
    • The hog_flows pass rewrites only the draft of a workflow whose live template's published content still holds the old URL when the pass starts. A template that fails keeps its workflow on the old URL, and a rerun writes the new URL as a new workflow version.
    • A workflow whose template's published content holds no old URL, from drift, a draft-only match or a --id <workflow id> run, still gets its live actions rewritten in place, as before this PR.
    • save_locked_template takes an already-changed template; the access check and the change stay in save_template.
    • --dry-run help says it skips the check that a transactional template can still send.
    • Master's comments on the moved save code are back.

After the move:

  • e220b6dd53: each workflow row checks its template under select_for_update before it locks the workflow, through the messaging facade's synced_content_contains. The pass-wide ownership lookup is gone.
  • 99b51d88a6: that check narrows templates by the run's --id, the same scope as the template pass.
  • a12f65d0cd, for a PostHog Review finding: that check filters on the workflow's own team, not the run's teams, so an --all-teams run no longer scans every template per workflow. A template only syncs into a workflow of its own team, so no answer changes.

🤖 Generated with Claude Code

Silthus and others added 12 commits October 31, 2026 09:06
…ts workflow

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…rite changes a template

rewrite_email_asset_url saved message template content directly, so a
transactional template's workflow kept its old copy. Template saves from
the API, the design patch and the rewrite now share one locked
save-and-sync path in services/template_saves.py.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…d draft

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A draft saved before a rewrite kept the old URL, and publishing it brought
that URL back. The rewrite now finds and rewrites draft_content too. A
draft-only rewrite saves the draft alone and writes no workflow version,
because sends never read the draft.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…rl rewrites

The command rewrote workflows first, so by the time a transactional
template synced its workflow the actions already matched and no new
version was written. Message templates now run first, in every run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- when message templates are rewritten in the same run, the hog_flows pass
  rewrites only the drafts of workflows linked to a live template, so a
  template that fails keeps its workflow on the old URL and a rerun writes
  the new URL as a new version
- save_locked_template takes an already-changed template; access checks and
  the change stay in save_template
- --dry-run says it skips the sendability check
- the new messaging code carries no docstrings or comments

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- the hog_flows pass leaves only the workflows of templates this run
  matched to the template rewrite, so a drifted workflow or a run with
  --id <workflow id> rewrites the workflow's live actions
- restore the comments and facade docstrings the template save move kept
  applicable

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e sync

- the hog_flows pass protects the workflows of templates whose published
  content holds the old url, found before the run; a template matched
  only by its draft never syncs, so its workflow's live actions get
  rewritten in place

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…pass

- the hog_flows pass reads which workflows a template content rewrite
  syncs once the template pass is done, so a template deleted during the
  run no longer keeps its workflow's live actions on the old url
- name the facade lookup after the rule it encodes

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@trunk-io

trunk-io Bot commented Nov 2, 2026

Copy link
Copy Markdown

Merging to master in this repository is managed by Trunk.

  • To merge this pull request, check the box to the left or comment /trunk merge below.

After your PR is submitted to the merge queue, this comment will be automatically updated with its status. If the PR fails, failure details will also be posted here

@github-actions

github-actions Bot commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

🤖 CI report

⚠️ Trunk lane — backend Python lane

This PR is assigned to the backend Python lane. It runs backend Python tests and may merge in parallel with PRs in other lanes.

✅ Duplication (Python) — clean

New Python code duplication introduced by this branch. Fails at 70+ tokens in app code, or 150+ tokens when both copies live in test files. Advisory while the gate proves itself: extract a shared helper instead of copying.

✅ Duplication (TypeScript) — clean

New TypeScript code duplication introduced by this branch. Fails at 70+ tokens in app code, or 150+ tokens when both copies live in test files. Advisory while the gate proves itself: extract a shared helper instead of copying.

⚠️ Comment density — 3% of added code lines are comments (16 of 486)

This section warns when comments are more than 3% of the code lines a PR adds, and alerts above 6%. Before agent-assisted PRs, the typical share was about 2%. Only full-line comments count. Docstrings, generated files, snapshots, migrations, and workflow files are left out.

Comments that restate the code, record how the change came about, or narrate the next line add noise for the next reader. Keep the comments that explain a reason the code cannot show, and remove the rest. See .agents/skills/writing-code-comments/SKILL.md for the house rules.

Files with the most added comment lines:

File Comment lines Added lines
products/messaging/backend/services/template_saves.py 7 85
products/workflows/backend/management/commands/rewrite_email_asset_url.py 5 51
products/messaging/backend/services/message_templates.py 3 19
products/messaging/backend/services/transactional_email.py 1 10

This check does not block merging. It updates on every push and clears when the share drops.

✅ Hogbox preview — ready, open the preview

▶ Open the preview

🔑 Login test@posthog.com / 12345678 (demo data)
🧩 Running this PR's backend on the PostHog :master base (frontend unchanged by this PR)
🔗 Link stable across rebuilds: a re-push swaps the box underneath, the URL stays
🔒 Access tailnet only (PostHog VPN)
🛠️ Admin inspect & debug state in hogland
💤 Idle sleeps after ~30 min idle (snapshot to S3, zero node cost) and wakes on your next visit in ~30s, behind a brief "waking up" screen

commit a12f65d · box box-aca742611c74 · ready in 1770s (push → usable) · build log · rebuilds on every push, torn down on close

@hosthog

hosthog Bot commented Nov 2, 2026 •

Copy link
Copy Markdown

HostHog preview — posthog-desktop-web

The previews for this PR have been torn down and no longer serve.

Silthus and others added 3 commits November 2, 2026 14:46
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…lock

The asset rewrite command decided once per pass which transactional
workflows their template still owned. A template save between that
lookup and a workflow's rewrite could put the old URL back into the
template and its new workflow version, and the workflow pass then
rewrote that version in place. The template kept the old URL while
the workflow sent the new one.

Each workflow row now locks the live template that syncs into it,
checks its published content, and only then locks the workflow. That
is the lock order a template save uses. A dry run makes the same check
without the lock.

Also covers a design edit of a transactional template that can no
longer send: it is rejected and rolls back.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With --id set to a workflow id, the template pass matches no template,
so no template sync writes the workflow's new version. The per-row
ownership check still found the linked template and rewrote only the
workflow's draft, which left live sends on the old URL. The check now
narrows templates by the same --id as the template pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@posthog posthog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PostHog Review · Standard · reviewed 99b51d8: 1 Should fix.

See this review and its findings in PostHog Review. Comment @posthog review --deep for a Deep review. A Deep review takes longer and looks deeper than a Standard review.

… team

An --all-teams rewrite checked each matched workflow against every message
template, because transactional_hog_flow_id has no index. A template only
syncs into a workflow of its own team, so filter on the workflow's team and
use the team_id index instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Silthus

Silthus commented Nov 4, 2026

Copy link
Copy Markdown
Contributor Author

Note

🤖 Claude Opus 5.5 responding on behalf of Michael

Closing: no user hits this today. It only affects a narrowed manual run of the asset URL rewrite command, and a default run already rewrites the workflows. The fix would reroute every template save, which isn't worth the risk.

@Silthus Silthus closed this Nov 4, 2026
@Silthus
Silthus deleted the fix/rewrite-email-asset-url-sync branch November 4, 2026 08:00

This branch was successfully deployed

1 active deployment
hogbox-preview — a12f65d0 Deployed Nov 3, 2026 by github-actions[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

hogbox-preview Spin up a per-PR PostHog preview on a hogland hogbox

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant