Skip to content

docs(agents): say the agent prompts are frozen and edited in the wizard - #530

Closed
edwinyjlim wants to merge 1 commit into
dag/c3-step-skillsfrom
dag/c4-freeze-note
Closed

edwinyjlim wants to merge 1 commit into
dag/c3-step-skillsfrom
dag/c4-freeze-note

Conversation

@edwinyjlim

@edwinyjlim edwinyjlim commented Nov 13, 2026 •

Copy link
Copy Markdown
Member

Replaced by #532, the same change on its own, based on main.

Summary

The note at the top of context/agents/README.md now says the agent prompts are frozen: the wizard reads its own copy at src/programs/<id>/agent/, and a prompt is edited there. This copy stays published, unchanged, for wizard versions that fetch their prompts.

Merge this when wizard-agent-dag reaches 100%.

Stack

The second half of the migration, behind wizard-agent-dag-skills. The first half, behind wizard-agent-dag, is in context-mill 1.65.0 (#447) and wizard 2.94.0 (PostHog/wizard#1551 to PostHog/wizard#1554).

PR Repo What it does How it fits in
A · Variant table · #528 context-mill Each integration variant lists every product's pages, and its pack ships them with a variant.json. On main. Adds files and tags only.
B · Step skills · #529 context-mill Publishes each orchestrator step skill once, as integration-<step>. On A. Adds skills only, and the skills they replace keep publishing.
C · Freeze note · #532 context-mill Says the agent prompts are frozen and edited in the wizard. On main, on its own. Documentation only. Merges when wizard-agent-dag reaches 100%.
PR 5 · Read repackaged skills · PostHog/wizard#1559 wizard With wizard-agent-dag-skills on, a task reads the step skill from B, completed from the pack from A. On main. Does nothing while the flag is off.

A and B are one GitHub stack and merge together. They only add files, skills and tags, and no run reads them until the flag is on.

Rollout

wizard-agent-dag-skills is a percentage flag, independent of wizard-agent-dag. Turning it off returns every run to the skills it reads today, and a run pinned to a context-mill release reads those throughout.

Step What changes for a run What is checked
1. A and B merge, and context-mill releases Nothing. The packs gain files and the menu gains entries that no run reads yet check:bundle-diff --additive and check:repackaged-skills on the release build
2. PR 5 merges, and the wizard releases with the flag off Nothing A run with the flag off matches the previous release
3. The flag is on for PostHog staff Staff runs read the repackaged skills pnpm verify:repackaged-skills against the release, and a workbench run with the flag on compared with one with it off. The error tracking difference on Nuxt, described in PostHog/wizard#1559, is settled
4. The flag is at 50%, for about a day Half of runs read the repackaged skills The two sides by the orchestrator_skills tag: run success, task failures, and any orchestrator repackaged skill unavailable
5. The flag is at 100% Every run reads the repackaged skills The same signals, for about a week
6. Cleanup Nothing A later PR deletes the flag and the old lookup. Context-mill stops publishing the replaced skills once older wizard versions have drained

C merges on its own, when wizard-agent-dag reaches 100%.

Test plan

  • Documentation only

🤖 Generated with Claude Code

@github-actions

Copy link
Copy Markdown

🧙 Wizard CI

Run the Wizard CI and test your changes against wizard-workbench example apps by replying with a GitHub comment using one of the following commands:

Test all apps:

  • /wizard-ci all

Test all apps in a directory:

  • /wizard-ci ai-observability
  • /wizard-ci basic-integration
  • /wizard-ci feature-flags
  • /wizard-ci mcp-analytics
  • /wizard-ci replay-vision
  • /wizard-ci revenue
  • /wizard-ci self-driving
  • /wizard-ci warehouse
  • /wizard-ci warehouse-seeded

Test an individual app:

  • /wizard-ci ai-observability/anthropic
  • /wizard-ci ai-observability/google-adk
  • /wizard-ci ai-observability/groq
Show more apps
  • /wizard-ci ai-observability/manual-capture
  • /wizard-ci ai-observability/openai
  • /wizard-ci ai-observability/openai-agents
  • /wizard-ci ai-observability/opentelemetry
  • /wizard-ci ai-observability/vercel-ai
  • /wizard-ci basic-integration/android
  • /wizard-ci basic-integration/angular
  • /wizard-ci basic-integration/astro
  • /wizard-ci basic-integration/django
  • /wizard-ci basic-integration/fastapi
  • /wizard-ci basic-integration/flask
  • /wizard-ci basic-integration/flutter
  • /wizard-ci basic-integration/javascript-node
  • /wizard-ci basic-integration/javascript-web
  • /wizard-ci basic-integration/laravel
  • /wizard-ci basic-integration/next-js
  • /wizard-ci basic-integration/nuxt
  • /wizard-ci basic-integration/python
  • /wizard-ci basic-integration/rails
  • /wizard-ci basic-integration/react-native
  • /wizard-ci basic-integration/react-router
  • /wizard-ci basic-integration/sveltekit
  • /wizard-ci basic-integration/swift
  • /wizard-ci basic-integration/tanstack-router
  • /wizard-ci basic-integration/tanstack-start
  • /wizard-ci basic-integration/vue
  • /wizard-ci feature-flags/django
  • /wizard-ci feature-flags/next-js
  • /wizard-ci mcp-analytics/custom-dispatcher
  • /wizard-ci mcp-analytics/go-sdk
  • /wizard-ci mcp-analytics/python-sdk
  • /wizard-ci mcp-analytics/ruby-sdk
  • /wizard-ci mcp-analytics/rust-sdk
  • /wizard-ci mcp-analytics/typescript-sdk
  • /wizard-ci replay-vision/javascript-node
  • /wizard-ci replay-vision/next-js
  • /wizard-ci replay-vision/react-vite
  • /wizard-ci revenue/stripe
  • /wizard-ci self-driving/astro
  • /wizard-ci self-driving/fastapi
  • /wizard-ci self-driving/nuxt
  • /wizard-ci self-driving/react-router
  • /wizard-ci self-driving/sveltekit
  • /wizard-ci warehouse/monorepo-env
  • /wizard-ci warehouse/multi-source-next
  • /wizard-ci warehouse/stripe-node
  • /wizard-ci warehouse/zero-source
  • /wizard-ci warehouse-seeded/next-stripe
  • /wizard-ci warehouse-seeded/next-stripe-declined

Test against a wizard branch:

  • /wizard-ci all wizard:my-branch

Add wizard:<branch> to any command above to pin the wizard branch. It defaults to main.

Results will be posted here when complete.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant