Skip to content

feat(wizard-ci): let a PR comment pin the wizard ref - #366

Merged
edwinyjlim merged 2 commits into
mainfrom
tom/wizard-ci-companion-ref
Aug 27, 2026
Merged

edwinyjlim merged 2 commits into
mainfrom
tom/wizard-ci-companion-ref

Conversation

@Gilbert09

Copy link
Copy Markdown
Member

Problem

The workbench wizard-ci workflow already accepts wizard_ref and context_mill_ref on the same dispatch channel. This trigger sent only context_mill_ref, and the wizard trigger sends only wizard_ref. So a change that spans both repos cannot be exercised before merge.

That is not hypothetical. #360 changes the warehouse skill's credential-batching instruction, and PostHog/wizard#1146 changes the runtime cap it has to match. They are one contract in two repos, and today a reviewer cannot run them together.

Changes

/wizard-ci <app> accepts an optional wizard:<ref> suffix. The trigger validates the ref, then adds wizard_ref to the dispatch payload. Without the suffix the key is omitted entirely, so the workbench applies its own main default and existing behaviour is byte-identical.

The app regex is untouched. A lone pin such as /wizard-ci wizard:foo falls back to all rather than reading the pin as an app name.

Ref validation matters here. The workbench interpolates the ref straight into a JS string literal inside github-script, in a job that holds the App token. The allowlist is [A-Za-z0-9._/-], max 200 characters, no .., no leading -, no leading or trailing /. That excludes every quote, backtick, backslash, $ and ;, so the literal cannot be broken out of. It also blocks path traversal and git option injection such as --upload-pack=. An invalid ref calls core.setFailed and dispatches nothing, rather than silently falling back to main.

Residual risk is unchanged from what context_mill_ref already allowed: a commenter can point CI at a branch that exists and burn CI minutes. No token scope is widened.

Test plan

  • actionlint: findings byte-identical to the pre-change baseline, all in untouched steps.
  • YAML parses; both job keys intact.
  • Behavioural: the script: block was extracted from the YAML and executed in Node against stubbed GitHub APIs, capturing the dispatch payload across every command form. Legacy forms produce identical payloads with no wizard_ref key; pin forms attach it; hostile refs fail with no dispatch.

Companion PR: PostHog/wizard#1153 mirrors this with a context-mill:<ref> suffix.

The workbench wizard-ci workflow already accepts wizard_ref and
context_mill_ref on the same dispatch channel. This trigger sent only
context_mill_ref. A change that spans both repos could not run before
merge.

Add an optional `wizard:<ref>` suffix to /wizard-ci. The trigger checks
the ref against a git ref allowlist. It then adds wizard_ref to the
dispatch payload. Without the suffix the trigger omits the key. The
workbench then keeps its own default of `main`.

Show the new syntax in the welcome comment.
@Gilbert09
Gilbert09 requested a review from a team as a code owner August 27, 2026 14:30
@github-actions

Copy link
Copy Markdown

🧙 Wizard CI

Run the Wizard CI and test your changes against wizard-workbench example apps by replying with a GitHub comment using one of the following commands:

Test all apps:

  • /wizard-ci all

Test all apps in a directory:

  • /wizard-ci ai-observability
  • /wizard-ci basic-integration
  • /wizard-ci mcp-analytics
  • /wizard-ci replay-vision
  • /wizard-ci revenue
  • /wizard-ci self-driving

Test an individual app:

  • /wizard-ci ai-observability/anthropic
  • /wizard-ci ai-observability/groq
  • /wizard-ci ai-observability/manual-capture
Show more apps
  • /wizard-ci ai-observability/openai
  • /wizard-ci ai-observability/openai-agents
  • /wizard-ci ai-observability/vercel-ai
  • /wizard-ci basic-integration/android
  • /wizard-ci basic-integration/angular
  • /wizard-ci basic-integration/astro
  • /wizard-ci basic-integration/django
  • /wizard-ci basic-integration/fastapi
  • /wizard-ci basic-integration/flask
  • /wizard-ci basic-integration/flutter
  • /wizard-ci basic-integration/javascript-node
  • /wizard-ci basic-integration/javascript-web
  • /wizard-ci basic-integration/laravel
  • /wizard-ci basic-integration/next-js
  • /wizard-ci basic-integration/nuxt
  • /wizard-ci basic-integration/python
  • /wizard-ci basic-integration/rails
  • /wizard-ci basic-integration/react-native
  • /wizard-ci basic-integration/react-router
  • /wizard-ci basic-integration/sveltekit
  • /wizard-ci basic-integration/swift
  • /wizard-ci basic-integration/tanstack-router
  • /wizard-ci basic-integration/tanstack-start
  • /wizard-ci basic-integration/vue
  • /wizard-ci mcp-analytics/custom-dispatcher
  • /wizard-ci mcp-analytics/typescript-sdk
  • /wizard-ci replay-vision/javascript-node
  • /wizard-ci replay-vision/next-js
  • /wizard-ci replay-vision/react-vite
  • /wizard-ci revenue/stripe
  • /wizard-ci self-driving/astro
  • /wizard-ci self-driving/fastapi
  • /wizard-ci self-driving/nuxt
  • /wizard-ci self-driving/react-router
  • /wizard-ci self-driving/sveltekit

Test against a wizard branch:

  • /wizard-ci all wizard:my-branch

Add wizard:<branch> to any command above to pin the wizard branch. It defaults to main.

Results will be posted here when complete.

@edwinyjlim
edwinyjlim merged commit 6d791cf into main Aug 27, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants