Skip to content

feat(wizard-ci): let a PR comment pin the context-mill ref - #1153

Merged
edwinyjlim merged 3 commits into
mainfrom
tom/wizard-ci-companion-ref
Aug 27, 2026
Merged

edwinyjlim merged 3 commits into
mainfrom
tom/wizard-ci-companion-ref

Conversation

@Gilbert09

Copy link
Copy Markdown
Member

Problem

The workbench wizard-ci workflow already accepts wizard_ref and context_mill_ref on the same dispatch channel. This trigger sent only wizard_ref, and the context-mill trigger sends only context_mill_ref. So a change that spans both repos cannot be exercised before merge.

That is not hypothetical. #1146 changes the credential-prompt cap in the runtime, and PostHog/context-mill#360 changes the skill text that has to match it. They are one contract in two repos, and today a reviewer cannot run them together.

Changes

/wizard-ci <app> and /wizard-ci-snapshots <app> [ref] accept an optional context-mill:<ref> suffix. The trigger validates the ref, then adds context_mill_ref to the dispatch payload. Without the suffix the key is omitted entirely, so the workbench applies its own main default and existing behaviour is byte-identical.

The app regex is untouched. One nicety: a lone pin such as /wizard-ci context-mill:foo would otherwise read as an app name, so it now falls back to all.

Ref validation matters here. The workbench interpolates the ref straight into a JS string literal inside github-script, in a job that holds the App token. The allowlist is [A-Za-z0-9._/-], max 200 characters, no .., no leading -, no leading or trailing /. That excludes every quote, backtick, backslash, $ and ;, so the literal cannot be broken out of. It also blocks path traversal and git option injection such as --upload-pack=. An invalid ref calls core.setFailed and dispatches nothing, rather than silently falling back to main.

/wizard-ci-snapshots already accepted a positional wizard_ref that was never validated, feeding the same interpolation. Since this touches that parser, it now gets the same check. That is the one place an existing input changes behaviour, and only for refs that could never have checked out anyway.

Residual risk is unchanged from what wizard_ref already allowed: a commenter can point CI at a branch that exists and burn CI minutes. No token scope is widened.

Test plan

  • actionlint: 4 shellcheck findings, byte-identical to the pre-change baseline, all in an untouched bash step. No new findings.
  • YAML parses; all three job keys intact.
  • Behavioural: the script: blocks were extracted from the YAML and executed in Node against stubbed GitHub APIs, capturing the dispatch payload for 27 command forms. All four legacy forms produce identical payloads with no context_mill_ref key. All pin forms attach the key. Eleven hostile refs — quote breakout, ../../etc/passwd, --upload-pack=, $(id), backticks, backslash, over-length, absolute, trailing slash — all fail with no dispatch.

Companion PR: PostHog/context-mill mirrors this with a wizard:<ref> suffix.

The workbench wizard-ci workflow already accepts wizard_ref and
context_mill_ref on the same dispatch channel. This trigger sent only
wizard_ref. A change that spans both repos could not run before merge.

Add an optional `context-mill:<ref>` suffix to /wizard-ci and to
/wizard-ci-snapshots. The trigger checks the ref against a git ref
allowlist. It then adds context_mill_ref to the dispatch payload.
Without the suffix the trigger omits the key. The workbench then keeps
its own default of `main`.

Also check the positional wizard ref that /wizard-ci-snapshots already
accepted. Show the new syntax in the welcome comment.
@Gilbert09
Gilbert09 requested a review from a team as a code owner August 27, 2026 14:30
@github-actions

Copy link
Copy Markdown

🧙 Wizard CI

Run the Wizard CI and test your changes against wizard-workbench example apps by replying with a GitHub comment using one of the following commands:

Test all apps:

  • /wizard-ci all

Test all apps in a directory:

  • /wizard-ci ai-observability
  • /wizard-ci basic-integration
  • /wizard-ci mcp-analytics
  • /wizard-ci replay-vision
  • /wizard-ci revenue
  • /wizard-ci self-driving

Test an individual app:

  • /wizard-ci ai-observability/anthropic
  • /wizard-ci ai-observability/groq
  • /wizard-ci ai-observability/manual-capture
Show more apps
  • /wizard-ci ai-observability/openai
  • /wizard-ci ai-observability/openai-agents
  • /wizard-ci ai-observability/vercel-ai
  • /wizard-ci basic-integration/android
  • /wizard-ci basic-integration/angular
  • /wizard-ci basic-integration/astro
  • /wizard-ci basic-integration/django
  • /wizard-ci basic-integration/fastapi
  • /wizard-ci basic-integration/flask
  • /wizard-ci basic-integration/flutter
  • /wizard-ci basic-integration/javascript-node
  • /wizard-ci basic-integration/javascript-web
  • /wizard-ci basic-integration/laravel
  • /wizard-ci basic-integration/next-js
  • /wizard-ci basic-integration/nuxt
  • /wizard-ci basic-integration/python
  • /wizard-ci basic-integration/rails
  • /wizard-ci basic-integration/react-native
  • /wizard-ci basic-integration/react-router
  • /wizard-ci basic-integration/sveltekit
  • /wizard-ci basic-integration/swift
  • /wizard-ci basic-integration/tanstack-router
  • /wizard-ci basic-integration/tanstack-start
  • /wizard-ci basic-integration/vue
  • /wizard-ci mcp-analytics/custom-dispatcher
  • /wizard-ci mcp-analytics/typescript-sdk
  • /wizard-ci replay-vision/javascript-node
  • /wizard-ci replay-vision/next-js
  • /wizard-ci replay-vision/react-vite
  • /wizard-ci revenue/stripe
  • /wizard-ci self-driving/astro
  • /wizard-ci self-driving/fastapi
  • /wizard-ci self-driving/nuxt
  • /wizard-ci self-driving/react-router
  • /wizard-ci self-driving/sveltekit

Test against a Context Mill branch:

  • /wizard-ci all context-mill:my-branch

Add context-mill:<branch> to any command above to pin the Context Mill branch. It defaults to main.

Results will be posted here when complete.

Comment thread .github/workflows/wizard-ci-trigger.yml Resolved
@veria-ai

veria-ai Bot commented Aug 27, 2026 •

Copy link
Copy Markdown

PR overview

All previously flagged issues have been addressed. No open security concerns remain on this pull request.

Security review

No open security issues remain on this pull request.

Fixed/addressed: 1 · PR risk: 0/10

@edwinyjlim edwinyjlim left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

made some upstream fixes to address veria comments

@edwinyjlim
edwinyjlim merged commit b494975 into main Aug 27, 2026
17 checks passed
@edwinyjlim
edwinyjlim deleted the tom/wizard-ci-companion-ref branch August 27, 2026 20:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants