Skip to content

docs: fix local stack setup and stale claims in READMEs - #4390

Open
gewenyu99 wants to merge 2 commits into
mainfrom
posthog/docs-accuracy-fixes
Open

gewenyu99 wants to merge 2 commits into
mainfrom
posthog/docs-accuracy-fixes

Conversation

@gewenyu99

@gewenyu99 gewenyu99 commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

Problem

The README's local stack steps don't produce a working wizard run: ./bin/start plus --local-posthog has no gateway, and the local LLM gateway section is stale.

Changes

  • New "Point wizard at local PostHog and the AI gateway" section: hogli dev:apply product_analytics --include ai-gateway, hogli up, the WIZARD_GATEWAY_* mint settings, phrocs, pnpm try ... --local-posthog --local-context-mill. Replaces the tsdown NODE_ENV and :3308 instructions
  • MCP config lives in services/mcp/.env, not .dev.vars
  • phrocs keys (s start, x stop), picker labels, apps/manifest.json registry, env table (POSTHOG_GATEWAY_TOKEN and others), --region scope, workflow app input paths, owners table synced to CODEOWNERS
  • mcp-stub READMEs: pi MCP client path, what the recorder actually redacts. ai-observability: six weather apps
  • Development keys: pass POSTHOG_PERSONAL_API_KEY (phx_, MCP access) and POSTHOG_GATEWAY_TOKEN (phs_ gateway service key, an op:// reference to 1Password run through op run). .env.example updated to match

Test plan

Docs only. The local flow matches a verified end-to-end run on 2026-09-24. Other claims were checked against origin/main of the workbench, the wizard and posthog.

🤖 Generated with Claude Code

gewenyu99 and others added 2 commits October 6, 2026 11:44
Checked against origin/main code. Docs only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@gewenyu99
gewenyu99 marked this pull request as ready for review October 6, 2026 16:05
@gewenyu99
gewenyu99 requested review from a team as code owners October 6, 2026 16:05

@johncwaters johncwaters left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

Automated review. Not written by a human.

Comment thread .env.example
POSTHOG_GATEWAY_TOKEN=phs_...
# Gateway service key for model calls: a phs_ project secret key with the llm_gateway:read scope.
# It's in 1Password. Keep the op:// reference and run with `op run --env-file .env -- pnpm ...`.
POSTHOG_GATEWAY_TOKEN=op://<vault>/<item>/credential

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

Automated review. Not written by a human.

This placeholder isn't valid shell. Bash parses <vault> and <item> as redirections, so source .env fails with vault: No such file or directory and returns non-zero. Every phrocs pane that runs source .env && ... (context-mill, mcp-inspector, wizard-build, wizard-build-dev, playground) then won't start after a fresh cp .env.example .env. Real op:// references with spaces in the item name break the same way.

Fix: quote it and drop the angle brackets, e.g. POSTHOG_GATEWAY_TOKEN="op://vault/item/credential".

Comment thread README.md
| `POSTHOG_REGION` | No | PostHog region (`us` or `eu`). Defaults to `us`. Can also be set via `--region` flag or workflow input. |
| `POSTHOG_PERSONAL_API_KEY` | For development | PostHog personal API key (`phx_`). Gives the wizard PostHog API and MCP access in CI mode, `--e2e` and `mcp-stub:record`. |
| `POSTHOG_WIZARD_PROJECT_ID` | For `--e2e` | The project the personal API key is scoped to. Used by `pnpm wizard-ci --e2e` (or pass `--project-id`) and `pnpm wizard-ci-snapshots`. |
| `POSTHOG_GATEWAY_TOKEN` | For development | The gateway service key for model calls: a `phs_` project secret key with the `llm_gateway:read` scope. It's in 1Password. Set it to `op://<vault>/<item>/credential` and run with `op run --env-file .env -- pnpm ...`. The PR evaluator and `pnpm wizard-ci --e2e` fail without it. |

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

Automated review. Not written by a human.

If .env keeps the unresolved op:// reference, the phrocs panes that need the token (wizard-ci-run --evaluate, wizard-ci-evaluate-pr, wizard-ci-snapshots) load it through env_file without op run. They get the literal op://... string, which passes the non-empty checks in services/pr-evaluator/evaluator.ts and services/wizard-ci/e2e.ts and goes to the gateway as the token, so you get a gateway auth error instead of the clear "no gateway token" message.

Fix: document starting phrocs as op run --env-file .env -- phrocs, or say these panes need a resolved key.

@Radu-Raicea
Radu-Raicea removed the request for review from a team October 7, 2026 09:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants