Repository navigation
feat(headless): agentic monorepo detection behind basic-integration-agentic-detection flag - #884
Conversation
…gentic-detection flag Headless basic-integration runs detect the framework deterministically at the repo root, which handles monorepos poorly. Mirror the self-driving pattern: before the (unchanged) integration program runs, the agentic project detector scans the repo and re-scopes session.installDir to the project it labels recommended — the main client/frontend/mobile app — and the program's own detection then runs inside that directory. - The agentic detector gains an opt-in `recommend` mode: the prompt asks for exactly one recommended project and the coerce layer enforces at most one, stripping the field entirely for scans that didn't opt in (self-driving's picker is byte-identical). - New headless-scope module owns the policy: flag gate, early idempotent auth, choose recommended (even with PostHog present) else first instrumentable, containment-checked path resolve, and fallback to the unchanged install dir on any failure. - The non-interactive runner calls it only on the headless fork; CI and interactive paths never enter the branch. Gated on the `basic-integration-agentic-detection` feature flag (default off), so behavior is unchanged until rollout. Generated-By: PostHog Code Task-Id: 06aaaf67-5aba-497e-bd9e-0220e2417442
🧙 Wizard CIRun the Wizard CI and test your changes against wizard-workbench example apps by replying with a GitHub comment using one of the following commands: Test all apps:
Test all apps in a directory:
Test an individual app:
Show more apps
Results will be posted here when complete. |
…e self-driving Move the scoping out of the shared non-interactive runner: instead of a mode fork with a dynamic import inside runNonInteractive, the headless install entry (ci-install) wraps the untouched integration program with withHeadlessAgenticScope — a ciPreRun preamble that re-points session.installDir, then delegates to the program's own ciPreRun. Same shape as self-driving, which composes integrationRunStep and supplies targetDir: the host decides the directory, the program stays a black box, and the runner stays mechanical. Generated-By: PostHog Code Task-Id: 06aaaf67-5aba-497e-bd9e-0220e2417442
… integration program Replace the entry-point wrapper with a declaration inside the program itself: ciPreRun's first phase runs the agentic scan when session.headless is set (new explicit session field, sibling of session.ci) and the basic-integration-agentic-detection flag is on. The flag gate stays in the scope module, where credentials exist to evaluate it. ci-install returns to stock; the runner's only change is threading headless into the session it builds. Generated-By: PostHog Code Task-Id: 06aaaf67-5aba-497e-bd9e-0220e2417442
…rop headless-scope Replace the headless-specific detection module with a chooser-agnostic one, @lib/detection/integration-projects: scan + classify (moved from self-driving's detect-agentic, now with the recommended flag), chooseProject (auto-choose policy for unattended runs), and resolveProjectDir (containment, previously duplicated as self-driving's integrationDir). Self-driving consumes it through a thin adapter — its picker is just a different chooser over the same report — and a future interactive basic-integration picker takes the same report with recommended as the default selection. The headless plumbing (early auth, flag gate, logging, analytics tags) moves into the basic-integration program, whose policy it is. Generated-By: PostHog Code Task-Id: 06aaaf67-5aba-497e-bd9e-0220e2417442
…can report Drop integration-projects.ts and all relocation of self-driving's mapping layer. The classified IntegrationProject vocabulary is picker-UI machinery the non-interactive path never needed: the raw agentic report already carries everything the pick requires (targetId = supported, hasPostHog, recommended). The program now runs the existing detector with registry targets, chooses via one pure two-find function, and re-points installDir with the same inline containment check self-driving uses. CI runs get the phase too (ciPreRun is non-interactive by definition), so the session.headless field and runner change are gone as well. Self-driving is untouched. Generated-By: PostHog Code Task-Id: 06aaaf67-5aba-497e-bd9e-0220e2417442
Mirror self-driving's layout: the CI/headless-specific detection machinery lives in posthog-integration/detect-agentic.ts (sibling of detect.ts, same shape as self-driving/detect-agentic.ts) and index.ts just imports and calls it — the program config's diff is one import and one line in ciPreRun. Generated-By: PostHog Code Task-Id: 06aaaf67-5aba-497e-bd9e-0220e2417442
Detection logic lives with the other detectors, not in the programs folder: posthog-integration/detect-agentic.ts becomes lib/detection/project-scope.ts (content unchanged), next to agentic.ts and framework.ts. The program keeps only the one-line ciPreRun call. Generated-By: PostHog Code Task-Id: 06aaaf67-5aba-497e-bd9e-0220e2417442
…driving Three fragments were copied from self-driving into project-scope: the registry→targets mapping, the scan purpose string, and the containment resolve from integrationDir. Each now has one home: - detectIntegrationProjects (project-scope.ts) owns targets + purpose; self-driving's detect-agentic calls it and keeps only its picker-facing classification. - resolveProjectDir (agentic.ts, beside coercePath — the caller-side half of the same LLM-path defense) replaces both the inline copy in project-scope and self-driving's integrationDir body. Generated-By: PostHog Code Task-Id: 06aaaf67-5aba-497e-bd9e-0220e2417442
- Tag agentic_detection=flag-off when the gate declines, so every run through the phase carries exactly one tag value. A failed flag fetch surfaces as an empty map, so during rollout a flaky flag endpoint no longer reads as a mysteriously low rollout percentage. - Replace the naming-convention coupling on gateway flag forwarding with an explicit predicate: isGatewayForwardedFlag in constants.ts skips wizard-local flags by name, so renaming the flag to match its wizard-* siblings can't silently start forwarding it. Both harness header builders use it. - Test the scopeInstallDirToProject orchestration: the three fallback exits (flag-off, error-fallback, none-fallback) and both success tags, with only the two network edges mocked. - Spread options first in detectIntegrationProjects so the targets and purpose it exists to own can't be overridden if the options widen. Generated-By: PostHog Code Task-Id: 06aaaf67-5aba-497e-bd9e-0220e2417442
…gging Replace the agentic_detection session tags with one `wizard: agentic detection` event fired on every path through the phase, so funnels can branch on the outcome property (flag-off | error | no-project | recommended | first-instrumentable) with no untracked exits. Success and no-project events carry the scan facts (duration, repo type, project/supported counts, whether the agent recommended anything, and the chosen framework/path); the error path also reports the exception to error tracking via captureException. Generated-By: PostHog Code Task-Id: 06aaaf67-5aba-497e-bd9e-0220e2417442
Generated-By: PostHog Code Task-Id: 06aaaf67-5aba-497e-bd9e-0220e2417442
isGatewayForwardedFlag's skip case was dead code — the flag doesn't start with 'wizard', so the prefix filter already excludes it — and it guarded only a hypothetical rename. If rename-safety ever matters, the right shape is an explicit allow-list of gateway-consumed flags (default-deny), which is its own PR. Both harness header builders are back to stock; the flag key keeps a one-line comment noting it isn't forwarded because it lacks the prefix. Generated-By: PostHog Code Task-Id: 06aaaf67-5aba-497e-bd9e-0220e2417442
Test evidence — headless agentic detectionTested Result: ships clean. 1332/1332 unit tests, detection chose the right project on 18/18 live scans (deterministic on the pick), and the full binary re-scopes a monorepo to the correct sub-app with the flag on. One honest nuance on the flag-off control is written up at the bottom — not a blocker. Test cases
1 · Live agentic detection — 6 repos × 3 runs = 18 scans
The chosen project was identical on all 3 runs of every repo. Only Full classified project arrays (run 1 of the three key repos)2 · ciPreRun phase — flag A/B (the exact code the PR changes)Real
3 · Full wizard binary — real monorepo, run to completionReal Next.js app as
Flag-on landed the SDK, Flag ON — full run log (transient spinner frames removed)Flag OFF — full run log (control)Nuance on the flag-off controlFlag-off still instrumented What the flag actually buys is determinism + the correct framework skill: flag-on deterministically detects Detection reports + logs captured by headless test harness; no PR code changed. |
… detection Documents flipping the basic-integration-agentic-detection flag, its blast radius (non-interactive runs and monorepos only, bounded fallback to root detection, read-only scan, path clamping), the wizard: agentic detection outcome events, and the dev/CI override. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Cut to why it exists, what it does, the flag, the code paths it touches, and what's next; drop the operational deep-dive. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The key skipped the wizard- prefix every other flag uses, so both gateway header builders dropped it and the rollout's LLM traces carried no $feature tag. Renamed to wizard-basic-integration-agentic-detection; the prefix filters now forward it like the rest. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…event The event carried counts and a has_recommendation boolean, so a rejected recommendation and the projects behind "supported_count: 0" were both invisible. Adds the project list (path, framework, target, PostHog, recommended) plus recommended_path / recommended_framework. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
recommended— and we take the recommended supported one, else the first instrumentable PostHog-free one.session.installDiris left untouched and the run falls back to the old root detection, exactly like flag-off.wizard-basic-integration-agentic-detectionflag, default off, so the scan never runs until we roll it out. Flag-off does move one thing: auth now happens at the top ofciPreRunrather than later, since the flag has to evaluate as the logged-in user.ciPreRun) changes; interactive runs are untouched, and it reuses the existing agentic detector (now with an opt-inrecommendmode) that self-driving already shares.wizard: agentic detectionevent so we can watch the outcome mix while rolling out.