Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,8 @@ exports[`getWizardCommandments > matches the published commandment list 1`] = `
"Never hallucinate a PostHog project token, host, or any other secret. Always use the real values that have been configured for this project (for example via environment variables).
Never write API keys, access tokens, or other secrets directly into source code. Always reference environment variables instead, and rely on the wizard-tools MCP server (check_env_keys / set_env_values) to create or update .env files.
Always use the detect_package_manager tool from the wizard-tools MCP server to determine the package manager. Do not guess based on lockfiles or hard-code npm, yarn, pnpm, bun, pip, etc.
Before writing to any file, you MUST read that exact file immediately beforehand using the Read tool, even if you have already read it earlier in the run. This avoids tool failures and stale edits.
Before writing to any file that already exists, you MUST read that exact file immediately beforehand using the Read tool, even if you have already read it earlier in the run. This avoids tool failures and stale edits. Brand-new files are the exception: create them directly — a read of a missing file can only fail and is never required first.
Never spend turns deleting or emptying wizard bookkeeping files such as \`.posthog-events.json\`. If a skill's cleanup step asks you to remove one, skip that step and move on — the wizard removes its own artifacts host-side after the run.
Treat feature flags, custom properties, and event names as part of an analytics contract. Prefer reusing existing names and patterns in the project. When you must introduce new ones, make them clear, descriptive, and consistent with existing conventions, and avoid scattering the same flag or property across many unrelated callsites.
Keep PostHog data capture at its defaults unless the user explicitly asks otherwise. Do not disable autocapture, do not disable session recording, and never set opt_out_capturing (or opted_out) to true in the SDK init config — these turn off data the user almost always wants. Note: posthog.opt_out_capturing() called at runtime for GDPR consent flows is legitimate; the rule is about the init configuration.
Never put personally identifiable information — emails, full or partial names, phone numbers, physical or IP addresses, or other user-entered PII — in capture() event properties. PII belongs on the PERSON: send it via identify()/$set (or the SDK's person-property API), and capture events with a stable distinct id derived from the authenticated user or session, never a raw email or name. This holds for every SDK, client and server.
Expand Down
23 changes: 23 additions & 0 deletions src/lib/agent/__tests__/commandments.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,29 @@ describe('getWizardCommandments', () => {
expect(getWizardCommandments()).toMatchSnapshot();
});

// The two rules below answer the most common end-of-run agent remarks
// (wizard#793 rollout): "do new files need a prior read?" and "how do I
// delete .posthog-events.json without rm?". Locked down so a future edit
// can't silently reintroduce the ambiguity.
describe('file handling rules', () => {
const text = getWizardCommandments();

it('scopes read-before-write to files that already exist', () => {
expect(text).toMatch(/any file that already exists/i);
});

it('exempts brand-new files from the prior read', () => {
expect(text).toMatch(/brand-new files are the exception/i);
expect(text).toMatch(/never required first/i);
});

it('tells the agent to leave wizard bookkeeping files alone', () => {
expect(text).toMatch(/\.posthog-events\.json/);
expect(text).toMatch(/skip that step/i);
expect(text).toMatch(/host-side after the run/i);
});
});

// Targeted assertions for the wizard_ask Path A translation rules.
// These are the rules a skill author depends on when leaving their prose
// unchanged — they need to keep working as the commandment list evolves.
Expand Down
4 changes: 3 additions & 1 deletion src/lib/agent/commandments.ts
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,9 @@ const WIZARD_COMMANDMENTS = [

'Always use the detect_package_manager tool from the wizard-tools MCP server to determine the package manager. Do not guess based on lockfiles or hard-code npm, yarn, pnpm, bun, pip, etc.',

'Before writing to any file, you MUST read that exact file immediately beforehand using the Read tool, even if you have already read it earlier in the run. This avoids tool failures and stale edits.',
'Before writing to any file that already exists, you MUST read that exact file immediately beforehand using the Read tool, even if you have already read it earlier in the run. This avoids tool failures and stale edits. Brand-new files are the exception: create them directly — a read of a missing file can only fail and is never required first.',

"Never spend turns deleting or emptying wizard bookkeeping files such as `.posthog-events.json`. If a skill's cleanup step asks you to remove one, skip that step and move on — the wizard removes its own artifacts host-side after the run.",

'Treat feature flags, custom properties, and event names as part of an analytics contract. Prefer reusing existing names and patterns in the project. When you must introduce new ones, make them clear, descriptive, and consistent with existing conventions, and avoid scattering the same flag or property across many unrelated callsites.',

Expand Down
12 changes: 0 additions & 12 deletions src/lib/agent/runner/harness/pi/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,6 @@
* follow-ups (#525, #524 skills) — v1 uses pi's built-in coding tools.
*/

import fs from 'fs';
import path from 'path';
import { getUI } from '@ui';
import { getLogFilePath, logToFile } from '@utils/debug';
import { getLlmGatewayUrl } from '@utils/urls';
Expand Down Expand Up @@ -566,16 +564,6 @@ export const piBackend: AgentHarness = {
analytics.capture(WIZARD_REMARK_EVENT_NAME, { remark });
}

// The skill plans events into .posthog-events.json then asks to remove it
// on completion; pi's `rm` is fence-blocked, so the agent can't — clean it
// up host-side rather than leave a stale (often empty) artifact (#15).
try {
const planFile = path.join(session.installDir, '.posthog-events.json');
if (fs.existsSync(planFile)) await fs.promises.rm(planFile);
} catch (err) {
logToFile(`[pi] .posthog-events.json cleanup skipped: ${String(err)}`);
}

const stats = agentSession.getSessionStats();
analytics.wizardCapture('agent completed', {
...runDurations(),
Expand Down
6 changes: 6 additions & 0 deletions src/lib/agent/runner/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ import {
type SwitchboardCtx,
} from './switchboard';
import { flushScanReport } from '../../yara-hooks';
import { removeProgramArtifacts } from './shared/artifacts';
import { registerCleanup } from '../../../utils/wizard-abort';

export type {
Expand Down Expand Up @@ -93,6 +94,10 @@ export async function runProgram(
// flushScanReport is idempotent (it zeroes scan state), so the overlap is a
// harmless no-op. No harness has to know reporting exists.
registerCleanup(() => flushScanReport(session));
// Same dual coverage for the program's declared run artifacts
// (ProgramConfig.cleanupArtifacts): removal is idempotent, so the
// abort-path/finally overlap is a harmless no-op.
registerCleanup(() => removeProgramArtifacts(session, programConfig));
try {
const binding = resolveProgramRunner(session, programConfig, boot);
if (binding.sequence === Sequence.orchestrator) {
Expand All @@ -107,6 +112,7 @@ export async function runProgram(
);
} finally {
flushScanReport(session);
removeProgramArtifacts(session, programConfig);
}
}

Expand Down
90 changes: 90 additions & 0 deletions src/lib/agent/runner/shared/__tests__/artifacts.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
import * as fs from 'fs';
import * as os from 'os';
import * as path from 'path';
import { removeProgramArtifacts } from '@lib/agent/runner/shared/artifacts';
import { buildSession } from '@lib/wizard-session';
import type { ProgramConfig } from '@lib/programs/program-step';
import { posthogIntegrationConfig } from '@lib/programs/posthog-integration/index';
import { EVENT_PLAN_FILE } from '@lib/programs/posthog-integration/constants';

function makeConfig(cleanupArtifacts?: readonly string[]): ProgramConfig {
return {
description: 'test program',
id: 'test-program',
steps: [],
cleanupArtifacts,
};
}

describe('removeProgramArtifacts', () => {
let installDir: string;

beforeEach(() => {
installDir = fs.mkdtempSync(path.join(os.tmpdir(), 'wizard-artifacts-'));
});

afterEach(() => {
fs.rmSync(installDir, { recursive: true, force: true });
});

it('removes a declared artifact left behind by the run', () => {
const planFile = path.join(installDir, EVENT_PLAN_FILE);
fs.writeFileSync(planFile, JSON.stringify({ events: [] }));

removeProgramArtifacts(
buildSession({ installDir }),
makeConfig([EVENT_PLAN_FILE]),
);

expect(fs.existsSync(planFile)).toBe(false);
});

it('is a no-op when the artifact does not exist', () => {
expect(() =>
removeProgramArtifacts(
buildSession({ installDir }),
makeConfig([EVENT_PLAN_FILE]),
),
).not.toThrow();
});

it('is a no-op when the program declares no artifacts', () => {
const bystander = path.join(installDir, 'index.js');
fs.writeFileSync(bystander, 'console.log("hi")');

removeProgramArtifacts(buildSession({ installDir }), makeConfig());

expect(fs.existsSync(bystander)).toBe(true);
});

it('refuses paths that escape the install dir', () => {
const outside = path.join(os.tmpdir(), `wizard-outside-${process.pid}`);
fs.writeFileSync(outside, 'do not touch');
try {
removeProgramArtifacts(
buildSession({ installDir }),
makeConfig([path.relative(installDir, outside)]),
);
expect(fs.existsSync(outside)).toBe(true);
} finally {
fs.rmSync(outside, { force: true });
}
});

it('never targets the install dir itself', () => {
removeProgramArtifacts(buildSession({ installDir }), makeConfig(['.']));
expect(fs.existsSync(installDir)).toBe(true);
});
});

// Contract: the basic-integration program owns the event-plan file, so its
// config must declare it for host-side cleanup. Skills tell the agent to skip
// the delete step (commandments) — this declaration is what actually removes
// the file after the run.
describe('posthog-integration cleanupArtifacts contract', () => {
it('declares the event plan file', () => {
expect(posthogIntegrationConfig.cleanupArtifacts).toContain(
EVENT_PLAN_FILE,
);
});
});
54 changes: 54 additions & 0 deletions src/lib/agent/runner/shared/artifacts.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
/**
* Host-side cleanup of the bookkeeping files a program's agent run leaves
* behind (declared as `ProgramConfig.cleanupArtifacts`).
*
* The agent can't do this itself: `rm` is fence-blocked on the pi harness and
* skills' "delete the plan file" steps burned turns on delete-vs-empty
* workarounds. The runner owns removal instead — this runs at the
* `runProgram` seam on every termination path, for every harness, and a
* commandment tells the agent to skip skill cleanup steps entirely.
*
* Infrastructure only: the file names are program knowledge and stay in each
* program's config. Removal is idempotent (missing files are a no-op) so the
* abort-path and finally-path overlap is harmless.
*/

import * as fs from 'fs';
import * as path from 'path';
import { logToFile } from '@utils/debug';
import type { WizardSession } from '@lib/wizard-session';
import type { ProgramConfig } from '@lib/programs/program-step';

export function removeProgramArtifacts(
session: WizardSession,
programConfig: ProgramConfig,
): void {
const artifacts = programConfig.cleanupArtifacts ?? [];
if (artifacts.length === 0) return;

const root = path.resolve(session.installDir);
for (const relPath of artifacts) {
const target = path.resolve(root, relPath);
// Declared paths are config, not user input, but keep removal strictly
// inside the install dir so a bad config entry can never reach out of the
// project (or delete the project root itself).
if (!target.startsWith(root + path.sep)) {
logToFile(
`[agent-runner] artifact cleanup refused (outside install dir): ${relPath}`,
);
continue;
}
try {
if (fs.existsSync(target)) {
fs.rmSync(target, { force: true });
logToFile(`[agent-runner] removed run artifact: ${relPath}`);
}
} catch (err) {
logToFile(
`[agent-runner] artifact cleanup skipped for ${relPath}: ${String(
err,
)}`,
);
}
}
}
5 changes: 5 additions & 0 deletions src/lib/programs/posthog-integration/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ import { requestDeepLink } from '@utils/provisioning';
import { openTrackedLink, withUtm } from '@utils/links';
import type { CloudRegion } from '@utils/types';
import { POSTHOG_INTEGRATION_PROGRAM } from './steps.js';
import { EVENT_PLAN_FILE } from './constants.js';
import { getContentBlocks } from './content/index.js';
import { buildCodingAgentPrompt } from './handoff.js';

Expand Down Expand Up @@ -53,6 +54,10 @@ export const posthogIntegrationConfig: ProgramConfig = {
id: 'posthog-integration',
steps: POSTHOG_INTEGRATION_PROGRAM,
getContentBlocks,
// The skill writes its event plan here; the runner deletes it host-side
// after the run (see ProgramConfig.cleanupArtifacts), so the agent never
// has to fight the fenced `rm` to clean it up.
cleanupArtifacts: [EVENT_PLAN_FILE],
// Basic integration runs without structured user input; drop wizard_ask
// so the model can't pop modal prompts mid-run. The runner forwards this
// list to the general-purpose subagent as well, so dispatched subagents
Expand Down
8 changes: 8 additions & 0 deletions src/lib/programs/program-step.ts
Original file line number Diff line number Diff line change
Expand Up @@ -235,6 +235,14 @@ export interface ProgramConfig {
* read it synchronously without resolving a deferred `run` function.
*/
reportFile?: string;
/**
* Bookkeeping files (relative to installDir) this program's agent run
* leaves behind — e.g. the event plan the skill writes and then cannot
* reliably delete because `rm` is fenced. The runner removes them
* host-side on every termination path, for every harness, so no skill
* or commandment has to teach the agent how to clean up.
*/
cleanupArtifacts?: readonly string[];
/**
* LearnCard deck rendered in the shared `RunScreen` while the agent
* runs. Lives at `<program>/content/index.tsx` by convention.
Expand Down
Loading