Repository navigation
Conversation
…ad-before-write rule End-of-run agent remarks from the wizard#793 rollout showed two dominant frictions: agents burning turns figuring out how to delete .posthog-events.json with rm fenced (delete vs empty, write-tools-only workarounds), and ambiguity over whether brand-new files need a prior Read before writing. - ProgramConfig.cleanupArtifacts: programs declare the bookkeeping files their run leaves behind; the runner removes them at the runProgram seam on every termination path, for every harness (same dual registerCleanup/finally coverage as flushScanReport). Replaces the pi-harness-only inline cleanup. - posthog-integration declares EVENT_PLAN_FILE. - New commandment: leave wizard bookkeeping files in place; skip skill cleanup steps that ask to remove them. - Read-before-write commandment now scopes to files that already exist, with an explicit brand-new-file exception. - Contract/unit tests for both; commandments snapshot updated. Generated-By: PostHog Code Task-Id: 9b33e996-4d8c-4906-8405-63fb4bcd7d07
🧙 Wizard CIRun the Wizard CI and test your changes against wizard-workbench example apps by replying with a GitHub comment using one of the following commands: Test all apps:
Test all apps in a directory:
Test an individual app:
Show more apps
Results will be posted here when complete. |
Collaborator
Author
|
Closing — wrong approach: this papered over the harness behavior with prompt rules and host-side cleanup. The real fix is making the pi harness's tool-use semantics match what Anthropic ships in Claude Code (new-file writes need no prior read; file deletion works). Reworking. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
End-of-run agent remarks from the pi rollout (tracking: #793) are dominated by two frictions — together they account for ~35 of the last ~49 remarks on flag-driven basic-integration runs:
.posthog-events.jsoncleanup (~25 remarks): the skill tells the agent to remove its event-plan file at the end of the run, butrmis fence-blocked, so every run burns turns on delete-vs-empty deliberation and write-tool workarounds. The pi harness already deleted the file host-side, but only pi, and the agent was still being asked to do it anyway.Changes
ProgramConfig.cleanupArtifacts— programs declare the bookkeeping files their agent run leaves behind. The runner removes them at therunProgramseam on every termination path, for every harness (same dualregisterCleanup/finallycoverage asflushScanReport). Product knowledge stays in the program config; the runner just executes it.posthog-integrationdeclaresEVENT_PLAN_FILE; the pi-harness-only inline cleanup is removed.Test plan
pnpm build && pnpm test && pnpm fix— 96 files / 1,277 tests pass (snapshot updated), lint clean.Also verified against a 1-file example app (
/tmp/wz-demo/example-appwith justindex.js), driving the real code paths on main vs this branch:Before (main):
After (this branch):
Follow-up (context-mill, not this repo): drop the skill's "delete the plan file" conclusion step so the instruction and the commandment stop conflicting.
LLM context
Authored from an audit of
wizard remarktelemetry on the wizard-switchboard dashboard; the before/after logs above were produced by a throwaway vitest spec drivinggetWizardCommandments()and the realremoveProgramArtifactsseam on both checkouts.Created with PostHog Code