Skip to content

fix: fail Pi setup runs when required resources are unavailable - #1433

Merged
marandaneto merged 1 commit into
mainfrom
fix/agent-setup-verification
Oct 11, 2026
Merged

marandaneto merged 1 commit into
mainfrom
fix/agent-setup-verification

Conversation

@marandaneto

Copy link
Copy Markdown
Member

Problem

The Pi harness collects [ERROR-MCP-MISSING] and [ERROR-RESOURCE-MISSING] from assistant output but can still return success after the agent stops because setup instructions are unavailable. The legacy harness already translates these markers into typed failures.

Both the linear run and task entry point lose the specific failure. This can also trigger unnecessary continuation or remark prompts.

Changes

  • Classify the two existing fatal setup markers in the completion module.
  • Stop continuation and remark prompts after either marker.
  • Return the existing MCP/resource failure and record aborted rather than completed telemetry.
  • Preserve cancellation, provider failure, and security precedence. Tool-result text and nonfatal skill-install fallback do not become setup failures.

No installer recipes, credentials, provider policy, dependencies, or general verification protocol changed. Wizard uses release-please, so this fix has no changeset file.

Test plan

  • Replayed the new tests against the original production files: six expected failures, including four false-success cases across both entry points.
  • Restored the fix and reran structured-run and completion tests: 30 passed.
  • Seven relevant test files passed 153 tests during implementation.
  • Typecheck, scoped formatting, ESLint, and the production bundle passed. ESLint retains three existing warnings.
  • Build lifecycle scripts were disabled to avoid smoke checks that could send live analytics. Live provider, gateway, MCP, ingestion, and full e2e checks were not run.
  • Reviewed the changed tests for distinct contracts: failure classification, no extra prompts or success telemetry, security precedence, tool-output boundaries, and nonfatal fallback.
  • autoreview --mode branch --base origin/main passed for c76442a53b7632b16e23ca9e6e92e525e021d668 with no actionable findings.

LLM context

Human-driven, agent-assisted. Pi and implementation subagents used Git, Vitest, local type/build checks, and isolated Pi autoreview. The patch only consumes existing fatal setup signals. The session is local and has not been published.

@marandaneto marandaneto self-assigned this Oct 9, 2026
@github-actions

github-actions Bot commented Oct 9, 2026

Copy link
Copy Markdown

🧙 Wizard CI

Run the Wizard CI and test your changes against wizard-workbench example apps by replying with a GitHub comment using one of the following commands:

Test all apps:

  • /wizard-ci all

Test all apps in a directory:

  • /wizard-ci ai-observability
  • /wizard-ci basic-integration
  • /wizard-ci feature-flags
  • /wizard-ci mcp-analytics
  • /wizard-ci replay-vision
  • /wizard-ci revenue
  • /wizard-ci self-driving
  • /wizard-ci warehouse
  • /wizard-ci warehouse-seeded

Test an individual app:

  • /wizard-ci ai-observability/anthropic
  • /wizard-ci ai-observability/google-adk
  • /wizard-ci ai-observability/groq
Show more apps
  • /wizard-ci ai-observability/manual-capture
  • /wizard-ci ai-observability/openai
  • /wizard-ci ai-observability/openai-agents
  • /wizard-ci ai-observability/opentelemetry
  • /wizard-ci ai-observability/vercel-ai
  • /wizard-ci basic-integration/android
  • /wizard-ci basic-integration/angular
  • /wizard-ci basic-integration/astro
  • /wizard-ci basic-integration/django
  • /wizard-ci basic-integration/fastapi
  • /wizard-ci basic-integration/flask
  • /wizard-ci basic-integration/flutter
  • /wizard-ci basic-integration/javascript-node
  • /wizard-ci basic-integration/javascript-web
  • /wizard-ci basic-integration/laravel
  • /wizard-ci basic-integration/next-js
  • /wizard-ci basic-integration/nuxt
  • /wizard-ci basic-integration/python
  • /wizard-ci basic-integration/rails
  • /wizard-ci basic-integration/react-native
  • /wizard-ci basic-integration/react-router
  • /wizard-ci basic-integration/sveltekit
  • /wizard-ci basic-integration/swift
  • /wizard-ci basic-integration/tanstack-router
  • /wizard-ci basic-integration/tanstack-start
  • /wizard-ci basic-integration/vue
  • /wizard-ci feature-flags/django
  • /wizard-ci feature-flags/next-js
  • /wizard-ci mcp-analytics/custom-dispatcher
  • /wizard-ci mcp-analytics/typescript-sdk
  • /wizard-ci replay-vision/javascript-node
  • /wizard-ci replay-vision/next-js
  • /wizard-ci replay-vision/react-vite
  • /wizard-ci revenue/stripe
  • /wizard-ci self-driving/astro
  • /wizard-ci self-driving/fastapi
  • /wizard-ci self-driving/nuxt
  • /wizard-ci self-driving/react-router
  • /wizard-ci self-driving/sveltekit
  • /wizard-ci warehouse/monorepo-env
  • /wizard-ci warehouse/multi-source-next
  • /wizard-ci warehouse/stripe-node
  • /wizard-ci warehouse/zero-source
  • /wizard-ci warehouse-seeded/next-stripe
  • /wizard-ci warehouse-seeded/next-stripe-declined

Test against a Context Mill branch:

  • /wizard-ci all context-mill:my-branch

Add context-mill:<branch> to any command above to pin the Context Mill branch. It defaults to main.

Results will be posted here when complete.

@marandaneto
marandaneto marked this pull request as ready for review October 9, 2026 17:22
@marandaneto
marandaneto requested a review from a team as a code owner October 9, 2026 17:22
@marandaneto
marandaneto requested a review from a team October 9, 2026 17:22
@marandaneto
marandaneto merged commit ce4fa0b into main Oct 11, 2026
21 checks passed
@marandaneto
marandaneto deleted the fix/agent-setup-verification branch October 11, 2026 11:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants