Skip to content

fix(server): scheduled runs that cannot get a worktree are reported as failed - #19774

Open
realhasanshoaib wants to merge 5 commits into
pingdotgg:mainfrom
realhasanshoaib:fix/scheduled-task-root-workspace
Open

realhasanshoaib wants to merge 5 commits into
pingdotgg:mainfrom
realhasanshoaib:fix/scheduled-task-root-workspace

Conversation

@realhasanshoaib

@realhasanshoaib realhasanshoaib commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

Problem

A scheduled task that starts each run in a fresh worktree says succeeded even when that worktree can never be made, for example in a folder that is not a Git repository, or when its setup script fails. The run dies before the agent starts, and the only trace is a failed thread nobody opens. We had seven agent-scheduled tasks in a No-project folder showing "succeeded" while none of them ever ran. Agents scheduling over MCP also cannot ask for the project folder instead of a worktree, which the Settings form already offers.

Score: 5/5.

Change

A scheduled run is now reported as failed whenever its workspace cannot be prepared:

  1. Before launch. For a worktree task, the scheduler asks the existing pre-launch check whether a worktree can be made. That check now also refuses a folder that is plainly not a Git repository. Such a run is recorded as failed with the reason, and no dead thread is created.
  2. After launch. A launch is accepted before its workspace is prepared, so a run is first recorded as succeeded. The scheduler then follows the launched run. If the run's first move out of preparation is to failed (a fetch error or a failing setup script, for example), that run is marked failed.
  scheduled run, unbound task
- threadLaunch.launch(...)                          -> "succeeded", whatever happens to the workspace
+ worktree task: checkWorktreeBase fails            -> "failed: <reason>", no thread
+ launch(...)                                        -> "succeeded" (dispatch accepted, as before)
+   run's first exit from preparation is "failed"   -> "failed: Workspace preparation failed. See thread …"
+   first exit is "starting"                        -> stays "succeeded" (a later provider failure is not a workspace failure)

schedule_task and update_scheduled_task take an optional workspace (root or worktree). An explicit worktree that cannot be made is refused when the task is saved. With no value, tasks keep today's worktree default. The task summary now shows workspace, and an update keeps it unless the binding or the workspace changes.

🟢 Changed (4)

Area One line
Scheduler Checks worktree tasks before launch; marks a run failed when its preparation fails
Thread launch check Also refuses a folder with no .git in it or above it
MCP scheduling tools Optional workspace; summary shows it; updates keep it
t3_thread_launch Says to use root when the folder is not a repository

⚪ Untouched (5)

Area One line
Web, desktop and mobile clients No UI change; their tasks now show the honest failed status
Default workspace for MCP tasks Still a worktree; #18583 changes that default
Stored tasks and database No migration; saved strategies are read as before
Bound tasks Still post into their thread; workspace ignored
Providers, dependencies, settings Unchanged

Undo by reverting the commits. If the repository check were wrong, worktree tasks in that folder would be recorded as failed instead of launching.

Scope and approval

This addresses part of #17309. Julius's triage comment confirms the failure. The status fix is a focused fix of an obvious bug: the scheduler reports success for runs that never started.

The workspace option is focused configuration of an established capability, as the template describes:

  • What exists: every scheduled task already stores a workspace strategy, and Settings → Scheduled tasks already lets a user pick between a new worktree and the project checkout.
  • What the option controls: only which of those two existing strategies an agent-created task saves. Nothing about how either strategy runs changes.
  • Why it stays inside that capability: it reuses the same pre-launch check the launch path already uses, and leaves the default alone.

How it relates to the other open PRs on #17309:

PR What it does Overlap with this PR
#18583 MCP schedule_task picks root for folders that are not Git repositories None in behavior: this PR keeps the worktree default and leaves that choice to #18583. Both edit the same MCP helper, so whichever lands second rebases.
#17451 No-project tasks run at root on save and at launch If it lands first, the pre-launch check must skip No project, or it would fail tasks #17451 makes work.

Verification

Live, against isolated servers on an M4 Mac mini (t3 serve --base-dir <temp>, loopback only, throwaway state). An outside MCP client signed in over OAuth with a pairing code, then scheduled and ran tasks in two projects: a plain folder, and a real Git repository whose setup script is exit 1. The same script ran against main and against this branch.

Step main This PR
Plain folder: run status succeeded failed
Plain folder: threads created 1, status failed 0
Git repo, failing setup: status right after the run succeeded succeeded
Git repo, failing setup: status after preparation succeeded failed
workspace: "worktree" in the plain folder saved (field ignored) refused: Project "Plain folder" (…) is not a Git repository, so it cannot get a worktree. Pass workspace:"root" to run in the project folder.
workspace: "root" saved as worktree (field ignored) saved as root

✅ Checked (7)

Check How Result
Run in a non-Git folder Live before/after above failed, no thread
Setup script fails after launch Live before/after above failed after preparation
Repository detection Real Git on temp folders plain and missing: not a repo; repo and subfolder: repo
Real No-project folder Read-only walk of ~/.t3/scratch no .git above it, so it counts as not a repository
.git walk on Windows paths The same loop run with Node's path.win32 on drive, UNC and drive-root paths ends at C:\, \\server\share\ and D:\
Tests, typecheck, lint, format vp test run on the touched test files plus every test that stubs the launch service, including the scheduler integration test (14 files); server and contracts typecheck; vp lint and vp fmt per file 179 passed, clean
Test value 25 hand-made mutants of the changed and guarded lines, including "a provider failure is blamed on setup" all caught

Test cleanup: in the worktree-check tables, two rows caught only mutants another row already catches. I deleted them ("a base ref that resolves" and "a previous checkout there never was"). Every remaining test in that slice catches a mutant no other test catches.

Technical details
  • Check and repository rule:
    // Git also answers "not a repository" for one it refuses to read, such as a
    // checkout owned by another user. Only a folder with no `.git` in it or
    // above it is plainly not one.
    const hasGitAbove = Effect.fn("ThreadLaunchService.hasGitAbove")(function* (cwd: string) {
    let folder = path.resolve(cwd);
    while (true) {
    if (yield* fileSystem.exists(path.join(folder, ".git"))) return true;
    const parent = path.dirname(folder);
    if (parent === folder) return false;
    folder = parent;
    }
    });
    const checkWorktreeBase: ThreadLaunchService["Service"]["checkWorktreeBase"] = Effect.fn(
    "ThreadLaunchService.checkWorktreeBase",
    )(function* (input) {
    const strategy = input.workspaceStrategy;
    if (strategy.type !== "worktree") return;
    const { baseRef } = strategy;
    // Anything this cannot establish is left for the launch to report: a
    // missing project, a failing Git, a repository Git refuses to read.
    const problem = yield* Effect.gen(function* () {
    const project = Option.getOrUndefined(yield* projects.getById(input.projectId));
    if (project === undefined) return undefined;
    const cwd = project.workspaceRoot;
    if (!(yield* git.isRepository(cwd)))
    return (yield* hasGitAbove(cwd))
    ? undefined
    : new ThreadLaunchNotRepositoryError({
    projectId: project.id,
    projectTitle: project.title,
    workspaceRoot: cwd,
    });
    // `:/text` is a commit search, which cannot be asked for a commit type.
    if (baseRef.startsWith(":")) return undefined;
    // `git worktree add` reads `-` as the previous checkout.
    if (yield* git.hasCommit({ cwd, refName: baseRef === "-" ? "@{-1}" : baseRef }))
    return undefined;
    // Provisioning fetches the base from origin when it can, and runs a
    // repository with no commits yet without a worktree.
    if (
    strategy.startFromOrigin === true &&
    (yield* git.remoteExists({ cwd, remoteName: "origin" }))
    )
    return undefined;
    // `git worktree add` also starts from a remote-tracking branch of that
    // name; any ref carrying it is reason enough to let Git decide.
    if (yield* git.hasRefNamed({ cwd, refName: baseRef })) return undefined;
    if (!(yield* git.hasCommit({ cwd, refName: "HEAD" }))) return undefined;
    return new ThreadLaunchBaseRefError({
    projectId: project.id,
    projectTitle: project.title,
    workspaceRoot: cwd,
    baseRef,
    });
    }).pipe(Effect.orElseSucceed(() => undefined));
    if (problem !== undefined) return yield* problem;
  • Scheduler, check before launch:
    // Effect.exit (not Effect.result) so defects and interruptions in the
    // dispatch are also captured and recorded as a failed run instead of
    // aborting before markCompleted. A launch is accepted before its
    // worktree exists, so a worktree that cannot be made is checked first;
    // otherwise the run would count as succeeded and fail inside its thread.
    const result =
    active.threadId === null
    ? yield* Effect.exit(
    (active.workspaceStrategy.type === "worktree"
    ? threadLaunch.checkWorktreeBase(active)
    : Effect.void
    ).pipe(
    Effect.andThen(
    threadLaunch
    .launch({
    commandId,
    projectId: active.projectId,
    title: active.title,
    modelSelection: active.modelSelection,
    runtimeMode: active.runtimeMode,
    interactionMode: active.interactionMode,
    workspaceStrategy: active.workspaceStrategy,
    initialMessage: {
    messageId,
    scheduledTaskId: active.id,
    text: prompt,
    attachments: [],
    },
    createdBy: active.createdBy,
    creationSource: active.creationSource,
    })
    .pipe(Effect.map((launched) => launched.threadId)),
    ),
    ),
    )
    : yield* Effect.exit(
    threadManagement
    .sendToThread({
    projectId: active.projectId,
    commandId,
    threadId: ThreadId.make(active.threadId),
    messageId,
    scheduledTaskId: active.id,
    text: prompt,
    attachments: [],
    modelSelection: active.modelSelection,
    // Scheduled prompts must not interrupt tools in the bound thread.
    mode: "queue",
    createdBy: active.createdBy,
    creationSource: active.creationSource,
    })
    .pipe(Effect.as(null)),
  • Scheduler, preparation watcher:
    // A launch is accepted before its workspace is prepared, so a run counts as
    // succeeded first. The first time the launched run leaves preparation says
    // how preparation ended; a later retry or provider failure is not about the
    // workspace. A failure is recorded against this run only: last_run_at is
    // the run's identity, so a later run's status is never overwritten.
    const recordFailedPreparation = Effect.fn("ScheduledTaskService.recordFailedPreparation")(
    function* (input: {
    readonly id: ScheduledTaskId;
    readonly startedAtIso: string;
    readonly threadId: ThreadId;
    readonly messageId: MessageId;
    }) {
    // The thread is new, so replaying its run updates from the start is cheap.
    const leftPreparation = yield* threadManagement
    .streamStoredEventsFrom({
    threadId: input.threadId,
    afterSequence: 0,
    eventType: "run.updated",
    })
    .pipe(
    Stream.map((stored) =>
    stored.event.type === "run.updated" &&
    stored.event.payload.userMessageId === input.messageId
    ? stored.event.payload.status
    : "preparing",
    ),
    Stream.filter((status) => status !== "preparing"),
    Stream.runHead,
    );
    if (Option.getOrUndefined(leftPreparation) !== "failed") return;
    yield* sql`
    UPDATE scheduled_tasks
    SET last_run_status = 'failed',
    last_run_error = ${`Workspace preparation failed. See thread ${input.threadId}.`}
    WHERE task_id = ${input.id}
    AND last_run_status = 'succeeded'
    AND last_run_at = ${input.startedAtIso}
    `;
    yield* notifyChanged;
    },
    Effect.catchCause((cause) =>
    Cause.hasInterruptsOnly(cause)
    ? Effect.void
    : Effect.logWarning("Could not record a failed workspace preparation", { cause }),
    ),
    );
  • MCP workspace choice:
    /**
    * Workspace strategy for a scheduled task created/updated over MCP. Bound
    * runs post into the existing thread, so the strategy is unused and stays
    * root. Unbound runs get a fresh worktree unless root is asked for; an
    * explicit worktree is refused up front when one cannot be made.
    */
    const scheduledTaskWorkspaceStrategy = (
    projectId: ScheduledTask["projectId"],
    boundToThread: boolean,
    workspace: "root" | "worktree" | undefined,
    ): Effect.Effect<ScheduledTask["workspaceStrategy"], OrchestratorMcpFailure> => {
    if (boundToThread || workspace === "root") return Effect.succeed({ type: "root" });
    const worktree = { type: "worktree", baseRef: "main", startFromOrigin: true } as const;
    if (workspace === undefined) return Effect.succeed(worktree);
    return threadLaunch.checkWorktreeBase({ projectId, workspaceStrategy: worktree }).pipe(
    Effect.as(worktree),
    Effect.mapError((error) =>
    failure(
    "invalid_request",
    `${error.message} Pass workspace:"root" to run in the project folder.`,
    ),
    ),
    );
    };
File Why
ThreadLaunchService.ts Git's repository check answers "no" for any error, including a repository owned by another user. So a folder only counts as not a repository when no .git exists in it or above it. Anything else stays "a failing Git" and is left to the launch, as before.
ScheduledTaskService.ts The check runs only for worktree tasks, inside the existing Effect.exit, so a refusal lands in the run's failed status. After markCompleted, a fiber in the service scope replays the new thread's run.updated events and takes the run's first status other than preparing. Only failed is recorded, so a later retry or a provider failure after starting does not change the result. The write is guarded by last_run_at and last_run_status = 'succeeded', so it never touches a later run.
OrchestratorMcpService.ts workspace handling. An update recomputes the strategy only when the binding really changes or workspace is passed.

Two limits, by design: a server restart between a preparation failure and its record leaves that run as succeeded (restart recovery cancels runs that are still preparing). And a webhook delivery's log keeps "started", which is what happened to the delivery.

GPT-6 Astra (high) reviewed two rounds and raised four issues, all fixed here. Repository errors could silently pick root. An update repeating bindToCurrentThread:false dropped an explicit root. A retried preparation could hide its first failure. A provider failure could be blamed on setup.

Model: Claude Opus 5.5. Harness: Claude Code in T3 Code. Reviewed by GPT-6 Astra (high).

🤖 Generated with Claude Code

…s failed

A scheduled task that launches a fresh thread per run with a worktree
strategy was recorded as "succeeded" as soon as the thread launch was
accepted, even when its worktree could never be made, such as in a folder
that is not a Git repository. The run then failed inside its thread before
the agent started, and the scheduler never said so.

- The pre-launch worktree check now also refuses a folder with no `.git`
  in it or above it. A repository Git refuses to read is still left to the
  launch.
- The scheduler runs that check before launching, so such a run is recorded
  as failed with the reason, and no dead thread is created.
- schedule_task and update_scheduled_task take an optional workspace
  (root or worktree), so an agent can choose to run in the project folder;
  an explicit worktree that cannot be made is refused up front. The task
  summary now shows the workspace, and an update keeps it unless the
  binding or the workspace changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 15, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-15T07:47:23.494910Z e0fd539 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@github-actions github-actions Bot added vouch:unvouched PR author is not yet trusted in the VOUCHED list. size:L 100-499 changed lines (additions + deletions). labels Oct 15, 2026
@macroscopeapp

macroscopeapp Bot commented Oct 15, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Approved at 6be5346

Macroscope's review found this PR approvable — This is a focused scheduler fix that records impossible worktree launches as failed without creating dead threads. The new MCP workspace selection is optional and preserves existing defaults, with coverage for repository validation and update behavior.

You can add or adjust custom eligibility rules. Learn more.

@coderabbitai

coderabbitai Bot commented Oct 15, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Path: .coderabbit.config.ts
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 07b65a34-1ffa-4b97-b498-1243f783ecb9
📥 Commits

Reviewing files that changed from the base of the PR and between 2911fbd and e0fd539.

📒 Files selected for processing: 1
  • apps/server/src/mcp/toolkits/project/handlers.test.ts
🚧 Files skipped from review as they are similar to previous changes: 1
  • apps/server/src/mcp/toolkits/project/handlers.test.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 6 remain after this review.

📝 Walkthrough

Scheduled tasks now accept root or worktree workspace choices. Worktree requests are checked for repository and base-ref validity. Scheduled runs record worktree-check failures and track preparation failures after launch.

Changes: Scheduled-task workspace selection

Layer / File(s) Summary
Worktree-base validation
apps/server/src/orchestration-v2/ThreadLaunchService.ts, apps/server/src/orchestration-v2/ThreadLaunchService.test.ts, apps/server/src/mcp/toolkits/project/handlers.ts, apps/server/src/mcp/toolkits/project/handlers.test.ts
checkWorktreeBase checks whether a .git directory exists in the project folder or a parent when Git reports that the folder is not a repository. It reports a distinct error when no repository is found. Project launch errors and tests cover these checks.
Workspace contract and MCP selection
packages/contracts/src/orchestratorMcp.ts, apps/server/src/mcp/OrchestratorMcpService.ts, apps/server/src/mcp/OrchestratorMcpService.test.ts, apps/server/src/mcp/toolkits/orchestrator/tools.ts
Scheduled-task inputs accept root or worktree, and task results expose the workspace strategy. MCP creation and updates resolve the strategy, validate worktree requests, and return it in task summaries. Tool descriptions document the choices.
Scheduled-run validation and preparation status
apps/server/src/scheduledTasks/ScheduledTaskService.ts, apps/server/src/scheduledTasks/ScheduledTaskService.test.ts
New-thread runs check the worktree base before launch. After a successful launch, the service monitors preparation events and can mark the matching run as failed.

Priority: ➖ Normal

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Bug fix

Sequence Diagram(s)

sequenceDiagram
  participant ScheduledTaskService
  participant ThreadLaunchService
  participant RunUpdateStream
  participant ScheduledTaskStore
  ScheduledTaskService->>ThreadLaunchService: Check worktree base and launch thread
  ThreadLaunchService-->>ScheduledTaskService: Return launched thread ID
  ScheduledTaskService->>RunUpdateStream: Read preparation updates
  RunUpdateStream-->>ScheduledTaskService: Report initial message status
  ScheduledTaskService->>ScheduledTaskStore: Mark matching run failed when preparation failed
Loading
Merge Risk: ⚪ Minimal · up to e0fd5

No merge-blocking issue is established for the selected change; normal checks remain appropriate.

Security architecture risk: 🔵 Low · up to 2911f

Scheduled tasks can now explicitly run in the shared project folder rather than a fresh worktree. Existing caller permissions and execution-mode limits remain in place. Failure reporting improves, but late status corrections do not survive every restart or monitoring failure.

Retained concerns
No architecture-level concerns identified.

Security review details

Security Blast Radius

  • observed — An authorized caller can now persist an unbound scheduled task that uses an ordinary project's shared checkout instead of a fresh worktree. No-project execution still uses managed folders. The task retains its established runtime and interaction modes; workspace selection does not itself grant broader execution modes.

Trust Boundaries and Controls

  • observed — The caller-controlled workspace value reaches persistent scheduling through existing MCP write/start controls. Scheduling requires the orchestration capability, resolves the target project, stores caller-limited modes, and checks binding ownership. Updates validate the existing task's modes and preserve them. The traced change adds workspace choice without removing these controls.
Pre-merge checks: Passed 4
Passed Passed checks: 4 passed
Check name Status Explanation
Title check Passed The title clearly and concisely describes the primary change: scheduled runs now report failure when worktree preparation fails.
Description check Passed The description is complete and follows the required Problem, Change, Scope and approval, and Verification sections. It explains the behavior, scope, related work, testing, observed results, and known…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

realhasanshoaib and others added 4 commits October 15, 2026 09:10
Root runs have nothing to check, and skipping the call keeps scheduler
fixtures that stub only `launch` working.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…orted as failed

A launch is accepted before its workspace is prepared, so a run whose
worktree fetch or setup script failed still read "succeeded". The scheduler
now waits for the launched run to leave preparation and, if it failed,
marks that run failed. The write is guarded by the run's start time, so a
later run's status is never overwritten. It also covers the hint that
t3_thread_launch gives for a folder that is not a Git repository.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The watcher read the run's current status, so a retried preparation could
hide its first failure, and a provider failure after setup could be blamed
on setup. It now replays the new thread's run updates and decides from the
run's first status other than "preparing".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@unmanbearpig

Copy link
Copy Markdown

If an agent updates a worktree schedule and passes workspace: "worktree" again, the base branch the user picked in the editor (say release) is silently replaced with main.

updateScheduledTask keeps the saved strategy only when workspace is omitted. Any value goes through scheduledTaskWorkspaceStrategy, which builds {type:"worktree", baseRef:"main", startFromOrigin:true} from scratch. A task whose base was changed in the web or mobile editor loses that change:

Saved strategy Update input Result
worktree, baseRef:"release", startFromOrigin:false {prompt} unchanged
same {prompt, workspace:"worktree"} baseRef:"main", startFromOrigin:true
same, repo has no main and no origin same update rejected, and the error suggests workspace:"root"
worktree, main, branch:"nightly" {enabled:false, workspace:"worktree"} branch dropped

The agent can't see this happen. list_scheduled_tasks reports workspace: "worktree" before and after, and MCP has no field to set the base back. An agent copying fields from the list output into an update will hit it, since the field name and value match. The people most exposed are on master or trunk repos, because they had to change the editor's main default. When origin exists but has no main, the precheck passes and the update saves a base that fails on every run.

The tool description promises "Only the provided fields change", so I'd treat a repeated type as no change:

const workspaceStrategy =
  !rebound &&
  (input.workspace === undefined || input.workspace === existing.workspaceStrategy.type)
    ? existing.workspaceStrategy
    : yield* scheduledTaskWorkspaceStrategy(existing.projectId, threadId !== null, input.workspace);

I checked this with a scratch test at e0fd539. The three bad cases keep their saved strategy. The binding and type transitions (root to worktree, worktree to root, existing_worktree to worktree, bound to unbound and back) behave as before. The existing test "an update keeps the workspace unless the binding or the workspace changes" only asserts summary.workspace. Asserting the saved strategy after a repeated workspace:"worktree" on a custom base would catch this.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:L 100-499 changed lines (additions + deletions). vouch:unvouched PR author is not yet trusted in the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants