Repository navigation
feat(host-service): generic worker_threads pool — git status + commit-files off the event loop - #5750
Conversation
…files off the event loop
|
Caution Review failedThe pull request is closed. ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (2)
📝 WalkthroughWalkthroughAdds a worker-thread runtime for host-service Git operations, including typed task contracts, pooling, retries, fallbacks, crash handling, packaging, and integration tests. Git status and commit-file router procedures now dispatch work through the host worker pool. ChangesHost worker Git execution
Estimated code review effort: 4 (Complex) | ~60 minutes Sequence Diagram(s)sequenceDiagram
participant GitRouter
participant HostWorkerPool
participant WorkerTaskRunner
participant HostWorker
participant SimpleGit
GitRouter->>HostWorkerPool: Submit Git task and gitEnv
HostWorkerPool->>WorkerTaskRunner: Queue task with timeout and dedupe options
WorkerTaskRunner->>HostWorker: Send task payload
HostWorker->>SimpleGit: Execute Git operation
SimpleGit-->>HostWorker: Return Git result
HostWorker-->>WorkerTaskRunner: Send success or serialized error
WorkerTaskRunner-->>HostWorkerPool: Resolve or reject task
HostWorkerPool-->>GitRouter: Return Git result or inline fallback result
Possibly related PRs
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Ready to review this PR? Stage has broken it down into 8 individual chapters for you: Chapters generated by Stage for commit c6ecda6 on Jul 19, 2026 5:36am UTC. |
🚀 Preview Deployment🔗 Preview Links
Preview updates automatically with new commits |
There was a problem hiding this comment.
Actionable comments posted: 3
🧹 Nitpick comments (2)
packages/host-service/src/workers/no-native-worker-imports.test.ts (1)
11-47: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winMove this cross-cutting invariant test to the integration-test directory.
This scans the entire worker subtree rather than testing one co-located source file. Move it under
packages/host-service/test/integration/and updateWORKERS_DIRto resolvepackages/host-service/src/workers.Based on learnings, cross-cutting host-service invariant scans belong under
packages/host-service/test/integration/.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/host-service/src/workers/no-native-worker-imports.test.ts` around lines 11 - 47, Move the worker graph purity test containing `FORBIDDEN`, `walk`, and the `worker graph purity` suite into `packages/host-service/test/integration/`. Update `WORKERS_DIR` to resolve the host-service `src/workers` directory from its new location, while preserving the existing scan patterns and assertions.Source: Learnings
packages/host-service/src/workers/host-worker-pool.test.ts (1)
79-90: 🩺 Stability & Availability | 🔵 Trivial | ⚡ Quick winAvoid a real 600ms wait in this test.
Use a fake clock or event-loop-based synchronization with a zero/minimal configured idle timeout. The current wall-clock sleep slows the suite and remains scheduling-sensitive.
Based on learnings, Superset test code should avoid real wall-clock timers.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/host-service/src/workers/host-worker-pool.test.ts` around lines 79 - 90, Update the “idle workers are reaped after idleTimeoutMs” test to avoid the real 600ms sleep and scheduling-sensitive assertion. Configure a zero or minimal idle timeout in makePool, then use fake-clock advancement or event-loop synchronization to trigger reaping deterministically while preserving the assertions that one worker is created and eventually reduced to zero.Source: Learnings
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@packages/host-service/src/workers/host-worker-pool.ts`:
- Around line 141-148: Update the circuit-opening logic in the host worker pool
to set inlineOnly and route future calls inline without immediately disposing
this.runner. Defer runner retirement and disposal until all outstanding calls on
the existing runner have settled, so queued and active tasks can complete
normally rather than receiving WorkerTaskAbortedError.
- Around line 110-123: Update the inline fallback paths in the host worker pool,
including the no-runner branch and the WorkerTaskError crash branch, to use a
shared inline executor instead of calling def.handler directly. Have that
executor honor WorkerTaskOptions timeoutMs and abort signal semantics so
already-aborted requests do not start and inline execution times out
consistently with runner execution.
In `@packages/host-service/src/workers/WorkerTaskRunner.ts`:
- Around line 289-295: Update the task dispatch flow in WorkerTaskRunner around
the WorkerTaskRequestMessage and slot.worker.postMessage call to catch
synchronous postMessage failures. On failure, reject and clean up the active
task, release the worker slot, and continue draining the queue without allowing
the exception to escape the event handler.
---
Nitpick comments:
In `@packages/host-service/src/workers/host-worker-pool.test.ts`:
- Around line 79-90: Update the “idle workers are reaped after idleTimeoutMs”
test to avoid the real 600ms sleep and scheduling-sensitive assertion. Configure
a zero or minimal idle timeout in makePool, then use fake-clock advancement or
event-loop synchronization to trigger reaping deterministically while preserving
the assertions that one worker is created and eventually reduced to zero.
In `@packages/host-service/src/workers/no-native-worker-imports.test.ts`:
- Around line 11-47: Move the worker graph purity test containing `FORBIDDEN`,
`walk`, and the `worker graph purity` suite into
`packages/host-service/test/integration/`. Update `WORKERS_DIR` to resolve the
host-service `src/workers` directory from its new location, while preserving the
existing scan patterns and assertions.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 4bf7fcc7-293d-4d3f-b6e2-9d0014785dde
📒 Files selected for processing (17)
apps/desktop/electron.vite.config.tsapps/desktop/src/main/host-worker/index.tspackages/cli/scripts/build-dist.tspackages/host-service/build.tspackages/host-service/package.jsonpackages/host-service/src/runtime/git/git.tspackages/host-service/src/runtime/git/index.tspackages/host-service/src/trpc/router/git/git.tspackages/host-service/src/workers/WorkerTaskRunner.tspackages/host-service/src/workers/define-worker-task.tspackages/host-service/src/workers/host-worker-pool.test.tspackages/host-service/src/workers/host-worker-pool.tspackages/host-service/src/workers/host-worker.tspackages/host-service/src/workers/no-native-worker-imports.test.tspackages/host-service/src/workers/tasks/git.tspackages/host-service/src/workers/test-fixtures/crashing-worker.tspackages/host-service/src/workers/worker-task-protocol.ts
Greptile SummaryThis PR introduces a generic
Confidence Score: 4/5Safe to merge with awareness of the crash-accounting issue; callers always get correct results via inline fallback, but the pool can be permanently switched to inline-only faster than intended when concurrent coalesced requests meet a crashing worker. The crash timestamp counter in recordCrash() is incremented once per pool.run() caller, not once per actual worker crash. When the coalesce strategy is active, every concurrent coalesced caller independently catches the WorkerCrashedError and calls recordCrash(), potentially opening the circuit from a single actual crash and permanently disabling the worker offload for the process lifetime. packages/host-service/src/workers/host-worker-pool.ts — specifically the run() / recordCrash() interaction under coalesce concurrency
|
| Filename | Overview |
|---|---|
| packages/host-service/src/workers/host-worker-pool.ts | Core pool class with crash-circuit logic; the coalesce+crash timestamp inflation bug means a single worker crash can permanently disable the pool under concurrent coalesced requests |
| packages/host-service/src/workers/WorkerTaskRunner.ts | Solid port of v1 runner with idle reaping; coalesce/latest-wins/abort handling all guarded correctly; exit-code-0 is treated as crash (minor) |
| packages/host-service/src/workers/tasks/git.ts | Clean task definitions; handlers build their own SimpleGit with the env received from the main thread; no native/singleton imports |
| packages/host-service/src/workers/host-worker.ts | Minimal dispatch loop over a build-time registry; duplicate-type guard on startup; error serialization on uncaught handler errors |
| packages/host-service/src/trpc/router/git/git.ts | Routes getStatus and getCommitFiles through the pool; credentials resolved in-process and passed as plain env; dedupeKey construction is correct |
| packages/host-service/src/runtime/git/git.ts | Clean extraction of createGitEnvResolver from createGitFactory; semantically equivalent to original, enables env-as-plain-data crossing to worker thread |
| packages/host-service/src/workers/host-worker-pool.test.ts | Good coverage: parity test on real fixture repo, idle reap, missing-bundle fallback, crash to inline retry to circuit open; coalesce+crash path not tested |
| packages/host-service/src/workers/worker-task-protocol.ts | Well-typed wire protocol; discriminated union for ok/error responses; serializeWorkerError handles non-Error throws correctly |
| packages/host-service/src/workers/define-worker-task.ts | Simple identity wrapper for worker task definitions; provides end-to-end generic typing between caller and handler |
| packages/host-service/build.ts | Adds second Bun.build entry for host-worker.ts; exits on failure; mirrors existing host-service.js build pattern |
| packages/cli/scripts/build-dist.ts | Adds cpSync of host-worker.js alongside host-service.js; consistent with existing pty-daemon side-by-side copy pattern |
| apps/desktop/electron.vite.config.ts | Adds host-worker as a second electron-vite main-process entry alongside pty-daemon; mirrors that pattern exactly |
| packages/host-service/src/workers/no-native-worker-imports.test.ts | Grep-based guard for direct native/singleton imports in src/workers/**; correctly relies on bundle build for transitive regressions |
Sequence Diagram
%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
participant tRPC as tRPC Router
participant Pool as HostWorkerPool
participant Runner as WorkerTaskRunner
participant Worker as Worker Thread
participant Git as git subprocess
tRPC->>tRPC: createGitEnvResolver resolves credentials in-process
tRPC->>Pool: pool.run(gitStatusSnapshotTask, input)
Pool->>Runner: runner.runTask(taskType, payload)
Note over Runner: coalesce returns inFlight promise if same dedupeKey
Runner->>Worker: postMessage task request
Worker->>Git: SimpleGit with env from payload
Git-->>Worker: parsed result
Worker-->>Runner: postMessage result
Runner-->>Pool: resolve
Pool-->>tRPC: GitStatusSnapshot
alt Worker crash
Worker-->>Runner: exit event
Runner-->>Pool: reject WorkerCrashedError
Pool->>Pool: recordCrash
Pool->>Pool: def.handler inline retry
Pool-->>tRPC: result via inline fallback
end
alt Crash budget exceeded
Pool->>Pool: "inlineOnly = true permanently"
Note over Pool: all future tasks run inline
end
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
participant tRPC as tRPC Router
participant Pool as HostWorkerPool
participant Runner as WorkerTaskRunner
participant Worker as Worker Thread
participant Git as git subprocess
tRPC->>tRPC: createGitEnvResolver resolves credentials in-process
tRPC->>Pool: pool.run(gitStatusSnapshotTask, input)
Pool->>Runner: runner.runTask(taskType, payload)
Note over Runner: coalesce returns inFlight promise if same dedupeKey
Runner->>Worker: postMessage task request
Worker->>Git: SimpleGit with env from payload
Git-->>Worker: parsed result
Worker-->>Runner: postMessage result
Runner-->>Pool: resolve
Pool-->>tRPC: GitStatusSnapshot
alt Worker crash
Worker-->>Runner: exit event
Runner-->>Pool: reject WorkerCrashedError
Pool->>Pool: recordCrash
Pool->>Pool: def.handler inline retry
Pool-->>tRPC: result via inline fallback
end
alt Crash budget exceeded
Pool->>Pool: "inlineOnly = true permanently"
Note over Pool: all future tasks run inline
end
Prompt To Fix All With AI
Fix the following 2 code review issues. Work through them one at a time, proposing concise fixes.
---
### Issue 1 of 2
packages/host-service/src/workers/host-worker-pool.ts:105-127
**Coalesced callers each record a crash on one worker failure**
When `strategy === "coalesce"` is used (e.g. `gitCommitFilesTask`), `runner.runTask()` returns the same shared `taskPromise` to every waiting caller. When the worker crashes, that promise rejects with `WorkerCrashedError` and _every_ caller's independent `pool.run()` catch-block fires, each calling `recordCrash()`. A single worker crash can therefore push N timestamps into `crashTimestamps` — one per concurrent coalesced caller. With `CRASH_BUDGET = 3`, three simultaneous `getCommitFiles` tRPC requests for the same `workspaceId:commitHash` pair hitting a single crashing worker opens the circuit immediately and permanently disables the worker pool for the rest of the process lifetime.
### Issue 2 of 2
packages/host-service/src/workers/WorkerTaskRunner.ts:246-258
**Graceful exit (code 0) while running a task is treated as a crash**
The `exit` handler always calls `handleWorkerFailure` with `WORKER_CRASH_ERROR_NAME`, even for exit code 0. If a future task handler or library somehow causes a clean `process.exit(0)` mid-task, the active task gets a `WorkerCrashedError`, `HostWorkerPool.recordCrash()` fires, and the crash budget is consumed. The `error` event guard only fires for uncaught exceptions, not for intentional exits. Checking `if (code !== 0 || slot.activeTaskId)` before treating the exit as a crash would narrow the crash-accounting to genuinely unexpected exits.
Reviews (1): Last reviewed commit: "feat(host-service): generic worker_threa..." | Re-trigger Greptile
There was a problem hiding this comment.
All reported issues were addressed across 17 files
You’re at about 93% of the monthly reviewed-line limit. You may want to disable incremental reviews to conserve quota. Reviews will continue until that limit is exceeded. If you need help avoiding interruptions, please contact contact@cubic.dev.
Architecture diagram
sequenceDiagram
participant Client as tRPC Client
participant TRPC as tRPC Router
participant Pool as HostWorkerPool
participant Runner as WorkerTaskRunner
participant Worker as worker_threads
participant Git as SimpleGit (off-thread)
participant Inline as Inline Fallback Handler
participant Cred as Credential Provider
Note over Client,Cred: NEW: git.getStatusSnapshot & git.getCommitFiles via worker pool
Client->>TRPC: git.getStatusSnapshot(input)
TRPC->>Cred: resolveGitTaskEnv(worktreePath)
Cred-->>TRPC: credential env data
TRPC->>Pool: run(gitStatusSnapshotTask, { worktreePath, gitEnv })
alt Bundle found & circuit open
Pool->>Runner: runTask("git/getStatusSnapshot", payload)
Runner->>Runner: ensureWorkerCapacity()
Runner->>Worker: postMessage({ kind: "task", taskType, payload })
Worker->>Worker: look up handler in registry
Worker->>Git: createUserSimpleGit().env(gitEnv)
Git->>Git: git status / diff (off event loop)
Git-->>Worker: result
Worker-->>Runner: postMessage({ kind: "result", ok: true, result })
Runner-->>Pool: resolved
Pool-->>TRPC: status snapshot
else Worker crashes
Worker->>Worker: exit/error
Runner-->>Pool: WorkerCrashedError
Pool->>Pool: recordCrash()
Pool->>Inline: def.handler(input) [retry inline once]
alt Crash budget exhausted
Pool->>Pool: inlineOnly = true, dispose runner
end
Inline-->>Pool: result
Pool-->>TRPC: status snapshot
else Bundle not found
Pool->>Pool: inlineOnly = true
Pool->>Inline: def.handler(input)
Inline-->>Pool: result
Pool-->>TRPC: status snapshot
end
TRPC-->>Client: status snapshot
Note over Client,CRED: NEW: Idle worker reaping (30s)
Runner->>Runner: scheduleIdleReaping()
alt Worker idle for 30s
Runner->>Runner: clear idle timer
Runner->>Worker: terminate()
end
Note over Client,CRED: NEW: Coalesce deduplication for getCommitFiles
Client->>TRPC: git.getCommitFiles(input)
TRPC->>Pool: run(gitCommitFilesTask, ..., { dedupeKey, strategy: "coalesce" })
alt In-flight request with same dedupeKey
Pool->>Pool: return existing promise
Pool-->>TRPC: coalesced result
else No in-flight request
Pool->>Worker: execute normally
Worker-->>Pool: result
Pool-->>TRPC: result
end
TRPC-->>Client: commit files
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
CDP stress re-run: 8-workspace × 20k-file churn, before vs after this PRSame harness as the pre-pool measurement (200 tracked-file appends/500ms per worktree × 8, background
Takeaways:
Idle baseline on this branch for reference: loop probe p95 1.9ms, query p50 0.7ms, echo p50 2.0ms. |
…ut/abort, crash dedupe, no-abort circuit, postMessage cleanup
Review findings addressed in 4ccc47d
All 9 pool tests pass (3 new regression tests, each verified to fail against the pre-fix code); typecheck + lint clean. |
There was a problem hiding this comment.
All reported issues were addressed across 3 files (changes from recent commits).
You’re at about 94% of the monthly reviewed-line limit. You may want to disable incremental reviews to conserve quota. Reviews will continue until that limit is exceeded. If you need help avoiding interruptions, please contact contact@cubic.dev.
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
…ed backlog at circuit-open, inline timeout parity, commit-files cap
There was a problem hiding this comment.
All reported issues were addressed across 5 files (changes from recent commits).
You’re at about 99% of the monthly reviewed-line limit. You may want to disable incremental reviews to conserve quota. Reviews will continue until that limit is exceeded. If you need help avoiding interruptions, please contact contact@cubic.dev.
Tip: Review your code locally with the cubic CLI to iterate faster.
Re-trigger cubic
…ash-retry, dedupe commit-files ahead of the cap
Links
plans/20260717-host-service-git-worker-pool.md(perf(host-service): bound large git watcher batches #5746)Summary
worker_threadspool inside host-service (packages/host-service/src/workers/) and movesgit.getStatus+git.getCommitFilescompute onto it. The worker spawns the git subprocesses itself, so stdout draining AND parsing leave the event loop that also relays terminal I/O and serves all tRPC/WS for the org.git-task-worker); v2 lost it. Stress tests showed backgroundgetStatusqueueing 3.7–14.6s under 8-workspace churn — this PR is the infrastructure that makes raising the limiter's concurrency safe (deferred, see Follow-ups).How It Works
WorkerTaskRunnerported from v1 desktop (v1 keeps its copy; it retires with v1) with two changes: idle reaping (workers terminated after 30s idle — host-service is per-org and long-lived, a 3-org machine must not hold 12 warm workers) and demand-sized spawning (v1 spawned the full pool on any task).defineWorkerTask<In, Out>defs, namespacedgit/*;host-worker.tsimports them into a build-time registry. Callers pass the same def toHostWorkerPool.run()for end-to-end typing. Adding a domain = one task module + one import.createGitEnvResolversplit out ofcreateGitFactory— the env is resolved in-process (needs the credential provider) and crosses to the worker as plain data; the worker builds its own SimpleGit.apps/desktop/src/main/host-worker/index.ts, secondBun.buildinpackages/host-service/build.ts, copy inpackages/cli/scripts/build-dist.ts. Resolution:SUPERSET_HOST_WORKER_SCRIPT_PATH→ side-by-side → workspace dist → inline fallback.Manual QA Checklist
git.getStatuson the superset repo through the real app returns a correct snapshot in worker mode (435ms; zero[host-worker-pool]fallback warnings in the host-service log;dist/main/host-worker.jsemitted)bun run build:hostemitsdist/host-worker.js; plain-node smoke (new Worker('./dist/host-worker.js')) round-trips the task protocol (ESM entry works under node)Testing
src/workers/host-worker-pool.test.ts(6 tests): worker↔inline parity on a real fixture repo (modified/staged/untracked buckets), unknown-task error propagation, idle reaping, missing-bundle inline fallback, crash → inline retry → circuit open. Mutation-verified: disabling reaping fails the reap test; removing the inline retry fails the crash test.src/workers/no-native-worker-imports.test.ts: worker graph stays free of native addons and process singletons (db/events/daemon); transitive regressions fail the host-worker bundle build.bun run typecheck(host-service + desktop),bun run lintclean.Design Decisions
min(4, cpus−1): the limiter stays the only queue for status refreshes; pooled tasks never meaningfully queue behind it.Known Limitations
scheduleBaseRefFetch's dedupe state is per-thread now — N workers can, worst case, schedule up to N base-ref fetches for the same worktree within one freshness window. Bounded and rare; noting rather than complicating the payload.Follow-ups
gitStatusRefreshLimiterconcurrency now that snapshots don't cost loop time (measure first)getDiff/listCommits/getBranchSyncStatustenantsSummary by cubic
Introduces a generic
worker_threadspool forpackages/host-serviceand movesgit.getStatusandgit.getCommitFilesoff the event loop to cut stalls and improve responsiveness. Addresses SUPER-1544 by restoring v1’s offload pattern, adding commit-file task deduping ahead of a small cap, and hardening the inline fallback (sync-throw safe, coalesced inline crash-retry).New Features
WorkerTaskRunnerpool with demand-based spawning and 30s idle reaping; concurrency defaults tomin(4, cpus−1)+2.postMessagefailures so bad payloads don’t wedge a slot.git/*tasks;createGitEnvResolverresolves credential env in-process and passes plain data to the worker.git.getStatusSnapshotandgit.getCommitFilesto the pool with 15s timeouts; commit-file tasks capped at 2 and deduped by key outside the cap so identical requests share one slot and one run.Dependencies
host-worker.jsside-by-side withhost-service.js:apps/desktopaddssrc/main/host-worker/index.tsto electron-vite.packages/host-service/build.tsemitshost-worker.js(ESM);packages/cli/scripts/build-dist.tscopies it next tohost-service.js.exportsentry./host-workerinpackages/host-service/package.jsonand resolution orderSUPERSET_HOST_WORKER_SCRIPT_PATH→ side-by-side → workspacedist→ inline fallback.Written for commit c6ecda6. Summary will update on new commits.
Summary by CodeRabbit