Repository navigation
[Bug]: OpenCode leaks orphaned /opencode serve processes that survive backend crash/restart #5241
Copy link
Copy link
Closed
Labels
bugSomething is broken or behaving incorrectly.Something is broken or behaving incorrectly.needs-triageIssue needs maintainer review and initial categorization.Issue needs maintainer review and initial categorization.
Description
Before submitting
- I searched existing issues and did not find a duplicate.
- I included enough detail to reproduce or investigate the problem.
Area
apps/server
Steps to reproduce
- Start T3 Code client + T3 Remote (i have the remote done via
npx t3@nightly serve --host "$(tailscale ip -4)") - Start a new session with a OpenCode provider and send a message
- Confirm the server is live and find its parent
ps -eo pid,ppid,pgid,args | grep "[o]pencode serve" - Abruptly kill the backend - Simulating a crash / kill -9 / force-quit "C^
- Observe the orphan - same command as step 2 -> the opencode serve is still alive but now with PPID = 1 (reparent ed to init)
- Restart the T3 backend and wait for the session-reaper threshold ->
ps -eo pid,ppid,pgid,etime,rss,args | awk '$2==1 && /[o]pencode serve/'-> the orphaned opencode serve would remain untouched as noting reclaims it only a manual kill or reboot of the machine would clear it.
Expected behavior
A provider server process that T3 spawns should have its lifetime strictly bounded by the T3 backend that owns it, regradless of how that backend terminates. Every one of these should tear the opencode serve down:
- Thread Stop button / Esc -> serve exits
- Graceful backend shutdown (normal quit, SIGTERM that completes teardown) -> all serves exit
- Abrupt backend death (crash, SIGKILL, OOM-kill, force-quit) -> all serves exit
- Backend restart -> no leftover opencode serve from prior runs; clean slate
- Prolonged inactivity -> idle serves reaped (the 30-min reaper)
Actual behavior
- Thread Stop / Esc -> serve exits
- Graceful quit /SIGTERM -> serves exit
- SIGKILL / crash / OOM / Force-quit -> serve survices - Orphaned to PID 1, never exits
- Backend restart -> prior serves still running, invisible to the new backend (still visible in the ui and actionable but not owned by the Backend Process) i think
- Inactivity reaper -> cannot reach orphans - it only closes session owned by a live backend
Damage — resources held by 28 orphaned opencode serve (measured live)
| Resource | Measured | Notes |
|---|---|---|
| Orphaned processes | 28 | all PPID == 1, PGID == <own pid>, ages 34 min – 10 days |
| RAM — PSS (realistic) | ~8.8 GB | unique + shared-fraction; ~93% of OpenCode footprint is waste |
| RAM — Private (USS) | ~8.8 GB | non-shareable, non-reclaimable until killed |
| RAM — RSS (nominal) | ~9.6 GB | double-counts shared libs |
| Swap pinned | ~1.6 GB | system actively swapping due to zombies |
| Worst single orphan | 752 MB RSS / 705 MB USS | fresh repro hit ~476 MB within minutes |
| CPU already consumed | ~6.5 days cumulative | 449 idle threads still get scheduled |
| Open file descriptors | 709 (max 31/proc) | toward per-user ulimit -n |
| Listening TCP ports | 28 | 127.0.0.1:33817–46541, ephemeral range |
opencode.db on disk |
13 GB | shared, cannot be checkpointed (see below) |
opencode.db-wal |
23 MB | cannot checkpoint — 56 zombie handles hold it open |
opencode.log |
128 MB | held open (append) by all 28 |
Top orphans by RSS:
| PID | Age | RSS | Port |
|---|---|---|---|
| 1295328 | 4d 21h | 752 MB | 38007 |
| 2414521 | 3d 22h | 486 MB | 44449 |
| 436765 | 34 min (fresh repro) | 476 MB | 40381 |
| 2839687 | 9d 21h | 415 MB | 41025 |
| 2439553 | 10d 3h | 372 MB | 41351 |
Impact
Major degradation or frequent failure
Version or commit
main @ e60821f or t3: 0.0.32-nightly.20260802.980
Environment
Linux 7.0.0-27-generic x86_64, Node v24.18.0. Opecode 1.18.3
Logs or stack traces
No stack trace or log line exists for this: abrupt backend death precludes logging, and the orphaned opencode serve then runs silently. The evidence is the process table — 28 orphaned serves (PPID=1,PGID==PID, ages up to 10 days) holding ~9 GB RAM, 28 loopback ports, and 56 handles on a shared 13 GB opencode.db whose WAL can no longer checkpoint.Screenshots, recordings, or supporting files
No response
Workaround
A potential option on how to fix this:
- Record the spawn i.e small JSON file or something,
- Reap on startup i.e for each record check if its pid is alive AND its PPID is 1 (i.e its been orphaned) -> kill it. if its pid is dead -> drop the entry. then rewrite the file with whatever is still legitimely alive.
but this depends on how often you "restart" T3 backend (i.e right now with lots of updates so would this be fine but a process which does not depend on the "restart" rate would be good later)
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
bugSomething is broken or behaving incorrectly.Something is broken or behaving incorrectly.needs-triageIssue needs maintainer review and initial categorization.Issue needs maintainer review and initial categorization.
Activity
juliusmarminge commentedon Oct 2, 2026
Thanks for taking the time to report this and provide the details. We revisited it during the orchestrator V2 cleanup.
Current OpenCode startup reaps recorded orphan servers using owner and process identity checks; runtime records new serve children. This addresses accumulation after a crashed T3 process on subsequent startup.
I’m closing this based on the current source and the evidence in this thread.
Source reviewed.
Source audit only. Reclamation occurs on restart, not immediately after SIGKILL; unrecorded legacy processes are outside ledger coverage.
If you still hit this on a current build, please reply with the app/server versions and the steps that reproduce it. We can reopen this if the original problem is still there.