Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
b31602c
docs(opencode): plan export integrity fix
Juliusolsson05 Sep 9, 2026
984fd5d
fix(opencode): capture complete CLI exports before switching
Juliusolsson05 Sep 9, 2026
c2c18a6
test(opencode): project the affected native export across providers
Juliusolsson05 Sep 9, 2026
36bf958
merge: bring OpenCode export integrity up to date with main
Juliusolsson05 Sep 12, 2026
b18032b
fix(opencode): settle CLI commands whose spawn fails before stdio exists
Juliusolsson05 Sep 12, 2026
7ad30b7
fix(opencode): terminate the whole CLI process tree on timeout, stop …
Juliusolsson05 Sep 12, 2026
c1be521
fix(opencode): unlink the CLI stdout capture before the command starts
Juliusolsson05 Sep 12, 2026
8f6c0ef
fix(opencode): give transcript transforms a large-conversation CLI de…
Juliusolsson05 Sep 12, 2026
c3aa47a
test(opencode): pin spawn failure to ENOENT and cover the late stop r…
Juliusolsson05 Sep 12, 2026
de80797
docs(opencode): align CLI helper WHY comments with the unlinked capture
Juliusolsson05 Sep 12, 2026
8c94da7
docs(opencode): record the main merge and review round in the export …
Juliusolsson05 Sep 12, 2026
5d15002
test(opencode): make process-tree fixtures ready before termination b…
Juliusolsson05 Sep 12, 2026
623a52b
fix(opencode): never signal a CLI process group after its leader is r…
Juliusolsson05 Sep 12, 2026
6359107
docs(opencode): state that the CLI deadline ends with the main process
Juliusolsson05 Sep 13, 2026
a380c2a
docs(opencode): record review round 2 in the export plan
Juliusolsson05 Sep 13, 2026
a89e5b4
test(opencode): gate main's hung-import tests on child readiness
Juliusolsson05 Sep 13, 2026
3096ed2
docs(opencode): record the inherited readiness race in the export plan
Juliusolsson05 Sep 13, 2026
c319e90
test(opencode): share CLI test polling helpers and reset spawn betwee…
Juliusolsson05 Sep 13, 2026
fef39a4
test(opencode): never deliver a group signal from the reaped-leader r…
Juliusolsson05 Sep 13, 2026
d0562d9
test(opencode): reclaim escaped fixture descendants when readiness fails
Juliusolsson05 Sep 13, 2026
ab5606a
docs(opencode): record review round 3 in the export plan
Juliusolsson05 Sep 13, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
160 changes: 160 additions & 0 deletions docs/superpowers/plans/2026-09-09-opencode-export-integrity.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,160 @@
# OpenCode Export Integrity

Refs #845. Independent detour from Grok integration; no merge is authorized.

The affected OpenCode 1.18.30 session exported invalid 65,536-byte piped stdout
but valid 14,566,990-byte file output with 804 messages and matching identity.
The installed-version source writes stdout without awaiting it, then calls
process.exit() in its CLI finally block. This is producer-side pipe truncation,
not Agent Code's 256 MiB buffer limit or a provider-specific translator bug.
The private probe exported read-only, reported metadata only, and deleted output.

1. Add a real child-process regression that writes a large native-shaped JSON
export and exits immediately, plus genuine-invalid-JSON and command-failure
cases. Use synthetic text in the transport stimulus, never personal prompts.
2. Capture finite OpenCode CLI stdout directly into a private temporary regular
file, not an asynchronous pipe. Keep process execution shell-free; wait for
completion before reading; preserve bounded output/error handling and cleanup.
Apply at the shared CLI boundary so large resolved config output is protected
too, not just outbound session export. Do not repair JSON or repeatedly export.
3. Run switching/CLI integration tests and typecheck. Re-run the affected native
export through the fixed production API and report only size/identity/counts.
4. Read-only review, fix verified findings, publish a focused PR linked to #845.
No Grok changes, app launch, source-session mutation, or automatic merge.

## Verification

The large abrupt-exit export and config regressions failed at 65,536 bytes with
the old pipe transport. The regular-file capture passes both. Coverage also
exercises import/model compatibility, private permissions, cleanup, actual
spawn failure, nonzero exit, invalid JSON, sparse oversized output and bounded
stderr. Read-only review identified post-spawn error handling that could retire
capture ownership before close; a failing lifecycle regression reproduced it.
Errors are now recorded and settled at close, including failed-spawn close.

The actual affected session passed the opted-in production export test without
printing or retaining its contents. The 256 MiB final-size check is a memory
bound; polling detects disk overflow during execution but is not a hard quota.
Versioned upstream evidence: anomalyco/opencode v1.18.30,
packages/opencode/src/cli/cmd/export.ts and packages/opencode/src/index.ts.

## Merge with main (e9ac8bdf)

Main's e9ac8bdf (Refs #864) had meanwhile bounded every CLI call with
execFile's `timeout` plus an owned abort listener, both SIGKILL, with a 30 s
default, so a hung empty-session import cannot hold OpenCode Terminal startup
or survive stop(). The merge (36bf9581) rebuilt both bounds by hand on
`spawn` and kept main's invariants: settle only from `close`, cancellation
wins over timeout or overflow, and an already-stopped caller creates nothing.
Timeout errors now read `timed out after N ms` instead of execFile's
`Command failed: ...`.

## Review round (2026-09-12)

Two independent reviews of the merged head (Codex: request changes; Claude:
approve with comments) led to these changes:

- A spawn that fails before stdio exists (EMFILE/ENFILE) has no stderr stream.
The `child.stderr!` dereference ran before the lifecycle listeners, so the
next-tick child `error` became an uncaught exception, which the crash hooks
turn into an app exit. The listeners now come first and stderr is optional.
- Every kill path SIGKILLed only the direct child. OpenCode's npm launcher
spawns the native binary with inherited stdio and cannot forward SIGKILL, so
the native process kept the capture and stderr pipe open, and timeout, stop
and overflow never settled. The merge had also dropped execFile's stdio
destruction. Commands now run in a private process group on POSIX, and one
terminate() destroys stderr and then SIGKILLs the group. Resolving the native
binary was rejected: it would couple Agent Code to OpenCode's install layout
and still miss helpers a native CLI starts itself.
- The capture was removed only by an async finally, which an app quit does not
await. It is now unlinked, with its directory, before spawn and read through
the retained descriptor with positional reads, because the child moved the
shared offset. The import payload is still read by path and is out of scope.
- Transform export and import inherited the 30 s startup default. They now
pass 5 minutes, while the startup import and profile probes keep 30 s. The
14.6 MB export's duration was never measured and was not measured in this
round, because no live run against real sessions was authorized. Transforms
carry no AbortSignal, so the deadline is their only bound.
- Declined: an app-wide shutdown drain for finite CLI operations. With the
unlink, nothing persists across a quit. An orphaned child on quit predates
this branch (execFile had the same exposure). An app-owned operation drain
belongs to the quit lifecycle and operation ownership work in #919 and #918.
- The comments no longer claim that `spawn` lacks `timeout`/`killSignal`.
Spawn failure must now report ENOENT, and the late-stop replay is covered.

Regression evidence: real two-process launcher trees for timeout, stop,
overflow, and a descendant that left the group; a spawn-failure double without
stderr that errors on the next tick; a zero link count observed inside the
running child; and no named capture while an import child runs. With its fix
reverted, each regression failed for the expected reason: an uncaught EMFILE,
a call still pending past its bound, a surviving descendant, a link count of 1,
or a named capture directory.

## Review round 2 (2026-09-12)

Both reviewers re-checked 8c94da7e (Codex: request changes; Claude: approve
with comments). Every round-1 disposition held. Changes:

- terminate() could signal a released process-group id. Node reaps the direct
child before `exit`, but `close` also waits for stderr EOF, so the deadline,
stop() and the size guard stay armed after the reap while an escaped helper
holds stderr, and by then the group id is free for reuse. A timeout or
overflow followed by stop() also signalled twice. The group is now signalled
at most once, and only while exitCode and signalCode are both null. After
the reap, terminate() only destroys stderr, which is enough to release
`close`. Trade-off: a same-group member that outlives an exited leader is no
longer killed. It can only append to the unlinked capture, and readCapture
is bounded by the fstat size. The npm launcher waits for its native child,
so the tree the group exists for is unaffected.
- The launcher-tree tests armed a 2 s deadline at spawn but allowed 5 s for
fixture readiness, and read the launcher pid from a possibly reparented
ppid. A failed readiness wait also left unobserved trees running. The tests
now live in opencodeCliSessions.processTree.system.test.ts behind a spawn
seam that returns only after the fixture reports, so readiness precedes
every bound and the bounds stay tight. The launcher passes its own pid, the
test owns the launcher's ChildProcess from spawn, and cleanup aborts the
command before awaiting it.
- The detached-group comment said a wedged child runs "to the deadline". The
deadline is a timer in Agent Code's main process and ends with it, so a
dev-terminal Ctrl+C that kills Electron leaves a wedged CLI unbounded.

- Inherited readiness race in main's hung-import stop/timeout tests
(e9ac8bdf). The tests waited up to 2 s for the fake CLI's status while the
timeout case's 1 s deadline was already running. During round-2
verification, at load average ~42 with most of swap in use, the stop case
failed before stop() was ever called. Those tests now use the same readiness
gate as the tree tests (testing/spawnReadiness.ts), own the child from spawn,
check its exit on the ChildProcess handle (SIGKILL within 2 s of the
trigger), and abort in cleanup. No budget was simply widened.

Round-2 regression evidence, each with its fix reverted:
- Signalling after the reap: the reaped-leader case fails with a recorded
negative-pid SIGKILL.
- Removing the once-only guard: the timeout-then-stop case records two group
signals.
- The round-1 kill-path mutations still fail the moved tree tests:
- direct-child kill only: all six cases pending past their bound;
- no stderr destroy: both escaped-helper cases pending;
- no group kill: descendants survive, or the timeout-then-stop case never
triggers its stop.

## Review round 3 (2026-09-12)

Both reviewers approved 3096ed25 with comments. The production fixes were
verified with real processes, and the remaining items were test-only or record
keeping:

- An escaped fixture descendant was not reclaimed when readiness failed,
because cleanup found descendants only through the ready status. The
launcher fixture now records the descendant's pid as soon as spawn()
returns. Cleanup kills that owned identity after the command settles, whether
or not readiness succeeded, and a forced-readiness-failure case for the
escaped fixture proves no survivor remains.
- A failed test could leave its queued spawn gate for the next test's spawn.
Both system files now reset the spawn mock after each test.
- The reaped-leader regression's spy passed calls through, so under the very
mutation it guards it would have delivered a real negative-pid SIGKILL. It
now records such requests without delivering them.
- The polling helpers that the file split duplicated now live in
testing/processWait.ts.
- The PR body was brought up to date for rounds 2 and 3.
10 changes: 9 additions & 1 deletion src/main/providerSwitch/transcriptEngine.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -103,6 +103,12 @@ describe('host transcript adapter registry', () => {

await expect(getHostTranscriptAdapter('opencode').read('/project', 'ses_source'))
.rejects.toThrow('has an unfinished turn')
// Transform exports are the large ones #845 fixed; the startup 30 s bound
// must not apply to them.
expect(mocks.exportOpencodeSession).toHaveBeenCalledWith(
{ binary: '/tool', cwd: '/project', timeoutMs: 300_000 },
'ses_source',
)
})

it('imports one projected OpenCode envelope instead of treating it as JSONL', async () => {
Expand All @@ -111,8 +117,10 @@ describe('host transcript adapter registry', () => {

await expect(getHostTranscriptAdapter('opencode').write('/project', [value]))
.resolves.toBe('opencode://session/ses_target')
// A projected import carries a whole conversation, so it must not inherit
// the 30 s bound meant for the terminal's empty-session startup import.
expect(mocks.importOpencodeSession).toHaveBeenCalledWith(
{ binary: '/tool', cwd: '/project' },
{ binary: '/tool', cwd: '/project', timeoutMs: 300_000 },
value,
)
})
Expand Down
22 changes: 20 additions & 2 deletions src/main/providerSwitch/transcriptEngine.ts
Original file line number Diff line number Diff line change
Expand Up @@ -170,6 +170,24 @@ const codexAdapter: HostTranscriptAdapter = {
},
}

// WHY transcript transforms get their own, much larger OpenCode deadline:
// runOpencode's 30 s default came from e9ac8bdf (Refs #864), where it bounds a
// hung *empty-session* import that would otherwise hold OpenCode Terminal
// startup. A transform exports or imports a whole conversation instead. The
// session that exposed #845 was a 14.6 MB / 804-message export, and its export
// duration was never measured (the live probe ran before any deadline existed).
// Applying the startup bound here could turn the fixed truncation into a
// timeout on exactly the large sessions #845 made switchable, duplicable and
// rewindable. Five minutes still ends a genuinely wedged CLI, and runOpencode
// SIGKILLs its whole process group when it does.
//
// These transform calls carry no AbortSignal (nothing between the IPC handlers
// and this adapter threads one), so the deadline is their only bound; stop()
// cancellation exists for the terminal startup import alone. The target-profile
// probes (`debug config`, `models`) keep the 30 s default because their cost
// does not grow with conversation size.
const OPENCODE_TRANSFORM_TIMEOUT_MS = 5 * 60_000

const opencodeAdapter: HostTranscriptAdapter = {
provider: 'opencode',
async read(cwd, providerSessionId) {
Expand Down Expand Up @@ -197,7 +215,7 @@ const opencodeAdapter: HostTranscriptAdapter = {
throw new Error('Projected OpenCode resume must contain exactly one export object.')
}
const binary = getToolPath('opencode', 'opencode')
const sessionId = await importOpencodeSession({ binary, cwd }, values[0])
const sessionId = await importOpencodeSession({ binary, cwd, timeoutMs: OPENCODE_TRANSFORM_TIMEOUT_MS }, values[0])
return opencodeTranscriptFile(sessionId)
},
sessionId(values) {
Expand Down Expand Up @@ -346,7 +364,7 @@ async function loadOpencodeSnapshot(
providerSessionId: string,
): Promise<TranscriptSnapshot> {
const binary = getToolPath('opencode', 'opencode')
const exported = await exportOpencodeSession({ binary, cwd }, providerSessionId)
const exported = await exportOpencodeSession({ binary, cwd, timeoutMs: OPENCODE_TRANSFORM_TIMEOUT_MS }, providerSessionId)
assertStableOpencodeExport(exported, providerSessionId)
const conversation = decodeOpencodeConversation(exported)
// OpenCode exports one complete native message per array position. That
Expand Down
37 changes: 37 additions & 0 deletions src/providers/opencode/runtime/opencodeCliSessions.live.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
import { existsSync } from 'node:fs'
import { homedir } from 'node:os'
import { join } from 'node:path'
import { expect, it } from 'vitest'
import { claudeNativeResumeProjector, codexNativeResumeProjector, decodeOpencodeConversation, opencodeNativeResumeProjector } from 'agent-transcript-parser'
import { exportOpencodeSession } from './opencodeCliSessions.js'

// Explicit read-only native verification. No source/target sessions are
// modified, and no personal transcript values or source IDs become fixtures.
it('exports an opted-in large native OpenCode session completely through the production capture boundary', async context => {
const sessionId = process.env.OPENCODE_EXPORT_SESSION_ID
if (!sessionId) context.skip('Set OPENCODE_EXPORT_SESSION_ID to opt into a large native session export')
const binary = process.env.OPENCODE_BINARY ?? join(homedir(), '.opencode', 'bin', 'opencode')
if (!existsSync(binary)) context.skip('Installed OpenCode CLI unavailable')
const value = await exportOpencodeSession({ binary, cwd: process.env.OPENCODE_EXPORT_CWD ?? process.cwd() }, sessionId!)
const info = value.info as { id?: string } | undefined
expect(info?.id === sessionId, 'export identity matches the requested native session').toBe(true)
expect(Array.isArray(value.messages)).toBe(true)
const bytes = Buffer.byteLength(JSON.stringify(value))
expect(bytes, 'fixture source must exceed the observed pipe truncation boundary').toBeGreaterThan(128 * 1024)
const conversation = decodeOpencodeConversation(value)
expect(conversation.entries.length).toBeGreaterThan(0)
// Exercise the real conversion hub with the previously failing native
// export, but do NOT publish target sessions or execute any imported tools.
const target = {
cwd: process.env.OPENCODE_EXPORT_CWD ?? process.cwd(),
targetSessionId: '00000000-0000-4000-8000-000000000001',
now: '2026-09-09T00:00:00.000Z', version: 'fixture', cliVersion: 'fixture',
model: 'fixture-model', modelProvider: 'fixture-provider',
}
for (const projector of [claudeNativeResumeProjector, codexNativeResumeProjector, opencodeNativeResumeProjector]) {
const projection = projector.projectNativeResume(conversation, target)
expect(projection.values.length, `${projector.provider} native projection is nonempty`).toBeGreaterThan(0)
expect(projection.report.counts.preserved, `${projector.provider} retained native conversation content`).toBeGreaterThan(0)
}
console.info(`Native export verified: ${bytes} JSON bytes, ${(value.messages as unknown[]).length} messages; identity matches.`)
}, 60000)
Loading