Skip to content

Codex agents cross-render transcripts: new empty Codex agent prints another agent's JSONL #173

Closed

Description

@Juliusolsson05

Severity: CRITICAL

A new, empty Codex agent renders the JSONL transcript belonging to a different Codex agent in the same project. Codex panes cannot reliably differentiate their own transcript from those of sibling Codex agents.

Symptom

  1. Open a project that already has a running/active Codex agent (Agent A).
  2. Create a second, brand-new Codex agent (Agent B) in the same project. Agent B has sent no messages — its feed should be empty.
  3. Agent B's feed starts printing Agent A's JSONL rollout transcript instead of staying empty.

The two agents bleed into each other: a fresh pane shows another pane's history.

Why this matters

Suspected root cause

Codex rollout files live in a global, date-bucketed tree (not per-cwd like Claude):

~/.codex/sessions/YYYY/MM/DD/rollout-<timestamp>-<uuid>.jsonl

So every Codex agent across every project shares the same day directory. Transcript lookup must therefore be exact-keyed on the session's own rollout id. Two spots look unsafe:

  • src/main/providerSwitch/shared.ts — findCodexRolloutPathBySessionId() collects suffix matches and sorts by mtime, returning the newest. If the lookup key is missing/ambiguous, it hands back whichever rollout was touched most recently — i.e. another active agent's file.
  • src/main/sessions/historyLoader.ts — findCodexRolloutPathByThreadId() matches with f.includes(threadId) (partial substring, not an exact -<uuid>.jsonl suffix match) and returns the first filesystem-order hit.

A fresh Codex session also has no resumeSessionId and is allocated a timestamp-based key rather than a stable session id (src/providers/codex/runtime/codexSession.ts, the shell-<timestamp> branch). A new empty agent has no rollout id of its own yet, so any history load for it can fall through to "the latest rollout in this project/day" — which is the sibling agent.

(Root cause is suspected, not confirmed — needs a debugging pass to pin which path fires.)

Acceptance criteria

  • A newly created, empty Codex agent shows an empty feed, full stop — regardless of how many other Codex agents exist in the same project or were active that day.
  • Codex transcript lookup is keyed on an exact, unique rollout/session identity for the pane — never a "newest in directory" or substring fallback.
  • Switching between sibling Codex agents in one project never cross-renders transcripts.

Notes

  • Claude does not hit this because it stores per-cwd (~/.claude/projects/<cwd-hash>/<sessionId>.jsonl); the bug is specific to Codex's global session tree.

Activity

  1. Juliusolsson05 commented on May 17, 2026

    @Juliusolsson05
    OwnerAuthor

    Investigation update: upstream Codex already has a stable thread id, but Agent Code's TUI/filesystem path cannot see it early enough

    Fresh Codex rollout ownership is tricky because upstream Codex creates the real identity before it creates the durable rollout file.

    Findings from vendor/codex-src:

    • Fresh non-ephemeral sessions allocate a ThreadId before any rollout file exists. Session::new uses ThreadId::default() for InitialHistory::New | Forked, and ThreadId::new() is a UUID v7 (protocol/src/thread_id.rs).
    • That thread id is passed into RolloutRecorderParams::new (core/src/codex.rs).
    • The rollout path is precomputed at recorder init, but the file is not created immediately. RolloutRecorder::new builds SessionMeta and returns with file = None for new sessions (rollout/src/recorder.rs).
    • The path format is:
    $CODEX_HOME/sessions/YYYY/MM/DD/rollout-YYYY-MM-DDThh-mm-ss-{thread_id}.jsonl
    
    • session_meta is written only when the deferred writer is materialized by persisted rollout items/flush, not at process/session startup. Upstream tests assert the rollout file does not exist after recorder creation and only appears after a persisted item plus flush.
    • SessionConfiguredEvent contains both the session_id and precomputed rollout_path, and core emits it as the first event. But SessionConfigured is filtered out of rollout persistence, so watching rollout files cannot see this early identity.

    Conclusion:

    A stable provider identity exists inside Codex before the first transcript line, but Agent Code's current TUI/filesystem observer does not consume that internal event. From the outside, before the first rollout write, simultaneous same-cwd TUI sessions are not distinguishable by filesystem state. This is the original reason Agent Code fell back to filesystem timing for fresh sessions.

    Implication for this issue:

    The strongest long-term fix would consume an explicit Codex event/API that gives us thread_id / rollout_path for the PTY we spawned. Without that, fresh-session ownership has to be inferred from local signals such as the prompt submitted through this exact headless instance or proxy-observed response ids.

  2. Juliusolsson05 commented on May 17, 2026

    @Juliusolsson05
    OwnerAuthor

    Investigation update: why first new rollout wins exists and why it breaks

    first new rollout wins was used because fresh Codex sessions do not have a provider id when Agent Code spawns them.

    Current implementation path:

    • SessionManager.spawn() creates an Agent Code-local sessionId, but that is not Codex's provider/thread id.
    • Codex's provider id appears later in rollout session_meta.payload.id.
    • The renderer captures that id from session_meta and persists it as providerSessionId so later resume/history loading can work.
    • Since Codex stores rollouts in a global date tree, not a per-cwd/per-pane file, CodexHeadless.tailNewRolloutFile() currently snapshots existing files and latches the first later rollout-*.jsonl.

    That relies on these assumptions:

    • only one fresh Codex process creates a rollout during the startup window;
    • the first new rollout observed by this watcher belongs to this PTY;
    • the spawned Codex creates its rollout after the priming snapshot;
    • once session_meta lands, the captured provider id is correct.

    Parallel orchestration breaks those assumptions. If N fresh Codex panes start around the same time, all N watchers observe the same global sessions tree. The first child rollout can be latched by every watcher. After latchedPath is set, later rollout files are ignored and only reported as ambiguity.

    Once the wrong file is tailed, the bad association is durable:

    • tailFile() emits entries from the sibling rollout;
    • CodexSession forwards them as jsonl-entry;
    • SessionManager stamps them with this pane's Agent Code sessionId;
    • the renderer captures the sibling rollout's session_meta.payload.id as this pane's providerSessionId;
    • the existing guard refuses to overwrite providerSessionId later.

    Conclusion:

    The fresh attach path is identity-by-timing. It is acceptable only for serialized launches. In parallel same-cwd orchestration, it can cross-wire committed transcripts and persist the wrong Codex provider id.

  3. Juliusolsson05 commented on May 17, 2026

    @Juliusolsson05
    OwnerAuthor

    Investigation update: why the rollout-tail ownership logic changed in #159/#160

    The large rollout-tail ownership switch came from #159/#160.

    Old resume model:

    • findRolloutByThreadId -> tailFile(existing) was treated as authoritative.
    • That worked when codex resume <id> appended to the original rollout.
    • It failed when Codex reconstructed/forked the conversation into a new rollout file. Agent Code bootstrapped the old file, then the committed transcript channel went permanently silent while proxy semantics kept streaming.

    In the captured #159 case, 67+ Responses turns produced only the initial JSONL bootstrap. The renderer symptoms looked like feed clearing, missing user rows, and MCP/tool-call rendering bugs, but the real root cause was lower: runtime.entries was frozen because the committed rollout tail was dead.

    The #160 fix moved resume from:

    one provider id maps to one file forever

    into:

    keep tailing the original file, briefly watch for a newer same-cwd fork, and switch only when the candidate proves lineage against the resumed transcript

    The important design choice was cwd + lineage, not cwd alone. Cwd alone is unsafe because Codex rollouts are global and orchestration can run sibling agents in the same cwd. A blind cwd-only switch would steal a child agent's file and paint it into the parent pane.

    Tradeoffs from that change:

    Why fresh sessions stayed weaker:

    A fresh Codex pane has no original rollout and no prior transcript ids to fingerprint. Before the first durable session_meta, the only filesystem signal is "a new rollout appeared." That is why the fresh path kept first new rollout wins plus ambiguity diagnostics. This issue shows that was an unfinished bridge, not a valid ownership rule for parallel orchestration.

  4. Juliusolsson05 commented on May 17, 2026

    @Juliusolsson05
    OwnerAuthor

    Investigation update: orchestration makes the race expected, not exotic

    Four child Codex agents can spawn concurrently because the orchestration path intentionally has no serialization point.

    Path shape:

    • orchestration_create_agent sends each MCP call through OrchestrationBridge.createAgent() with an independent requestId.
    • The bridge tracks pending requests and emits them to the renderer independently.
    • The renderer calls createOrchestrationAgent() for each request.
    • SessionManager.spawn() mints independent Agent Code sessionIds and starts independent Codex PTYs.

    Renderer-side isolation is mostly correct:

    • orchestration children are stamped with orchestrationParentId / orchestrationRootId;
    • they are inserted as detached Dispatch sessions;
    • Dispatch nests them visually under the parent, using the same shape as linked agents.

    The isolation failure is below Dispatch: Codex rollout ownership.

    Fresh Codex sessions call tailNewRolloutFile(). That watcher snapshots the global ~/.codex/sessions/YYYY/MM/DD tree and then latches the first new rollout-*.jsonl it sees. If several same-cwd Codex sessions start together, every watcher can see the same first rollout and attach to it.

    Local transcript timing matches this risk: multiple Codex subagent rollouts were created in the same second, in the same cwd, with the same parent thread id but different child ids/nicknames. That is exactly the shape where per-session first new rollout watchers can collapse onto one file.

    What the renderer can fix:

    • row grouping;
    • focus/selection;
    • titles/labels;
    • parent-child metadata;
    • warnings if headless reports transcript ambiguity.

    What the renderer cannot fix:

    If main/headless emits the same rollout entries under four different Agent Code sessionIds, the renderer has no authoritative way to recover the true owner. It can only render the already-misattributed committed stream.

    Conclusion:

    Do not serialize orchestration just to hide this. Parallel child agents are a valid product requirement. The durable fix is to make fresh Codex rollout claiming prove ownership or fail closed.

  5. Juliusolsson05 commented on May 17, 2026

    @Juliusolsson05
    OwnerAuthor

    Investigation update: proposed ownership proof hierarchy

    Fresh Codex rollout ownership should not be cwd or first new rollout wins.

    Recommended proof hierarchy:

    1. Exact provider id

      If the pane already has providerSessionId or is resuming, attach only to a matching session_meta.id / rollout filename suffix. This is authoritative, but unavailable for brand-new sessions until after a rollout is chosen.

    2. Spawn-scoped proxy response lineage

      When proxy streaming is enabled, each Agent Code Codex session has its own ResponsesProxy. Correlate that private proxy's observed response.id, item ids, and call ids to candidate rollout fields such as turn_context.turn_id, payload turn_id, payload.id, or payload.call_id.

      This is the strongest fresh-session proof because proxy scope is per Agent Code session and separates same-cwd siblings. Ignore non-turn endpoints such as responses/compact, memories/trace_summarize, models, etc.

    3. Prompt lineage

      For orchestration-created agents, record the submitted prompt ownership key at submitPrompt() time and match it against the candidate rollout's first real user message (event_msg:user_message, falling back to non-bootstrap response_item user text). Normalize like Codex optimistic reconciliation does. This is useful, but not perfectly unique when workers share templated prompts.

    4. Item-id lineage

      Keep the resume/fork strategy already present: same cwd plus overlap of opaque rollout ids. For fresh sessions this becomes useful only after proxy or committed response items exist; it is not available before the first model/request items.

    5. Timestamps

      Use spawn time, submit time, session_meta.timestamp, file mtime, and first request time only as bounds/tie-breakers. They should reject impossible candidates, not prove ownership.

    6. Cwd

      Cwd is a necessary filter only. Same-cwd orchestration siblings are expected, so cwd alone must never attach or switch a pane.

    7. Request/proxy metadata

      Proxy requestId is local to a proxy process and useful inside proxy logs, but it is not written into rollout. Headers/body are forensic signals unless we add a deliberate correlation field.

    Failure rule:

    If multiple candidate rollouts reach the same best proof level, do not attach by recency. Keep the optimistic/proxy live surface, mark transcript ownership ambiguous, and wait for a stronger signal. A wrong attach is worse than a delayed committed transcript because it paints another agent's conversation into the pane and then persists the wrong providerSessionId.

    Implementation shape:

    Replace fresh first-new latch with a bounded candidate scorer. Buffer same-window rollout candidates, parse early session_meta / turn_context / first-user lines, ingest proxy-observed ids when available, and accept only the unique highest-confidence candidate. The existing resume fork detector has the right shape: cwd is a gate, lineage is the proof.

  6. Juliusolsson05 commented on May 17, 2026

    @Juliusolsson05
    OwnerAuthor

    Investigation update: smallest safe implementation boundary and tests

    This symptom is most likely the fresh-session live rollout tail, not initial-history loading and not feed rendering.

    Key facts:

    • A new Codex pane with no providerSessionId does not load initial history. Fresh Codex runtimes start with transcriptStatus: ready; initial history is loaded only when a provider session id exists.
    • Therefore an empty fresh pane printing a sibling transcript is almost certainly coming from live rollout tailing.
    • The unsafe boundary is CodexHeadless.tailNewRolloutFile().
    • Once the wrong file is tailed, session_meta is captured and forwarded as this pane's providerSessionId, poisoning ownership for that pane.

    Smallest safe implementation boundary:

    1. Fix fresh rollout claiming inside packages/codex-headless/src/CodexHeadless.ts, specifically tailNewRolloutFile() plus the local input/write boundary.
    2. Do not solve this in Feed or renderer state. By the time the renderer sees jsonl-entry, the file has already been associated with the wrong pane.
    3. For fresh sessions, do not tail a candidate rollout merely because it is new. Require an ownership proof before tailFile():
      • candidate cwd matches this headless cwd; and
      • candidate contains a user prompt matching a prompt submitted through this headless instance, or equivalent locally-derived submit/proxy fingerprint.
    4. Until a local prompt exists, ignore or hold same-cwd rollout candidates. A brand-new empty pane must remain empty.
    5. Keep resume fork detection separate. The resume path already has a lineage-overlap guard; Codex agents cross-render transcripts: new empty Codex agent prints another agent's JSONL #173 is the fresh-session path that lacks a lineage/prompt guard.
    6. Also harden exact lookup separately: src/main/sessions/historyLoader.ts still uses f.includes(threadId). That is probably not the empty-fresh-pane root cause, but it should become an exact -${providerSessionId}.jsonl / UUID match like the provider-switch helper.

    Proposed tests:

    • Fresh attach rejects sibling before local submit: start a fresh headless with temp CODEX_HOME; create a same-cwd sibling rollout before any local write; assert no rollout-entry, no session_meta, and no active tail.
    • Fresh attach accepts own first prompt: after local sendPrompt("own prompt") or equivalent bracketed paste, create a rollout whose first user item matches; assert entries emit and getSessionMeta()?.id is that file's id.
    • Same-cwd prompt mismatch: after local submit B prompt, create sibling rollout with A prompt; assert it is rejected/held, then create matching rollout and assert it is accepted.
    • Resume fork guard regression: same-cwd candidate with shared lineage ids switches; same-cwd candidate without lineage overlap does not. This likely requires extracting the candidate classifier into a testable helper.
    • Agent Code provider-id capture: feed a session_meta burst through the bulk jsonl path and assert providerSessionId is set only on the routed sessionId.
    • Orchestration fresh session: createOrchestrationAgent() creates a fresh Codex child with no resumeSessionId / providerSessionId; assert no initial-history load is requested until the child's own rollout session_meta arrives.

    Current coverage gap:

    There are rendering/session-ownership scripts, but no focused test for fresh Codex rollout attach ownership. That is the gap #173 needs covered.

  7. Juliusolsson05 commented on May 17, 2026

    @Juliusolsson05
    OwnerAuthor

    SQLite/state DB investigation: can it replace first new rollout wins?

    Short version: SQLite helps with session listing and thread_id -> rollout_path lookup, but it is not enough by itself to reliably map a fresh Agent Code Codex pane/PTY to the correct fresh Codex thread under parallel same-cwd starts. The DB has the right durable metadata once a thread is materialized, but it does not carry any per-process, PTY, pane, or launch-token identity we can correlate back to the child process we spawned.

    Relevant upstream findings:

    • Codex's state DB is state_5.sqlite under sqlite_home, defaulting to $CODEX_HOME unless sqlite_home / CODEX_SQLITE_HOME overrides it:
      • vendor/codex-src/codex-rs/state/src/runtime.rs
      • vendor/codex-src/codex-rs/core/src/config/mod.rs
    • The threads table stores id, rollout_path, created_at, updated_at, source, model_provider, cwd, title/sandbox/approval/git metadata, plus later agent metadata (agent_nickname, agent_role, agent_path). It does not store pid, pty, terminal id, parent process id, launch nonce, or Agent Code pane id.
      • vendor/codex-src/codex-rs/state/migrations/0001_threads.sql
      • vendor/codex-src/codex-rs/state/migrations/0013_threads_agent_nickname.sql
      • vendor/codex-src/codex-rs/state/migrations/0021_thread_spawn_edges.sql
      • vendor/codex-src/codex-rs/state/migrations/0022_threads_agent_path.sql
    • Process identity exists only in the logs DB path (logs.process_uuid), not in threads, and it is not a durable mapping from our PTY/pane to the thread row.
      • vendor/codex-src/codex-rs/state/migrations/0010_logs_process_id.sql
      • vendor/codex-src/codex-rs/state/src/log_db.rs
    • For a new session, RolloutRecorder::new precomputes the rollout path and SessionMeta, but explicitly defers file creation/open until a persist/flush path.
      • vendor/codex-src/codex-rs/rollout/src/recorder.rs
    • The SQLite thread row is written from the rollout writer after session_meta is written/materialized, via write_session_meta -> sync_thread_state_after_write -> state_db::apply_rollout_items.
      • vendor/codex-src/codex-rs/rollout/src/recorder.rs
      • vendor/codex-src/codex-rs/rollout/src/state_db.rs
    • Upstream already knows the authoritative rollout path before the file is materialized and includes it in SessionConfiguredEvent.rollout_path. Internally, upstream ThreadManager consumes SessionConfigured as the first event and stores the rollout path on the thread.
      • vendor/codex-src/codex-rs/core/src/codex.rs
      • vendor/codex-src/codex-rs/protocol/src/protocol.rs
      • vendor/codex-src/codex-rs/core/src/thread_manager.rs

    Current Agent Code / codex-headless behavior:

    • Fresh sessions still watch ~/.codex/sessions and attach to the first new rollout-*.jsonl:
      • packages/codex-headless/src/CodexHeadless.ts
    • That file already documents the exact ambiguity: if a second new rollout appears during startup, headless keeps the first and emits a possible wrong-session diagnostic.
    • Resume is stronger because it has a known thread id and can find the existing rollout by filename/thread id.
    • Session listing is filesystem/rollout based, with cwd filtering from decoded session_meta, not SQLite.
      • packages/codex-headless/src/transcript/SessionList.ts
      • src/providers/registry.main.ts

    Local proof from the current DB:

    • ~/.codex/state_5.sqlite does record spawned subagent rows with source = {"subagent":{"thread_spawn":...}}, agent_nickname, agent_role, cwd, first_user_message, and thread_spawn_edges linking them to the parent thread.
    • That is useful after the correct child thread is known.
    • It does not say which Agent Code pane/PTY was the owner of each child when several children start in the same cwd at nearly the same time.

    Can SQLite solve #173?

    Not completely. A DB-backed fresh attach could improve over raw filesystem watching by querying new threads rows where cwd == spawned cwd and created_at/updated_at >= spawn time, then taking the new row's rollout_path. That would avoid accidentally attaching to a different cwd. But under parallel same-cwd sessions, SQLite has the same fundamental ambiguity as the filesystem: multiple new rows can appear with the same cwd/source/time window, and neither row says which OS process or PTY produced it.

    Also, row timing is not earlier than the current durable rollout signal. For fresh sessions, SessionConfigured is emitted before initial history recording and before rollout materialization; the DB row is written when the rollout writer writes session_meta. So SQLite cannot give us an earlier fresh identity than the file unless Agent Code can observe upstream's structured SessionConfiguredEvent.

    Implementation path if we still want SQLite as a better fallback:

    1. Add a small read-only SQLite helper in codex-headless that opens ${CODEX_SQLITE_HOME || CODEX_HOME}/state_5.sqlite.
    2. On fresh start, snapshot existing threads.id before spawning/tailing.
    3. After spawn, poll for new threads rows with matching normalized cwd, existing rollout path, and created_at/updated_at after the spawn window.
    4. If exactly one candidate appears, tail threads.rollout_path.
    5. If zero candidates, fall back to current new-file watcher.
    6. If multiple candidates appear for the same cwd, treat attach as ambiguous and do not silently pick one. Continue using screen/proxy live data and surface a diagnostic.

    The real fix is still to stop inferring identity from shared storage. Best options:

    • Use upstream structured startup identity if Agent Code can access it: SessionConfiguredEvent.rollout_path is already the authoritative mapping.
    • Ask upstream for a PTY/CLI-accessible startup event or --print-session-configured-json style channel.
    • Ask upstream to persist a launch correlation token supplied by Agent Code into session_meta / threads, e.g. AGENT_CODE_PANE_ID or --session-source agent-code:<pane-id>, so DB/filesystem lookup can filter by a unique value instead of cwd/time.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions