Define trace/trajectory file ownership and preserve JSONL integrity across concurrent processes #185

Activity

1. justrach commented on Jul 12, 2026

   

   justrach

   on Jul 12, 2026

   Owner

   More actions

   Investigated end-to-end (against current main). The three mechanisms in the report are all confirmed present. Here's what's implementable now vs. blocked.

   Trace — solvable (unique-per-writer file, model 2)

   Give the trace a per-process name harness.trace.<12-hex>.jsonl (fresh io.random id per process, same source as the score run_id — no wall clock). This eliminates the cross-process truncation and gives provenance in the filename. It's fully wireable: the one place that surfaces the fixed name to the agent (the harness.trace.jsonl marker in main_system_prompt, src/prompts.zig) is rewritten at prompt-build time to the real path, and /trace + the startup banner print the actual file. Reader-facing change: consumers glob harness.trace.*.jsonl instead of a fixed name (same family, identical format). A prototype patch exists and passes zig build test (176).

   Trajectory — BLOCKED on the Zig 0.16 std

   The shared-offset race (writer.pos = st.size then a positioned write) needs an atomic append. Zig 0.16's Io.File exposes no O_APPEND equivalent — Io.Dir.CreateFileOptions/OpenFileOptions offer only read/truncate/exclusive/advisory lock/permissions/resolve_beneath, and the POSIX backend never sets APPEND. The only primitive is a whole-handle advisory flock, which for a long-lived accumulating file would serialize entire concurrent sessions, not individual appends. So atomic cross-process append is not achievable without either a std change (add O_APPEND) or a bespoke locked-append writer that fights the buffered File.Writer's own pos tracking. Not doing that fragile thing here.

   Net

   • Acceptance criteria met by the trace fix: single documented policy, no cross-process truncation, complete-record + zero-bytes-on-failure preserved, provenance that survives restarts (not PID-based).
   • Still open/blocked: trajectory atomicity (std limitation), startup detect/repair/quarantine of a torn final record, and the restart / forced-termination / multi-process-stress tests (the trajectory line is the blocker for the stress criterion).

   Keeping this open. The trace half can land independently if we accept the glob-instead-of-fixed-name reader change; the trajectory half wants an upstream O_APPEND (or a documented decision to accept per-epoch trajectory files instead of one accumulating file).
2. justrach commented on Jul 24, 2026

   

   justrach

   on Jul 24, 2026

   Owner

   More actions

   Resolved on main by the run-scoped file split in 5aed7f1 (#246). src/trace.zig:1-8:

   Every invocation writes its own .graff/traces/<run-id>.jsonl (performance telemetry) and .graff/trajectories/<run-id>.jsonl (DGM archive nodes plus the experimental behavioral lifecycle/belief stream), so independent graff processes never truncate or seek/write through the same file. Every record is stamped with the run id, process id, and runtime session id before the event's own fields.

   That answers both halves of this issue directly:

   • Ownership — each file is owned by exactly one process for its lifetime, keyed on run id (tracePath/trajectoryPath at src/trace.zig:43-49). There is no shared cursor to race on, so the startup-truncation and same-append-position failure modes are structurally gone rather than merely locked around.
   • Integrity — records stay pre-serialized and flushed whole, and now additionally carry run id + pid + session id, so a reader can attribute every line.

   Readers were updated to match: the archive reader scans all .graff/trajectories/*.jsonl files rather than assuming one (src/trace.zig:336).

   Reopen if a shared-file path still exists somewhere I have not spotted.
3. justrach

   closed this as completedon Jul 24, 2026