Repository navigation
Conversation
All six gaps from #168, client side (the coordinated backend half is live on harness-telemetry as of 2026-07-11, so v2 scores verify in prod): - Gap 1: scoreVariants' prov string now carries niches[i] — workflow tournament scores land in a real MAP-Elites cell instead of the anonymous one that promotion can never match. - Gap 2: the score envelope is v2 — niche and provider_class join the HMAC-signed message, so a score for one cell/tier cannot be replayed into another by mutating unsigned transport fields. All signing sites (scoreVariants, runEval, /score) emit v2; user-controlled envelope fields are sanitized (tab/CR/LF) and fingerprints hex-validated BEFORE signing so signed bytes always equal ingested bytes; run_id included. SDK signers/verifiers regenerated to match (v2-first with v1 legacy fallback), pinned by a cross-language golden vector. - Gap 3: fleetEvent asserts (debug builds) that a propose's prompt_text hashes to its claimed prompt_sha; oversized genomes (>64KiB, cap raised from 8KiB) are skipped rather than sent truncated, since a truncated genome can neither validate nor serve. - Gap 4: the wire scale is canonically [0,1]. runEval and scoreVariants divide their 0-100 local scores at the emission boundary (display, eval_best, and targets keep the 0-100 UX); out-of-range evals skip the score AND the paired fleet events; /score accepts an explicit scale ("unit"|"percent") and rejects unknown scale values loudly, with the documented [0,1]/(1,100] heuristic when absent. - Gap 5: scoreVariants proposes the genome before submitting, so a judged variant's cell always has the genome text promotion needs. - Gap 6: run telemetry now carries the same session run_id as score telemetry (and the run branch of the OTLP serializer actually emits it), giving operational and fitness rows a stable join key. zig build test 162/162; SDK regen byte-idempotent; backend suite 23/23 against the same golden vectors. Fixes #168 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
DGM: score metadata completeness for fleet promotion (fixes #168)
release/v0.0.200: codex WS delta re-anchor fix + idle timeout
…a errors (#122) std.process.raiseFileDescriptorLimit() in main() before subcommand dispatch lifts the soft NOFILE cap to the OS hard cap (Darwin: OPEN_MAX 10240 vs the 256 default), so parallel tool/subagent/MCP fan-out in one turn no longer dies with ProcessFdQuotaExceeded. tools.failure() and the background-job spawn path now explain an fd-quota hit and what to do about it instead of a bare error name (the old job-spawn advice — run it in the foreground — was actively wrong for this error). Verified end-to-end vs a scripted lmstudio mock: under ulimit -n 256 a bash tool child sees 10240 with this fix, 256 on v0.0.200. Co-Authored-By: blackfloofie <265516171+blackfloofie@users.noreply.github.com>
The 429/5xx retry branch leaves a 'retry' trace note; the transport-flake
branch (HttpConnectionClosing, resets, truncated TLS reads) said so on
screen but wrote nothing to harness.trace.jsonl, so provider flakiness was
invisible when diagnosing a session after the fact. Leave the same
breadcrumb with the error name.
Verified against an accept-then-close socket: all 6 flake retries now show
as {ev:retry, detail:HttpConnectionClosing} with backoff-spaced timestamps.
Co-Authored-By: blackfloofie <265516171+blackfloofie@users.noreply.github.com>
… slice 2b) The measured leak was path-specific: slice 2a (PR #162) moved the openai per-SSE-event parse trees to the scratch arena, but the codex/Responses and Anthropic paths still parsed EVERY stream event onto the process-lifetime session arena, and tool-argument JSON + oauth-refresh internals did too. Measured on the codex SSE path (local Responses mock, ReleaseFast, 30 turns): session arena +28 KB/turn monotonic before, +0.0 after (flat at 409 KB); RSS slope +8.8 -> +3.6 KB/turn (residual is history-scale, not leak). - parseResponses: events parse on the scratch arena; only what escapes the request is detached onto the session arena (output items via dupeJsonValue — stepResponses appends them to history verbatim — plus usage/id/error). Covers both codex transports (the WS path funnels frames into the same function as synthetic SSE lines). - assembleAnthropic: full assembly (event trees, delta accumulators, stitched message) on scratch; ONE deep copy of the finished message onto the session arena at return replaces the old whole-stream-on-arena scheme. - stepResponses/stepOpenAI tool-argument parses -> scratch (execution-only; the history message carries the arguments string, not the tree). KB-scale per file-edit tool call. - oauth.refreshOAuthKey call sites: internals (file read + JSON parse, ~1-2KB EVERY request on kimi/xai login sessions) -> scratch; the token is duped to survive. The per-request scratch reset moves to the top of request() so the refresh can use it. - util.dupeJsonValue: deep-copy helper + reset-and-clobber test. - GRAFF_MEM_DEBUG=1: per-turn 'mem' event (--json) reporting session/scratch arena capacity — allocator-level leak telemetry; RSS sampling alone cannot tell an arena leak from gpa noise (and a Debug-allocator build inflates RSS slope ~50x, which is what made earlier numbers look scarier). - benchmarks/README: document memory_session.py and the leak-vs-context distinction. Co-Authored-By: blackfloofie <265516171+blackfloofie@users.noreply.github.com>
…om context-window rejections (#174) Three-part fix for the wedged-session failure (meter reads 95k/270k while the backend rejects every request for exceeding the window): - recordUsageResponses takes max(server total, full-input estimate): the WS-chained total_tokens tracks a conversation the server has pruned (prior-turn reasoning is discarded from chained context), while every turn's FIRST request resends the full local history including all retained encrypted reasoning items. The meter now reflects what that resend costs (serialized input bytes / 4 via a counting discard writer, zero-alloc), so compact@80% fires before the wall — the same correction codex CLI applies (ContextManager get_non_last_reasoning_items_tokens). - 'exceeds the context window' rejections pin the meter to the window: the rejected request never returns usage to correct a stale meter, so without this the ApiError compact-and-recover gate can never pass and the session wedges permanently (every retry resends the same oversized history). - compact() drops prior-turn Responses reasoning items before the summary request (dropPriorTurnReasoning): the backend discards them from chained context anyway, and they dominate resend bloat on long high-effort sessions — this keeps the compaction request itself under the wall it is escaping. Current-turn reasoning stays (required between a function_call and its output). Verified against a scripted Responses mock (fat reasoning items, chained usage lying at total_tokens=100): meter grows 100 -> 10k over 6 turns instead of sitting at 100; a mid-session context rejection compacts and the session continues (history collapse visible as context_tokens 4115 -> 100); with the mock rejecting EVERY oversized body, every rejection now recovers instead of wedging. Unit tests for the estimator and the reasoning trim. Closes #174 Co-Authored-By: blackfloofie <265516171+blackfloofie@users.noreply.github.com>
…scratch The forced-refresh fallback stored the scratch-backed token slice when the session-arena dupe failed — a would-be UAF at the next request's scratch reset. Keep the previous key instead (the retry then fails cleanly), same policy as the near-expiry refresh site. Co-Authored-By: blackfloofie <265516171+blackfloofie@users.noreply.github.com>
…orm the session (#177) post() was built on client.fetch, which never exposes the Request: a request whose body SEND failed left reader.state == .ready, Request.deinit read that as "connection still clean", and the DEAD connection went back into the shared keep-alive pool. findConnection hands the free list out LIFO, so every retry — and every later same-host request — pulled the same corpse: the observed storm where a ~230k-token compaction request failed 6/6 and then, after /clear, a tiny [title] request failed 6/6 in the same process while live WS/SSE turns kept working (they bypass the pool). The streaming path has poisoned exactly this failure for a while (agent_stream.zig's errdefer conn.closing = true, "WriteFailed forever after (observed in the wild)"), and RetryPlan's doc comment even claimed postWatched had the same protection — it never did. - post(): client.request + sendHeadTask (same wire behavior as fetch for our POSTs) with the errdefer poison on every error path, plus an explicit poison on the 429/5xx early-returns that leave body bytes unread on the connection. - RetryPlan doc: describe the protection that now actually exists. - Regression test on a real loopback listener: conn 1 is closed unread so an 8MB send fails mid-write; the follow-up tiny POST must dial fresh and reach conn 2's 200 (it re-failed on the pooled corpse before the fix). Red-checked: 165/166 with the poison disabled, 166/166 with it. Fixes #177 Co-Authored-By: blackfloofie <265516171+blackfloofie@users.noreply.github.com>
…ets goal/ultracode steering (#178) The explicit "ultracode" scan ran over the fully ASSEMBLED turn message — base prompt plus the goal/todos/eval harness notes — so any standing goal containing the word kept re-triggering the explicit banner and workflow steering on every turn. And /clear kept root.goal and root.ultracode_mode (only /new reset them — an artifact of /new landing later, not a design choice), so even "context cleared — fresh conversation" didn't stop it: the observed session bannered ULTRACODE on a plain prompt right after /clear, every time. - applyUltracodeSteering takes the raw typed text alongside the assembled msg and scans ONLY the raw text for the codeword; the note still lands on the assembled msg. The oneshot path (session_run) already scanned the raw prompt; the REPL mainloop now matches it. - /clear resets the same conversation steering as /new (goal + ultracode_mode) via a shared helper, and says so when a standing goal is dropped. - Regression tests: codeword arriving via an appended harness note is not explicit; typed codeword still is (case-insensitive); persistent mode still notes without bannering; the /clear//new reset helper. Fixes #178 Co-Authored-By: blackfloofie <265516171+blackfloofie@users.noreply.github.com>
http: poison send-failed connections so one WriteFailed can't storm the session (#177)
ultracode: scan the codeword on the raw prompt only + /clear resets goal/ultracode steering (#178)
…t can't hang the binary The regression test's first assertion (expectError WriteFailed) had no safety net: if the 8MB send ever failed with something other than WriteFailed, the test returned before the second post() ran, leaving the server task parked in its second accept() and hanging the deferred fut.await. The trigger is unreachable today (Io.Writer collapses every send error to WriteFailed; the loopback RST lands on the first bytes), but the guard mirrors the second post()'s existing throwaway-dial release so the test can never hang regardless. Surfaced by the pre-PR adversarial review; test-only, no production change. Co-Authored-By: blackfloofie <265516171+blackfloofie@users.noreply.github.com>
Adds a loopback test for post()'s 429/5xx branch — previously the whole branch was uncovered; only the send-failed path had a test. conn 1 answers a keep-alive 500 with an oversized (>600B capture buffer) body; post() must partial-drain it, return error.ServerError, and the session must recover on the next request (fresh dial to conn 2's 200). Honest scope note (verified by disabling the poison): unlike the send-failed case, std does NOT re-pool a partially-read response connection on its own, so post()'s explicit 5xx poison is defensive/redundant — this test exercises the 5xx path end-to-end, it does not isolate that poison. The load-bearing #177 regression (a failed SEND leaves the reader deceptively .ready, so std DOES re-pool the corpse) remains covered by the send-failed test, which provably goes red when the poison is removed. Surfaced by the pre-PR adversarial review. Co-Authored-By: blackfloofie <265516171+blackfloofie@users.noreply.github.com>
The graff login codex PKCE callback (localhost:1455) served a single bare line of HTML. Replace it with a branded, dark-mode-aware confirmation card (centered, system font, prefers-color-scheme, charset=utf-8 so the ✓/◆ render) and bump the write buffer to fit it. Same close-tab-and-return semantics — correct for a CLI login. Folded into v0.0.201. Co-Authored-By: blackfloofie <265516171+blackfloofie@users.noreply.github.com>
fix(tools): raise the open-file limit at startup + actionable fd-quota errors (#122)
fix(trace): record transport-flake retries in the session trace
fix(codex): context meter tracks the full-history resend + recover from context-window rejections (#174)
fix(memory): keep per-request parse garbage off the session arena (#124 slice 2b)
…in fs errors #179: edit_file's atomic zigpatch splice (and write_file overwriting an existing file) landed new content on a fresh inode at the default 0644, dropping a 0755 executable bit. Capture the mode before the rewrite and re-apply it after (best-effort; a new file keeps the default). #183: read_file/edit_file/write_file surfaced bare 'error: FileNotFound'. Map the expected fs errors at the operation site to a message naming the tool, the supplied path, and the failure — edit points at write_file for creation, write distinguishes a missing parent dir. Unexpected errors still route to the generic failure path. Fixes #179 Fixes #183 Co-Authored-By: blackfloofie <265516171+blackfloofie@users.noreply.github.com>
A session with no user message was saved at startup as an 'Untitled session', cluttering the CLI/GUI pickers. Add hasMeaningfulState() (>=1 user message, or a goal, todos, or tool activity) and skip the write in saveSession until the conversation is meaningful. Existing files are never deleted; a goal-only session is still saved. Fixes #184 Co-Authored-By: blackfloofie <265516171+blackfloofie@users.noreply.github.com>
…re ApiError (#187) A child turn that failed at the API/transport layer collapsed to an opaque 'error: ApiError'. Read the child's captured last_api_error before its arena frees and return an is_error result naming the provider status/message, a coarse category (quota/transport/model/invalid/auth), and whether retry is safe. Fixes #187 Co-Authored-By: blackfloofie <265516171+blackfloofie@users.noreply.github.com>
…ersisted) Adds scripts/test-e2e-newer-issues.py (CI-wired) — runs the actual graff binary with a fake key + GRAFF_FORCE_STALL_ONCE (no network) and asserts a blank conversation leaves no *.session.json while a conversation with a user message is saved. Verified as a real guard: it fails on the pre-fix binary. Co-Authored-By: blackfloofie <265516171+blackfloofie@users.noreply.github.com>
On Windows the 8 MB body buffers and the peer reset surfaces as a read-side error at receiveHead, not error.WriteFailed at send — so the first assertion's 'must be WriteFailed' check failed the test (174/175) on windows-latest. post()'s errdefer poisons the connection on ANY error, so accept any first-request failure and let the real assertion (the next request recovers via a fresh dial) stand. Only an unexpected success is wrong. No production change; the shipped binary was unaffected (release cross-compiles Windows without running tests). Co-Authored-By: blackfloofie <265516171+blackfloofie@users.noreply.github.com>
test(http): make the #177 send-failed test cross-platform (fixes Windows CI)
…and on a later line (#182) #181: the goal command was save-only — it persisted the standing goal but never started a turn, while the confirmation implied work had begun. The goal case now persists the goal then submits exactly one turn for the objective (mirroring the TUI's set-goal-and-run-it), guarded against double-submit by an in-flight ref. /goal (show) and /goal clear stay save-only; no persisted conversation falls back to the save-only path. #182: parseSlashCommand only matches when the whole trimmed draft is one command, so a known command starting a later line was sent silently as prose. findLineLeadingCommand detects a known command at the start of a continuation line (anchored per-line, so mid-line tokens, paths, URLs, and quoted commands don't false-positive) and warns before submit, offering return-to-editing or send-as-text. Typecheck (tsc -b) clean; 15/15 useCommandRouter unit tests pass. Fixes #181 Fixes #182 Co-Authored-By: blackfloofie <265516171+blackfloofie@users.noreply.github.com>
The Codex Responses backend (chatgpt.com/backend-api/codex/responses) now rejects a top-level max_output_tokens with "Unsupported parameter: max_output_tokens" on gpt-5.6-* models, hard-failing every turn including the title task. openai/codex never sends it at the request top level — there it is only a tool argument (exec_command/wait output truncation). Drop the field from the Responses payload; output length is bounded by the model's own cap and our prompts. Tests now guard its absence. Co-authored-by: Codegraff <blackfloofie@codegraff.com>
… Responses Dedicated regression test: buildBody's .responses branch must never emit a top-level max_output_tokens (normal, compaction, or title mode) while still carrying the required Responses fields — the gpt-5.6 backend rejects the parameter. Complements the absence assertions in the WS-delta test. Co-authored-by: Codegraff <blackfloofie@codegraff.com>
fix(codex): drop top-level max_output_tokens from Responses requests
justrach
force-pushed
the
feat/246-behavioral-learning
branch
from
July 20, 2026 02:00
b159670 to
869a2a8
Compare
On Linux, a default openDir can yield an O_PATH handle; fchmod on O_PATH
fails EBADF, which std.Io (0.17-dev) escalates to a programmer-bug panic
that catch {} cannot swallow. The kimi catalog hydration added in 7e08374
made this reachable mid-session: once ~/.kimi/device_id exists, any
/model query that may use kimi re-enters initIdentity -> secureDir and the
process panics (seen as the JSON live-control test losing all events after
'set_model provider/model keyless' on Linux CI; macOS never hit it because
its dir handles are always chmod-able).
Co-Authored-By: Codegraff <blackfloofie@codegraff.com>
…utput_tokens The midturn/transactional scenarios still expected explicit 16k/4k/16k top-level output caps, but #247 removed the field entirely (gpt-5.6 rejects it). CI never caught the stale expectation because the json-controls step failed first and the job stopped. Assert the new contract: the field must be absent from every Responses request. Co-Authored-By: Codegraff <blackfloofie@codegraff.com>
…st (#247) Co-Authored-By: Codegraff <blackfloofie@codegraff.com>
Add a backward-compatible behavioral event stream with private local JSONL, bounded metadata/content projections, collector authentication, and terminal upload deadlines. Keep prompt fingerprints local to the behavioral channel. Add a content-addressed local learning store and strict learn CLI with pinned adapter snapshots, recomputable paired evaluation, stale-run binding, manual promotion, rollback, verification, and protected learned-policy precedence. Document the Phase 1 privacy and trust boundaries and exercise the lifecycle in unit, provider-boundary, HTTP, and Linux E2E tests. Co-Authored-By: Codegraff <blackfloofie@codegraff.com>
Reopen directory handles before syncing so Linux never fsyncs an O_PATH descriptor. Correct Zig 0.16's Windows no-follow handle flag before reads so asynchronous completion is handled instead of reaching an unreachable path. Add focused regressions for path-only directory sync and exact no-follow reads. Co-Authored-By: Codegraff <blackfloofie@codegraff.com>
) std.meta.fields is a compileError deprecation on the 0.17.0-dev CI toolchain; iterate comptime std.meta.fieldNames + @field instead, matching the pattern already used in trace.zig. Behavior unchanged (271/271 on zig 0.16). Co-Authored-By: Codegraff <blackfloofie@codegraff.com>
…rt into sibling modules (#246) The rebase onto main pushed trace.zig to 1233 lines and telemetry.zig to 799, tripping the CI source-size guard. Split along existing cohesion boundaries: - behavior_trace.zig: BehaviorKind/BehaviorTrace + envelope writer (trace.zig re-exports the public names; call sites unchanged) - behavior_trace_tests.zig: the behavior lifecycle/upload tests - telemetry_net.zig: validatedAuthKey, otlpLogsUrl, mock collectors + their bounded-deadline POST tests - telemetry_tests.zig: dupDetail/writeOtlp payload tests No behavior change: 273/273 unit tests + learn E2E pass on zig 0.16. Co-Authored-By: Codegraff <blackfloofie@codegraff.com>
#246) Main's new 699-line guard (scripts/check-zig-lines.sh) flagged six files after the rebase. Split along existing cohesion boundaries, keeping re-exports so call sites are unchanged: - behavior_upload.zig: tests -> behavior_upload_tests.zig - learn_eval.zig: stats helpers -> learn_stats.zig, tests -> learn_eval_tests.zig - learn_store.zig: validation -> learn_store_validate.zig, tests -> learn_store_tests.zig - main.zig: behavioral sink wiring -> behavior_trace.Boot (boot/link/ finishAndClose preserves the LIFO teardown order), /bash test -> main_tests.zig No behavior change: 276/276 unit tests, learn E2E, fmt, and the size guard all pass on zig 0.16. Co-Authored-By: Codegraff <blackfloofie@codegraff.com>
justrach
force-pushed
the
feat/246-behavioral-learning
branch
from
July 20, 2026 05:21
a724d78 to
1a33a01
Compare
…or loss (#246) The local behavioral JSONL was dead in every real session: the DGM trajectory file is exclusively created first at .graff/trajectories/<run_id>.jsonl from the same run id, so the behavioral exclusive create always hit PathAlreadyExists and silently disabled local capture. Move the stream to .graff/behavior/<run_id>.jsonl, print one stderr warning when the local sink cannot open, and report local_sink in the uploaded run_started projection. Distinguish pre-write drops from sink failures in writeBehaviorLine: an oversized or unserializable event now drops (counted, and declared as local_dropped on run_finished so every seq gap reconciles) instead of permanently killing the local file while upload kept reporting the run complete. Fail closed on missing secure entropy by disabling upload rather than degrading the commitment-reference key. Gate postBatch on waitForClientReady like postOtlp (CA-bundle prewarm race, #131), enforce at comptime that TurnMetrics stays content-free u64 counters, flush buffered OTLP telemetry on the one-shot fatal path, and widen a flaky wall-clock test bound. Co-Authored-By: Codegraff <blackfloofie@codegraff.com>
verifyRun let duplicate_candidate overwrite identical_parent, diverging from runCommand's precedence and making any run with two parent-identical candidates permanently unpromotable (CandidateMismatch on a legitimate record); mirror the writer exactly and cover the round trip in the E2E. Un-wedge failed init: validate the parent genome before the store tree is created, and let initAt resume over the residue of an interrupted init (a learn tree without refs/active.json) instead of returning AlreadyInitialized forever. Refuse a default-target rollback when the active transaction is itself a rollback (it would silently move forward and reinstate the genome just rolled away from). Deny PATH, ComSpec, SystemRoot, and LD_/DYLD_ prefixes in pass_env, case-insensitively. Fail closed on missing secure entropy for trial nonces and scratch names, clean up the scratch dir when its open fails after create, print a bounded stderr excerpt when a mutator/evaluator fails, and accept case-variant pinned paths on case-insensitive filesystems (the sha256 pin is the authenticity boundary; symlinked components still resolve differently and still fail). Co-Authored-By: Codegraff <blackfloofie@codegraff.com>
) scripts/score_run.py recomputes everything from the event stream alone: strict Phase 1 audit (envelope, seq/gap reconciliation against local_dropped, dense turns, commitment fields) plus behavioral metrics (mispredict rate, commitment coverage, mean repair gap), and with --profile replay --baseline --game the reference RHAE math. Validated against the published schema-harness dataset: the converted ft09 trajectory reproduces its 100.0 and bp35 its non-saturating 93.51 exactly, per-level scores included (issue #246's validation criterion). scripts/replay_schema_harness.py performs the mechanical conversion into codegraff.behavior.v1; real traces carry kinds beyond the documented nine (turn_fallback), which the replay profile tolerates like the reference scorer while strict mode stays pinned. scripts/test-behavior-trace.py runs in CI: a real one-shot session must produce BOTH the behavioral file and the legacy DGM trajectory for the same run id (regression for the exclusive-create collision), pass the strict audit, and the scorer/replay math is checked on hermetic fixtures. Co-Authored-By: Codegraff <blackfloofie@codegraff.com>
#255) GRAFF_BEHAVIOR_TRACE=full (new value; off/0/false/no still disable, anything else stays lifecycle-only) turns on tool_started/tool_finished/action_taken/ text_delta on BehaviorTrace, gated by rich + turn_active + started, mirroring recordExpectedAction's locking pattern. tool_started caps args at 4096 bytes (embedded verbatim when valid JSON, else a UTF-8-safe string; always a string when truncated); tool_finished never carries result content; action_taken fires only for write/shell tool classes (behavior_upload.toolClass, now pub); text_delta caps at 2048 bytes. A new BehaviorTrace.reserveCallId() reserves a monotonic per-run call_id under the existing mutex. eventLocked() now gates the uploader by kind (uploadEligible): the four rich kinds are local-file only, never offered to behavior_upload.Upload, so they can't fail the deployed collector's batch and aren't counted in dropped_events. New Tracer.toolStarted/toolFinished forward from exec.zig's execTool (call_id reserved before any gate/dispatch, threaded through both completion paths including gate denials); Tracer.textDelta forwards from providers.zig's runTurnWithFallback with the root turn's final text (no single per-segment choke point exists across provider streaming paths). scripts/score_run.py's strict profile now accepts the four kinds: requires turn, requires call_id on tool_started/tool_finished, enforces call_id pairing (tool_finished without tool_started is an audit error), and reports new derived metrics tool_calls/tool_errors/text_deltas (actions already existed). scripts/test-behavior-trace.py's check_scorer gained a synthesized rich stream covering pairing plus the mismatch failure case. Also fixes two pre-existing compile/assertion breaks on this branch found while getting `zig build test` green: writeBehaviorLine's LineResult enum return (a test still compared it against bool) and two stale expectObjectKeys key lists that predated the local_sink/local_dropped fields. docs/behavioral-trajectories.md moves the four kinds from "reserved, not emitted" to a new "Opt-in rich capture" section (fields, caps, the local-only rule, and the local privacy posture change in full mode); the Phase 1 collector-rejection note is preserved and reinforced. README's behavioral paragraph gets one sentence on the opt-in. Co-Authored-By: Codegraff <blackfloofie@codegraff.com>
…/model_mispredicted producer (#256) runEval (src/agent_eval.zig) now commits to `{kind:"eval"}`/`{pass:true}` via BehaviorTrace.recordExpectedAction before running the configured --eval command, using an opaque per-invocation commitment_id ("eval-<turn>-<k>", k = self.eval_iter). It resolves that commitment with recordMisprediction only when the command exits nonzero, fails to spawn, or the parsed score misses --until; a met target leaves the commitment unresolved, matching the docs' "no misprediction = success" contract. Only static/opaque values ever reach either typed field - never the eval command text or its stdout/stderr. Also fixes two pre-existing breaks already present at the branch tip (79da927): behavior_trace_tests.zig's oversized-line test compared writeBehaviorLine's LineResult enum against a bool (compile error), and the metadata-projection test's expected key lists were missing local_sink/ local_dropped, which a prior commit had already added to the wire format. Updates docs/behavioral-trajectories.md and README.md's behavioral- trajectories section to name the eval-driven loop as the first production adapter instead of claiming none exists. Tests: behavior_trace_tests.zig gets a producer-pattern test using the public BehaviorTrace APIs directly (one failure -> turn_committed then model_mispredicted, same commitment_id/turn; one success -> commitment only). agent_eval_tests.zig adds a focused runEval integration test that spawns real /bin/sh commands, wired through a real Tracer+BehaviorTrace, asserting the same pattern end-to-end and that the local JSONL never contains the command text or its output. Co-Authored-By: Codegraff <blackfloofie@codegraff.com>
tests/learn_e2e.py now generates each fixture tool as an interpreter + pinned-input Program: a tiny launcher (.cmd on Windows, .sh on POSIX) execs the pinned CPython interpreter with the real script, whose original path travels as a Program `input` so invoke()'s args substitution rewrites it to the scratch snapshot before exec. CreateProcess cannot run a .py's shebang line directly, so this is the only shape that works on Windows; the mutator keeps the previous plain shebang model unchanged on POSIX, while the evaluator always goes through the shim so the substitution path is actually exercised (and proven, via a passing local run) on this non-Windows machine too. No src/learn_eval.zig change was needed: snapshotExtension() preserves a program's extension into the scratch copy, and Zig's std.process already resolves .cmd natively on Windows (WindowsExtension covers bat/cmd/com/exe), so the launcher's extension carries through correctly. .github/workflows/ci.yml: added a "Local learning lifecycle and security regression (Windows)" step to the windows job, running tests/learn_e2e.py against zig-out/bin/graff.exe after the build. Also fixed two pre-existing behavior_trace_tests.zig failures uncovered while getting `zig build test` green (unrelated to #257, left over from an earlier LineResult/local_sink/local_dropped refactor on this branch): a stale `!writeBehaviorLine(...)` bool check against the now-enum LineResult return, and two expectObjectKeys() allowlists missing the local_sink/local_dropped fields the upload projection already emits. Co-Authored-By: Codegraff <blackfloofie@codegraff.com>
CI pins a 0.17.0-dev nightly whose Type.Struct member layout differs from 0.16 stable, so @typeinfo(TurnMetrics).@"struct".fields compiled locally but failed on every CI job. std.meta.fieldNames plus @FieldType express the same closed-set u64 invariant on both toolchains. Co-Authored-By: Codegraff <blackfloofie@codegraff.com>
Owner
Author
|
Closing as already shipped. This PR's base is Head commit Reopen if any part of #246 is still outstanding. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
codegraff.behavior.v1lifecycle stream under.graff/trajectories/<run_id>.jsonlgraff learnCLI for pinned mutation/evaluation, manual promotion, explicit two-key auto-promotion, verification, and append-only rollbackSupporting collector PR: justrach/zigrepper#143
Phase 1 boundary
This establishes the behavioral backend and local learning engine without claiming a fully automatic generic predict → act → verify → repair loop. Local evidence remains authoritative; collective remote promotion authority is not implemented.
Validation
git diff --checkzig build test --summary all— 200/200 passedzig buildpython3 tests/learn_e2e.py --graff zig-out/bin/graffDeployment
No Codegraff or collector deployment was performed.
Part of #246 (Phase 1 plus review hardening; phases 2-4 tracked on the issue)