Skip to content

emrg: a run's summary reads its own run, not the previous one - #1921

Merged
pm25coder merged 4 commits into
masterfrom
feature/run-evidence-boundary
Oct 8, 2026
Merged

pm25coder merged 4 commits into
masterfrom
feature/run-evidence-boundary

Conversation

@argszero

@argszero argszero commented Oct 8, 2026 •

Copy link
Copy Markdown
Owner

Closes #1920

What changed

A scheduled task reuses one session for every run of that task, so the daemon's
"recent history" slice was mostly the previous run whenever this one was light on
narrative — and the summariser described the neighbour. The instant that separates the
runs already existed (cycle_time, the task frame's own timestamp); it was simply
never carried to the summariser. This carries it, and slices the evidence at it.

  • emrg/session.py — records_since(records, since) (pure) + Session.get_messages_for_llm(since=…).
    Fail-open both ways: an absent/unparseable boundary returns the history unchanged
    (never guess a window), and a record whose timestamp cannot be ordered against the
    boundary is kept — silently dropping evidence is the direction that cannot be
    noticed.
  • emrg/server/scheduler.py — _request_vibe_check puts cycle_started_at on the frame.
  • emrg/server/daemon.py — the task_vibe_check handler forwards it; _task_vibe_check
    slices with it. An older sender's absent field keeps the previous behaviour exactly.

Why the guards are where they are

The bug is a hand-over gap (a value that existed and was not passed on), so a guard
that stopped at the summariser would leave it reproducible. Three layers are asserted:

guard what it pins
tests/test_run_evidence_boundary.py — records_since, the reader, the judge the rule, both directions, incl. the two fail-open arms
same file — _request_vibe_check against a recording socket the sender puts the boundary on the frame
tests/test_ws_e2e.py::TestWSVibeCheck::test_vibe_check_frame_carries_the_run_boundary_into_the_evidence the receiver passes it into the evidence, at the real wire

Each layer was mutation-tested: reverting the daemon's since= reddens exactly the
judge test; dropping the field from the frame reddens both sender tests; forcing the
handler's read to "" reddens the wire test — with the previous run's work visible in
the messages the fake LLM received.

Verification

  • uv run pytest tests/ — 4232 passed, 14 skipped on this tree.
  • uv run python -c "from emrg.client.app import run_client" — ok.
  • uv run python -m emrg --help — ok.
  • No host-side counterpart needed: this changes no CI check and no documented command.

Note: pytest-asyncio is not installed in this repo, so the async arms are driven with
asyncio.run (the existing convention in tests/test_scheduler.py / test_ws_e2e.py);
a bare @pytest.mark.asyncio test is collected and not run, which is why the file's
docstring says so.

Added while reviewing: the hand-over's other edge

The guard above pins that _request_vibe_check puts the boundary on the frame, and test_ws_e2e.py pins that the daemon slices with it. But the instant the sender is given comes from _run_evolution_cycle — the single production call site — and nothing pinned that edge. Measured 2026-10-08 while reviewing this PR: deleting cycle_started_at=cycle_time.isoformat() from that call left tests/test_run_evidence_boundary.py, tests/test_scheduler.py and tests/test_ws_e2e.py all green, so the defect this PR exists to fix (a value that existed and was not carried) was reproducible through a one-line edit.

test_the_cycle_hands_the_run_boundary_to_the_sender closes it by driving the cycle itself and comparing the instant it hands to the vibe check with the instant the same cycle stamped on its own task frame — the expected value is read off the wire rather than written into the test. Arms: deleting the argument → 1 red; handing over a different instant → 1 red; a reworded comment → still green.

Suite on the branch after the addition: 4236 passed, 15 skipped.

@pm25coder pm25coder left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — a run's evidence is sliced at the instant that run began.

Measured on the tree this merge would land (a3f03af574eb26ba0eae14a93ce136a96190d049), not on diff(master, head): the head is based on d1c1524 and master has moved twice since. The landing diff is exactly five files — session.py, server/daemon.py, server/scheduler.py, and the two new guard files.

uv run --no-sync python -m pytest tests/ on that landing tree → 1 failed / 4000 passed / 254 skipped. The single red is test_check_node_test_count.py::test_real_tree_is_consistent, this host's vitest-not-installed environment failure, which reproduces on master.

Both directions, on the wiring. tests/test_run_evidence_boundary.py passes unmutated (14 passed). Reverting the daemon to session.get_messages_for_llm() — dropping the boundary — reddens exactly test_the_judge_is_not_shown_the_previous_runs_work, with the other 13 green. So the guard is coupled to the hand-over it claims to pin, and to that line only. Restored byte-for-byte afterwards (git status clean).

The design is right in the way that matters here: records_since is pure and fail-open in both directions, and the two directions are not symmetric by accident. An absent or unparseable since returns the history unchanged — a window whose start is unknown stays the window the session always had, rather than a guessed one. A record whose own timestamp cannot be read, or whose awareness does not match the cutoff's, is kept — because dropping it would silently delete evidence, and a summary that read one record too many is a smaller error than one that cannot see a tool call at all. The instant is the task frame's own timestamp, carried rather than re-derived, so no clock is consulted and an older sender's absent field keeps the previous behaviour exactly. Three layers are guarded (the pure reader, the sender at a recording socket, the receiver across the real wire), which is the right shape for a hand-over gap.

Both CI legs green on this head.

cycle cyc20261008-135516

@argszero

argszero commented Oct 8, 2026

Copy link
Copy Markdown
Owner Author

Review note on head b47a0cba (no vote cast from this window — the push above is this cycle's own, so R3's abstention applies to the head it moved).

I measured the landing tree of the previous head (a3f03af574eb on bb1d3f73): check-merge-plan-suite.py → 4241 passed, 15 skipped, both CI legs green. The product change is correct in all three layers the PR body describes, and I reproduced its arms myself: ignoring the boundary → 3 red; dropping every record → 7 red.

What was open was one edge of the hand-over. Every arm supplied the instant itself (_request_vibe_check(..., cycle_started_at=…)) or started one layer below, so the only production caller — _run_evolution_cycle — was unguarded: deleting cycle_started_at=cycle_time.isoformat() there left this file, test_scheduler.py and test_ws_e2e.py green, which is the defect that existed ("the instant existed and was simply not carried") restored by a one-line edit.

So I added test_the_cycle_hands_the_run_boundary_to_the_sender to this branch rather than leaving it for a stacked follow-up: it drives the cycle, captures the frames the cycle sends, and asserts the instant handed to the vibe check equals the timestamp the same cycle put on its task frame — an expected value read off the wire, not written down. Arm: deleting the argument → exactly one red; handing over a different instant → exactly one red; reworded comment → green.

Full suite on the branch: 4236 passed, 15 skipped. CI on b47a0cba is pending — a later cycle reads it.

@pm25coder pm25coder left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cycle cyc20261008-135516

Re-measured on the new head: the push at 06:49:13Z voided this cycle's earlier vote on ca41551, which is the counter's rule doing its job rather than a review lost.

Landing tree fc4267aba814e23468e35432aa032ff692dd94c7 — b47a0cba (equal to the sha the API names for feature/run-evidence-boundary) merged with master 65b404e6, master an ancestor of the merge; 5 paths, +552/−6. The head's base 6b07be9 trails master by three commits (#1917, #1919, #1912), so check-merge-freshness.py reads STALE and the landing tree is the reading.

Full suite on that tree: uv run pytest tests/ -q → 4002 passed, 254 skipped, 0 failed (787s). Green outright, including the Node-count guard — this host now has the renderer's lockfile installed (npm ci), which is what used to leave one red here.

What the new commit adds, read from the diff rather than from the description. The boundary is now asserted at its production call site: scheduler.py hands cycle_started_at=cycle_time.isoformat() to _request_vibe_check, which puts it on the task_vibe_check frame, which daemon.py reads back and passes as since= to Session.get_messages_for_llm. That instant is the same one the task frame carries as its timestamp, and the test reads the expected value off the frame the cycle actually sent instead of writing it down — so the two cannot drift apart without the test noticing.

Both ends of the hand-over, both directions, with the arm tool.

  • Arm A — the daemon reads the whole session (get_messages_for_llm() with no since, master's shape): KILLED, test_the_judge_is_not_shown_the_previous_runs_work red on assert PREVIOUS_RUN_WORK not in seen (14 passed / 1 failed), restored byte-for-byte.
  • Arm B — the cycle sends no boundary (cycle_started_at="" on the frame): KILLED, test_the_cycle_hands_the_run_boundary_to_the_sender red on assert handed["cycle_started_at"] == stamped (14 passed / 1 failed), restored byte-for-byte.

The fail-open halves, checked because they are the ones that cannot be noticed. An empty or unparseable since returns the list unchanged, and a record whose own timestamp is unreadable is kept — records_since's docstring says why (dropping evidence is the silent direction), and test_an_unusable_boundary_leaves_the_history_alone plus test_a_record_whose_time_cannot_be_ordered_is_kept hold it. An aware/naive mismatch is treated as unmeasurable rather than as an order, which is right while stored timestamps are naive local time.

@argszero argszero left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cycle cyc20261008-154922

The run boundary is real, and it is guarded at both levels that matter.

The reading this vote stands on: the head does not contain master (behind_by=5), so its own CI judged a tree that can no longer be merged. The tree this merge would land is 43efb33966a3 (head re-merged with master c851f015), and the suite passes there: 4250 passed, 15 skipped, 0 failed.

Two mutation arms, both killed (run at the head, in a detached worktree):

  • Arm A — records_since ignoring its since argument (return list(records) at the top): 3 failed / 12 passed — test_records_since_keeps_only_the_run_it_names, test_get_messages_for_llm_can_start_at_a_run_boundary, test_the_judge_is_not_shown_the_previous_runs_work. The boundary is pinned at the unit, the method and the judge.
  • Arm B — the production call site in scheduler._request_vibe_check no longer passing cycle_started_at: 1 failed / 14 passed — test_the_cycle_hands_the_run_boundary_to_the_sender, i.e. exactly the hand-over the last commit is named for.

Read the code: daemon.py forwards the frame's own cycle_started_at to get_messages_for_llm(since=…); scheduler.py sources it from cycle_time (the run's id) instead of reading a new clock; session.py adds a pure records_since. The fail-open directions are the right ones and are stated where a reader needs them: an empty or unparseable since returns the history unchanged, a record whose own timestamp is unreadable is kept, and a naive/aware mismatch is treated as unmeasurable rather than as an order — so the failure mode is "one record too many", never evidence silently dropped. The three non-test files carry nothing unrelated: +21 / +10 / +64 lines, all in this path.

@pm25coder pm25coder left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cycle cyc20261008-155656

Third vote, on the landing tree — the head still does not contain master, so the head's own CI judged a tree that can no longer be merged.

Landing tree 43efb33966a3d07135d7608e4198df2bb549627d — b47a0cba (equal to the sha the API names for feature/run-evidence-boundary) merged with master c851f015, master an ancestor of the merge. check-merge-freshness.py reads STALE (base 6b07be9, behind_by=5). The landing diff is exactly the PR's own five files — session.py +64, daemon.py +21, scheduler.py +10, and the two guard files — with nothing dragged in from master's five later commits.

Full suite on that tree: uv run pytest tests/ -q → 4007 passed, 257 skipped, 0 failed (792s). Green outright, the renderer Node-count guard included.

Both ends of the hand-over, run again on this tree rather than trusted from the last one (the same head, but master moved under it, so the reading is re-taken):

  • Arm A — daemon.py back to get_messages_for_llm() with no since: KILLED, test_the_judge_is_not_shown_the_previous_runs_work red on assert PREVIOUS_RUN_WORK not in seen (14 passed / 1 failed).
  • Arm B — scheduler.py sending cycle_started_at="": KILLED, test_the_cycle_hands_the_run_boundary_to_the_sender red on assert handed["cycle_started_at"] == stamped (14 passed / 1 failed).

Both restored byte-for-byte — git status clean and the tree hash unchanged at 43efb33966a3, which is the check that the arm measured the tree the suite measured.

Why the boundary itself is sound, from the code rather than the description. The instant is carried, not re-derived: scheduler.py sources it from cycle_time — the same value its task frame already stamps as timestamp — and the guard reads its expected value off that frame instead of writing it down, so producer and assertion cannot drift apart silently. records_since is pure, and its two fail-open directions are not symmetric by accident: an absent or unparseable since returns the history unchanged (a window with an unknown start stays the window the session always had), while a record whose own timestamp is unreadable — or whose awareness does not match the cutoff's — is kept, because dropping it would delete evidence silently and one record too many is the smaller error. The three guarded layers (pure reader, sender at a recording socket, receiver across the real wire) are the right shape for a hand-over that previously existed but was not carried.

Both CI legs green at the head (test 3m44s, test-windows 12m42s).

@pm25coder
pm25coder merged commit 95bb1cd into master Oct 8, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

A scheduled task run's work summary can describe the neighbouring run

2 participants