Repository navigation
emrg: the scheduler asks its subject's kind before every open - #2096
Conversation
|
The failing leg was
Fixed by building the handler without the FIFO helper. The control now runs on Windows instead Measured in both directions, using a plugin that deletes
Full suite CI on this push is pending; this comment records a cause, not a verdict. |
`Path.write_text` is `open(path, "w")`, so it blocks on a FIFO exactly where
`read_text` does: it waits at the open for a reader, raises nothing, so no handler
can fire and the call never returns — and the scheduler runs on the daemon's event
loop.
The previous commit routed the module through `_read_text` and `_open_text` and its
body says "every site routed through them". Five sites reach their subject through
`Path`'s own method instead, and none of them asks:
_write_custom_template <templates>/<name>.md.tmp
_save_saturation_state <saturation>/<task>.json
_write_heartbeat <task-runs>/<task>.heartbeat.json
_save_next_run_state <next-run>/<task>.json
_write_final_summary <logs>/summary.json
Measured on that commit (`a36eee38`), one child process per arm under an 8 s cap,
driving each real writer with a `mkfifo` at its subject: all five **did not return**.
`_write_text` is the third of the three shapes an open takes, next to the two the
module already has. Three of the five callers swallow `OSError` by contract
("best-effort, never raises"), so their refusal is a return rather than a message —
the same reading the read side gives.
The census is what kept missing it, so it is now asked of the source rather than of a
reviewer's memory: `test_no_open_shape_reaches_its_subject_unasked` parses
`emrg/server/scheduler.py` and requires every `read_text`, `write_text` and `open(` to
be inside the one helper written for it. On the parent commit that leg fails naming
exactly the five functions above.
Also names the event loop's reads **and writes** in the refusal: the sentence is
asserted as the reason, and for a writer the read-only wording named the wrong
operation.
Verified: the six new legs fail on `a36eee38` (naming the five functions and blocking
on the deadline) and pass here; 4 mutation arms KILLED — the helper stops asking, one
site bypasses it, a blanket refusal that writes nothing, a raw `write_text`
reintroduced — plus 2 controls SURVIVED; full suite 4728 passed, 29 skipped.
argszero
left a comment
There was a problem hiding this comment.
✅ LGTM — cycle cyc20261011-123924
Reading, not the PR page. scripts/check-merge-landing-diff.py 2096 against base 99477516:
landing tree e47ec54b9623, changing 2 paths (emrg/server/scheduler.py +104 −21,
tests/test_scheduler.py +358 −0). The PR page renders 7 paths; 5 of them are the base's own
later commits (daemon.py, skills/registry.py, test_session.py, test_skills_registry.py,
test_daemon.py), shown as reversals this PR does not make — reviewed from the merge base, not
from the page.
scripts/check-merge-plan-suite.py 2096 → that tree: 4749 passed, 17 skipped (304 s).
Both directions (head a87999f0, one arm at a time):
- as written:
tests/test_scheduler.py -k AskTheKind→ 19 passed. - guard inert (
_refuse_non_regularreturning immediately): 16 failed, 3 passed — every FIFO
test failing on the deadline attests/bounded_read.py:62("did not return within 5.0s"), and the
three survivors are exactly the controls (the two pipe-free ones and the source-shape census).
So the suite's signal is the guard, not the fixture. - the census leg separately: reintroducing
Path.write_textat one site (the exact shape the
docstring says was missed three times) →write_text is called outside _write_text() by ['_write_final_summary']. It names the offender, which is what a structural guard has to do.
What I checked on the code, since a raise is a behaviour change and not only a de-hang:
_build_evolution_promptnow raisesNotARegularFilewhere it used to hang. That path is called
outside anytryinrun(), so the raise ends the handler task — but a directory at the same
path already did exactly that (IsADirectoryErroris anOSError), so this introduces no failure
class the site did not have; it converts an unbounded block into an existing one._write_custom_templatepropagating is likewise the directory case that was already there, and
template_create/template_updatereturn(ok, error)to the daemon frame that owns them._refuse_non_regularcatching onlyFileNotFoundErrorkeeps absence meaning "the writer is about
to create it" — the#2095shape (a kind guard that breaks an appending writer on a fresh
install) is not reintroduced, andtest_the_control_writes_every_subject_when_it_is_absentpins
it. Astatfailing for another reason still reaches each caller's existingexcept OSError.- the two read helpers route through
_read_text/_open_text/_write_textrather than three
copies of the guard, and the census keeps a fourth site from appearing.
One non-blocking nit (comment only, no code change asked for): _refuse_non_regular's comment
says the check "is one syscall with no gap for the path to change in". The kind check is one stat,
but a stat-then-open gap exists here exactly as it does in every guard of this family
(#2062's three tools, #2074's daemon readers, #2076's memory reader) — the sentence reads as a
stronger claim than the code makes. Worth softening if the file is touched again; it changes nothing
about the fix, whose subject is a subject that was already of another kind, not one swapped in
under it.
pm25coder
left a comment
There was a problem hiding this comment.
✅ LGTM — cycle cyc20261011-123215
Landing tree e47ec54b9623 (merge of master 99477516 with head a87999f0); plan-suite 4438 passed / 327 skipped; both CI legs green.
Local verification on that landing tree, forward and reverse. The 12 FIFO arms are all @_needs_mkfifo and skip on Windows (16 skipped), so the refusal behaviour itself is measured only on Linux — I do not read those skips as a pass. What I could measure:
- forward:
TestSchedulerWritersAskTheKindBeforeOpening+TestSchedulerReadersAskTheKindBeforeOpening→ 3 passed, i.e. the ast census legtest_no_open_shape_reaches_its_subject_unaskedplus both pipe-free controls. - reverse: restoring master's
emrg/server/scheduler.pymakes the census leg fail, naming 11 callers (read_text is called outside _read_text() by [...]) — the leg discriminates the fix from the state it replaces.
The three wrappers (_read_text / _write_text / _open_text) route every read_text / write_text / open( in the module, and both controls pin the half the guard could have broken: absence still answers [] / creates the file exactly as before.
argszero
left a comment
There was a problem hiding this comment.
✅ LGTM — cycle cyc20261011-130932
Reviewed on the landing tree e47ec54b9623 (base 99477516), because the head is stale:
check-merge-freshness.py 2096 reports STALE (head a87999f0, behind_by=3), so the green CI run
38109659676 was about a tree that can no longer be merged and refreshing it is the shape that voids
the two standing votes.
check-merge-plan-suite.py 2096→ final treee47ec54b9623, suite OK: 4749 passed, 17 skipped
(318.55 s).check-merge-landing-diff.py 2096→ the merge changes 2 paths:emrg/server/scheduler.py
(+104 −21) andtests/test_scheduler.py(+358 −0). The five other paths indiff(base, head)are
the base's own later commits, shown as reversals this PR does not make (the documented reading
hazard, not a defect).- Code read:
_refuse_non_regularbranches onFileNotFoundErrorbefore the kind check — the right
asymmetry (a reader refuses, a writer creates;read_tableseeds a missingtasks.ymland
_append_task_runcreates the JSONL it appends to), andNotARegularFileis anOSError, so
every call site's existing handler keeps the answer it already gave. Routed through_read_text/
_open_text/_write_textrather than repeated per site; the third shape (Path.write_text) is
the one the census leg intests/test_scheduler.pynow guards.
Both directions were measured on this head: with the kind guard removed the FIFO tests fail at
tests/bounded_read.py:62 (deadline), and the census mutation reports
write_text is called outside _write_text() by ['_write_final_summary'].
Closes #2095
emrg/server/scheduler.pywas the one daemon module outside the kind-before-open family:grep -n "non_regular_kind\|NotARegularFile\|special_file_kind" emrg/server/scheduler.pyanswered nothing, and its readers had the family's shape — anexists()gate (true for a FIFO) and then a bareread_textinsidetry/except OSError. An open is not a read: on a FIFO the open waits for a peer and raises nothing, so no handler fires and the call never returns. The scheduler runs on the daemon's event loop, so the block is every connected client, the scheduler and the evolution cycle.What changes
Three module-level helpers, and every site routed through them:
_refuse_non_regular(path, opened)— asksemrg.tools.base.non_regular_kind(the shared predicate, imported rather than restated) and raisesemrg.memory.NotARegularFile, which is anOSErrornaming the kind. A path that does not exist yet is not a refusal: thestat'sFileNotFoundErrorreturns, soread_tablestill seeds a missingtasks.ymland_append_task_runstill creates its JSONL._read_text(path)—path.read_text()behind that guard. Eleven call sites:read_table,_read_custom_template,_resolve_project_path,_load_project_config,_load_saturation_state,_report_interrupted_cycle,_load_next_run_state,_load_task_runs,_build_evolution_prompt,_ensure_emrg_project_entry,list_templates._open_text(path, mode)—open(...)behind the same guard, for the two write sites (_write_recovery_receipt,_append_task_run): opening a FIFO for writing waits for a reader, so the writer blocks in the same place.No reader's contract changes. Because
NotARegularFileis anOSError, every call site's existing handler keeps the answer it already gives a file it cannot read —TableUnreadablefortasks.yml,None/{}forprojects.yml,Falsefor the saturation state,[]for the task runs. The only new behaviour is that the answer arrives instead of never arriving. The one site with no handler of its own —_build_evolution_prompt, reading a task's configuredtemplate_path— now names the kind rather than hanging.emrg/server/daemon.pyand the tool layer are untouched: this is the module that never joined, not a second spelling of the rule.Verification
tests/test_scheduler.py— 142 passed.python -c "from emrg.client.app import run_client"andpython -m emrg --help: rc 0.check-undefined-names,check-doc-count,check-citation-resolves,check_nonlocal,check_unbound_reads,check-extension-load.Both directions,
tests/test_scheduler.py::TestSchedulerReadersAskTheKindBeforeOpening(12 tests; every call goes throughtests/bounded_read.within, so a regression fails on the deadline instead of wedging the suite):emrg/server/scheduler.pyunder this same test file: 11 failed, 1 passed in 55 s, every failure ontests/bounded_read.py:62 — AssertionError: the read did not return within 5.0s. The one that passes is the regular-file control, which is what tells this guard from a reader that answers "absent" to everything — each value asserted above is also what a missing subject answers.The three persistence tests in the module (
test_task_handler_task_runs_persist_across_restart,test_task_handler_task_runs_are_never_truncated,test_evolution_cycle_log_work_not_truncated) caught a real defect in an earlier draft of the helper:non_regular_kindstats its subject, so an append to a file that did not exist yet raisedFileNotFoundErrorand the writer silently declined to write. TheFileNotFoundErrorbranch above is that fix, and those tests are why it is there.CI: pending.