You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository was archived by the owner on Nov 21, 2026. It is now read-only.
Repository navigation
This repository was archived by the owner on Nov 21, 2026. It is now read-only.
benchmarks: tooling for the post1 rerun of the paper's embedded cells (build-only mode, wheel label, one wheel per cell, embedded-only stages) #583
The wheels-only 26.11.1.post1 is going to be rerun on the mini for the paper cells it can move, and the post1 rows override the 26.11.1 rows of those cells, with all repetitions of a cell from one wheel (DECISIONS #297, #298). Row 93 of CAMPAIGN section 7 (#581, PR #582) is the plan. What the harness cannot do today is carry out that plan: there is no way to stop a cell after build(), rows do not say which wheel file they ran on, the generated stages mix embedded and served arms and have no cell selection, and nothing refuses a cell that would end up with repetitions from two wheels.
I am adding the tooling, all of it additive and off by default, so nothing of the live n=3 pass changes: a build-only mode (runner.py --build-only, a four-line hook in each of the four lanes, rows marked row_kind = "build_only" and refused anywhere but a _post<N>.jsonl results file), a wheel label on the rows of a stage that opts in (wheel_label, wheel_name, wheel_sha256), a per-cell choice of one wheel at landing (land_stage.py --post1-reps N, nonzero exit on a cell that would mix wheels or is short of repetitions), an embedded-only stage generator with a cost estimate and a selftest, a short procedure, and a rollback by wheel label.
One finding on the way: engine_pin.verify compares the engine jar version with the stage's pin, and the post1 pin is 26.11.1.post1 while the jar of a post1 wheel is 26.11.1, so the runner would refuse every post1 cell. The same file is changed by #554, so that fix is a two-line change that I will apply to whichever of the two lands second.
I am not running anything: the mini stays untouched until the live pass has ended and I have the go. The PR is a draft.
Done in humemai/mmdbbench: the tooling is on main, ported from PR #584 (commit 682659f). It is runner.py --build-only, the wheel label on rows (post1_rows.py), land_stage.py --post1-reps N, make_post1_rerun_stages.py, the procedure in experiments/POST1-RERUN.md, and test_post1_rerun.py; engine_pin.py strips .postN when it compares a pin with the jar version. Which cells are rerun is still the plan of CAMPAIGN row 94. I am closing the issue here because this repository is being archived.
Activity
tae898 commentedon Nov 20, 2026
Done in humemai/mmdbbench: the tooling is on
main, ported from PR #584 (commit682659f). It isrunner.py --build-only, the wheel label on rows (post1_rows.py),land_stage.py --post1-reps N,make_post1_rerun_stages.py, the procedure inexperiments/POST1-RERUN.md, andtest_post1_rerun.py;engine_pin.pystrips.postNwhen it compares a pin with the jar version. Which cells are rerun is still the plan of CAMPAIGN row 94. I am closing the issue here because this repository is being archived.