This repository was archived by the owner on Nov 21, 2026. It is now read-only.
Repository navigation
Benchmarks: the publish side renders 2026-11 rows only - #432
Merged
Merged
Conversation
tae898
force-pushed
the
publish-2611-only
branch
2 times, most recently
from
October 21, 2026 16:23
7177974 to
31660a7
Compare
…re and its end-to-end publish test (#410) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… developer's results/ (#427) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ir docs render 2026-11 rows only (#427) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… and freezes are deleted, the allowlist names the one set (#427) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…#427) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… from one suffix; stale skeleton and export_web comments go (#427) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…sion row; the fixture carries synthetic PQ rows (#427) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
tae898
force-pushed
the
publish-2611-only
branch
from
October 21, 2026 16:42
31660a7 to
aebed1e
Compare
This was referenced Oct 21, 2026
Closed
Merged
tae898
added a commit
that referenced
this pull request
Oct 21, 2026
… with a test that runs them (#433) (#434) Each failure fixed at its cause: fixture rows that no 2026-11 run could write (realistic_2611 in build_fixture.py), and gates that still encoded the retired instruments or the all-arms roster. page_check also crashed on an unset j_bad since #432. Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
This was referenced Oct 22, 2026
This was referenced Nov 20, 2026
tae898
added a commit
to humemai/mmdbbench
that referenced
this pull request
Nov 20, 2026
…rs write this repository's paths (#100) * experiments: port the 4 lines of origin PR 590 that this repository lacked (refs #25, #26) Origin: humemai/arcadedb-embedded-python@6eff4f1 (PR 590, branch docs/hygiene-bench-20261111, a draft there) PR 590 fixes drift in CAMPAIGN.md, PROTOCOL.md and PAGE-SPEC.md. CAMPAIGN.md stays in the origin, so its 13 lines are not here. The other 4 lines were absent from this tree, in 2 files, and each is true of this tree's code, checked by grep: - PROTOCOL.md, instrument sentence: export_web._instrument_of is now publish_instrument.instrument_of (publish_instrument.py:24; no _instrument_of exists) - PROTOCOL.md, tolerance sentence: page_check._check_prose is now page_check._pin_check (page_check.py, derives 0.5 * 10 ** -decimals) - PROTOCOL.md, NOT CHECKED sentence: the (0, 1) return is page_check._check_page_atomicity, when its results log is missing - PAGE-SPEC.md, the re-run paragraph: STALE_UNTIL_RERUN and LIFECYCLE_STALE are not in export_web.py, so the paragraph now says the list was removed Line numbers differ from the PR (main here moved 18 lines in PAGE-SPEC.md), so each old line was matched by its text, once, not by position. One deviation from the PR text: the PAGE-SPEC.md paragraph says "#432", a PR of the origin, and here that would read as an issue of this repository, so it is written humemai/arcadedb-embedded-python#432, as PAGE-SPEC.md already writes #396. PROVENANCE.md gets row 10. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * experiments: the stage generators write the paths of this repository (refs #92, #26) Phase D of the retirement of humemai/arcadedb-embedded-python switches the mini's harness checkout to this repository at a stage boundary. It was blocked because make_2611_stages.py wrote REPO=$HOME/repos/humemai/arcadedb-embedded-python and R=$REPO/benchmarks/experiments into every stage (HEAD, and the pair install PAIR). A stage generated after the switch would have run the old harness. - REPO is BENCH_STAGE_REPO at generation, default $HOME/repos/humemai/mmdbbench (the mini clones this repository as mmdbbench). A value that bash could read as more than a path (space, quote, semicolon, parentheses, `..`) is refused, because it is written unquoted. R is $REPO/experiments. - This repository has no bindings/python, so a stage that read the pinned wheel from $REPO/bindings/python/dist would have aborted at its first wheel guard. The wheel directory is now $BINDINGS_REPO/bindings/python/dist, the variable build_c25_wheel.sh and verify_pair_c25.sh already need, defaulting to the stage's own REPO (a worktree that holds its own dist/ keeps working). verify_pair_c25.sh gets it exported by the stage. - make_post1_rerun_stages.py follows: its REPO anchor and leak check read stage_repo(), and its wheel stage uses the same BINDINGS_REPO default, so post1 stages built from a tree still patch cleanly (its --selftest passes). - stage_drafts_2611.sha256 regenerated (every draft changes: 139 lines); test_stage_generator_2611.py drops BENCH_STAGE_REPO from the generator's environment, and adds tests: no generated stage (comparator drafts and the release chain) names the old repository or benchmarks/experiments, they pass bash -n, BENCH_STAGE_REPO is honoured, six bad values are refused. - README.md "Paths that stay as they are" no longer lists the generators; PROVENANCE.md row 11. Not changed on purpose: stage scripts that exist as records, and the BENCH_LAND_REMOTE and merge_campaign --remote defaults, which must flip on the day of the switch, not before (the stages still running from the old checkout land from there). Not run here: pytest, which the laptop guard blocks; the generator's own --check and the post1 --selftest pass, the drafts regenerate, and bash -n passes on a generated release chain. CI runs the suite. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * experiments: refuse a bad BENCH_STAGE_REPO before any stage is written; the override test checks the default path, not every citation (refs #92) CI of the first push failed in two new tests. The generator checked the path inside the per-file loop, so it refused only after it had written the first stage (stage_repo() is now called right after the arguments are parsed). The override test asserted that no generated stage contains humemai/mmdbbench, but the stage template cites #62 in a comment; it now asserts that the default checkout path is gone. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #427
DECISIONS #196: no page renders a 2026-10 or September row again, and DBBench publishes first with November data. The exporter and the page tooling get one mode. Based on #429 (measure side) because that PR makes
BENCH_INSTRUMENTother than 2026-11 an import error; merge order is PR-B (#431), #429, then this one, and GitHub retargets this PR to main when #429 merges.export_web.pygoes from about 10,100 to 7,600 lines: no_OCTOBER_ENV, no September or skeleton branches, no notes that describe 2026-10 rows (GAV, dense unindexed delete, sparse whole-record, releases-behind line, ts-doc OOM, top-parts refusal, pinned-heapjvm_heaptext). Newpublish_instrument.pyholds the rule once (PUBLISH_INSTRUMENT,refuse_other_instruments,results_dir()), and every freeze, exporter and gate refuses a row of another instrument with a message that says so. One set of names:runs_paper.csv,web_benchmarks.json,generated/. The preview route, the October campaign and the skeleton publish are deleted, withruns_paper_oct.csv,runs_skeleton_laptop.csv,web_benchmarks*.jsonandgenerated_oct/.What 2026-11 rows needed that the publish side lacked: the served ArcadeDB lean twin (
arcadedb_http_client_role= sensitivity) was collapsed into the headline cell by the freeze's dedupe, so the key now includes the role, the freeze keeps the twin rows, the tables and gates read headline rows, and the payload attaches the twin to the headline entry asclient_twin. The l4 plain-read columns and the idle-state, sparse warm-up, dense id index and official-driver sentences are generated from what the rows stamp.refresh_web_page.pyhas one target: the payload goes tosrc/data/dbbench-benchmarks.jsonin the humem.ai checkout and the tableId coverage check readssrc/lib/dbbench/lanes.ts. The prose-number checks against the old ArcadeDB page are gone because humem.ai removed every number.land_stage.py,publish_results_asset.py,page_check.py,claims_check.pyandversion_consistency_check.pyare 2026-11 only;campaign_switch_check.pyis now a re-pin checklist.Proof is
test_publish_2611_e2e.py: it freezes and exports a small fixture (fixtures/publish_2611, real 2026-10 rows restamped 2026-11 plus the 2026-11 row fields, built bybuild_fixture.py) and checks the payload, the matched-recall dense columns, the per-phase memory column, the twin, the paper tables and the refusals.conftest.pypoints the whole suite at that fixture, so no test reads an untrackedresults/directory; the suite passes from a clean checkout (960 passed, 24 skipped, 1 failed:test_stage_generator_2611.py::test_check_refuses_another_instrument_in_the_environmentasserts the word "mix" in a message that #429 changed; it belongs to the stage-generator PR).Docs rewritten to the one mode: PUBLISHING.md, PAGE-SPEC.md, READING-RESULTS.md, the publish rows of CAMPAIGN.md and the bindings benchmark pages (strict mkdocs build passes).
🤖 Generated with Claude Code