Skip to content

Education and Plugins still carry all three --resolve defects, and no guard compares a vendored gen-manifests.py to the canonical #4777

Closed

Description

@rbuergi

#4775 fixes three ways gen-manifests.py --resolve reported ✓ … the merge can be committed over a question it had not answered, and the four byte-identical satellite copies are re-copied with it (Reinsurance#226, Crm#131, SocialMedia#209, Manufacturing#104).

Two copies are left carrying all three, and neither can be re-copied. Measured 2026-09-19 on origin/main:

repo md5 lines config? still = (git(...) or "") suffix classifier --exclude-standard
core canonical (before #4775) 6cc07aa32f23… 1404 n/a yes 3× yes
Reinsurance / Crm / SocialMedia / Manufacturing 6cc07aa32f23… 1404 yes yes 3× yes
MeshWeaver.Education 1c35825e36b1… 963 no yes (:700) 3× yes
MeshWeaver.Plugins a0b4fa03603a… 1215 yes yes (:856) 3× yes

All three defects, same shapes, different line numbers. So the blast radius was six repositories, not five, and #4775 closes four of them.

Why a re-copy does not work for these two

Education has no scripts/gen-manifests.config.json — it predates the centralization and still carries a module-level SKIP. The canonical refuses outright rather than guessing:

✗ gen-manifests: …/scripts/gen-manifests.config.json not found. This is the PLATFORM's canonical
script and it does not guess which top-level directories are packages…

That refusal is correct (self-test case 5 exists to keep it), so adopting the canonical here means declaring the config first, keeping it equal to validate-repos.py's SKIP. Education's validate-repos.py also enumerates through is_course_dir(d) rather than a skip set, so the equality that check-skip-sets.py enforces elsewhere has to be established, not assumed.

Plugins has the config, but its vintage loads project-closure.py eagerly at module import (_CLOSURE_SPEC.loader.exec_module at module scope, :73), while the canonical loads it lazily and only for a repo that declares hashModuleSources: true. That is a behaviour difference, not a format one.

The good news for Plugins: adoption is version-neutral

The load-bearing risk in any adoption is that a different hashing surface moves a moduleVersion, which is a release event (#878: a version that does not move is built, shelved and fetched by nobody). Measured by loading both modules against Plugins' real main tree:

Plugins: old enumerates 61, canonical enumerates 60, same=False
  only-old=['.agents']   only-new=[]
  moduleVersion mismatches: 0 of 61

Zero of 61 versions move. The single enumeration difference is .agents — an untracked dot-directory present on disk and tracked by nothing — which is exactly the case the canonical's dot rule was added for: the older vintage would demand a manifest.lock for a scratch directory. So the port is a code reconciliation, not a release.

Nothing guards these copies

check-resolver-copy.py compares each repo's vendored scripts/resolve-platform.py against the canonical (AST, docstrings stripped, RED_FROM 2026-09-15) — and its own header names gen-manifests.py as the precedent for why: "The last time one script was vendored per repo the six copies drifted to five vintages and each fix landed in one of them (gen-manifests.py, #1426)."

The precedent still has no guard. Three vintages today; this fix reduces it to three (canonical+4, Education, Plugins) only because four were byte-identical by luck, and nothing keeps them that way.

A second subject for check-resolver-copy.py — it already takes --canonical as a parameter and compares ASTs, so COPY_PATH becoming a list is most of the work — would turn "did the fix reach every copy?" into a question CI answers instead of one a person has to remember to ask.

What would close this

  1. Port the three fixes into Plugins (version-neutral, measured above) — or adopt the canonical after reconciling the eager/lazy closure load.
  2. Declare Education's config, keep it equal to its validate-repos.py enumeration, then adopt the canonical.
  3. Extend check-resolver-copy.py to scripts/gen-manifests.py, advisory-then-red on a dated flip, as it did for the resolver.

Until (3), a fix to the canonical reaching every copy is a manual sweep, and this issue is the evidence that manual sweeps are how a fleet gets to five vintages.

Activity

  1. added
    choreCI, build, dependencies, cleanup - no user-visible behaviour change
    area:buildCore subsystem: the build - manifests, bundles, bakes
    on Sep 20, 2026
  2. rbuergi commented on Sep 21, 2026

    @rbuergi
    ContributorAuthor

    Triage 2026-09-21: left open, chore, no severity

    Blocked on two different things, and they are not the same kind of blocked:

    1. Education and Plugins each need a bespoke change, not a re-copy — Education has no scripts/gen-manifests.config.json (it predates the centralization and still carries a module-level SKIP, and the canonical refuses rather than guessing), and Plugins' copy has diverged to 1215 lines against the canonical's 1404. Each is a per-repo port, in a repo core cannot commit to.
    2. The guard this issue asks for — "no guard compares a vendored gen-manifests.py to the canonical" — is core work and is not blocked on anything but someone doing it. That half is the durable one: the four byte-identical satellites were re-copied with --resolve must not report a merge it could not check #4775 and could silently diverge again tomorrow, and the precedent is already in the tree (check-node-test-launcher.py reds a vendored run-node-tests.py; check-resolver-copy.py exists for the same class of question).

    The measurement stands as written — all three --resolve defects present in both remaining copies, at different line numbers (Education :700, Plugins :856), so the blast radius was six repositories and #4775 closed four.

    🚨 One caution for whoever writes the guard, because it is the trap this issue is itself an instance of: a guard that compares declarations rather than effective behaviour passes having checked nothing. That is exactly #4774, where check-skip-sets.py compares validate-repos.SKIP == gen-manifests.skip_set(root) — equal declared sets, different effective enumerations, green. A gen-manifests.py copy guard should therefore compare the file (a hash, or a shrink-only divergence ratchet naming what differs), not a list of features it is supposed to have.

  3. rbuergi commented on Sep 22, 2026

    @rbuergi
    ContributorAuthor

    Item 3 is #5195; items 1 and 2 are satellite acts, re-measured and named

    Re-measured today on every satellite's origin/main (never a working tree): the table holds in direction as well as distance. Education (cf0d195) and Plugins (bdfe4ce) carry all three defects at the quoted lines; the four re-copied are byte-identical to the canonical at the #4775 merge and three canonical commits behind today's (138 code lines each). And check-resolver-copy.py compares nothing today — scripts/resolve-platform.py is absent on all six mains — so the "no guard" claim holds.

    #5195 (core, item 3): the guard gains a subject table; --subject gen-manifests compares the vendored copy to the canonical the lane already fetches at its scripts ref, with the same function-level report. Advisory by declaration, no flip date — and that is the decision, stated so it is not re-derived: the resolver's dated flip was safe because a resolver copy is deletable; node-repo-resolve-locks.yml refuses a repo without scripts/gen-manifests.py and runs THAT copy for --resolve, so a red on drift is a red on every canonical change (four in the three days after #4775) for a file nobody may delete. The flip's precondition is the resolve lane resolving with the platform's canonical, after which a copy is optional and the subject gets its red_from. --red-from is refused for the subject so a workflow argument cannot schedule it by accident. What is red on every run: an unreadable canonical, an unparsable copy. A pin older than the change prints NOT compared by name, never a silent skip. On its next validate run each satellite will print: Education 706 code line(s) differ, Plugins 507 (incl. copy LACKS _unmerged_paths), the four 138.

    Handed back:

    • MeshWeaver.Plugins (item 1): port the three fixes into scripts/gen-manifests.py (:856 still = (git(...) or ""); :781/:782/:878 suffix classifier; :889 --exclude-standard with no --ignored companion) or adopt the canonical after reconciling the eager project-closure.py load at :70-73. Version-neutral per the issue's own measurement (0 of 61 moduleVersions move). Note the copy also carries a fix the canonical lacks (addf16f45, "must not fail on a lock the merge DELETED") — a distance guard cannot see that direction, which is why the report names functions.
    • MeshWeaver.Education (item 2): declare scripts/gen-manifests.config.json equal to validate-repos.py's enumeration, flip centralized-gen-manifests: true, adopt the canonical.
    • Core, after both: the resolve lane fetches its own --resolve; then the subject's flip.
  4. added
    bugSomething isn't working
    sev:MMedium - secondary path broken, or easy workaround
    and removed
    choreCI, build, dependencies, cleanup - no user-visible behaviour change
    on Oct 9, 2026
  5. rbuergi commented on Oct 9, 2026

    @rbuergi
    ContributorAuthor

    Triage (severity backfill per Doc/Architecture/IssueTaxonomy): relabelled chore → bug, sev:M — the Education and Plugins copies of gen-manifests.py --resolve still print ✓ … the merge can be committed over a question they did not answer (re-measured 2026-09-22 in the thread): a false pass in developer tooling. M, not H: developer-facing, with the canonical script as the workaround.

  6. rbuergi commented on Oct 9, 2026

    @rbuergi
    ContributorAuthor

    Closing. Both remaining copies are fixed on their main, and the guard exists.

    • MeshWeaver.Education. Systemorph/MeshWeaver.Education#390 merged as e2c9070ba1. It ports all three --resolve must not report a merge it could not check #4775 fixes into this repo's own copy, the one node-repo-resolve-locks.yml runs for --resolve:

      • _unmerged_paths returns None when git cannot answer, and the closing check refuses on it;
      • locks are classified against plugin_dirs() (exactly <module>/manifest.lock);
      • ignored files that hash_files() reads block the resolve, and ignored files outside every module do not.

      hash_files() is untouched (--check ✓, no version moves). There are three new self-test cases, and each was proven to fail (exit 1, naming its assertion) with only that fix reverted.

    • MeshWeaver.Plugins. Already fixed. Its scripts/gen-manifests.py on main 0b8f875478 now has the canonical's shape: still = unmerged_paths(root), _unmerged_paths, and ls-files --others --ignored --exclude-standard beside the standard listing. It differs from core's canonical 495a150668 by 74 diff lines, all in one later canonical change: "a published tag wins over a trunk lock re-claiming its number". None of the three --resolve defects remains.

    • The guard. chore(ci): the resolver guard compares scripts/gen-manifests.py too — advisory by declaration, its flip is a change not a date (#4777) #5195 (merged 2026-09-22) compares a vendored gen-manifests.py to the canonical. It is advisory by declaration; the reason is on that PR.

    Not done here: adopting the canonical in Education. That needs scripts/gen-manifests.config.json declared equal to validate-repos.py's is_course_dir enumeration first. It is optional now that the copy is correct.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:buildCore subsystem: the build - manifests, bundles, bakesbugSomething isn't workingfeature:gen-manifestsFeature: gen manifestssev:MMedium - secondary path broken, or easy workaround

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions