Skip to content

fix(gen-manifests): hermetic self-test — fixtures never inherit the caller's PR event #5847

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
meshweaver-cloud merged 1 commit into from
Sep 28, 2026
Merged

fix(gen-manifests): hermetic self-test — fixtures never inherit the caller's PR event #5847

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 13 additions & 1 deletion .github/scripts/gen-manifests.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1627,6 +1627,10 @@ def _loose_worktree_paths(root: Path, locks: list[str]) -> list[str] | None:
return sorted(loose - conflicted)


# The run context the self-test's fixtures must never inherit (see main()'s --self-test).
_SELF_TEST_ENV_ISOLATED = ("GITHUB_EVENT_NAME", "GITHUB_EVENT_PATH", "GITHUB_BASE_REF", "MW_LOCK_BASE")


def self_test() -> int:
"""Runs --resolve's real function in throwaway repos, including the case it must REFUSE."""
import tempfile
Expand Down Expand Up @@ -2656,7 +2660,15 @@ def main() -> int:
f"verified against {'the published tags and the trunk' if trunk else 'the local tags'})")
return 0
if "--self-test" in args:
return self_test()
# 🚨 HERMETIC: the fixtures must not see the CALLER's run. `_lock_base` and
# `is_manifest_lock_settlement_pr` read GITHUB_EVENT_NAME / GITHUB_EVENT_PATH / MW_LOCK_BASE,
# so on the settlement PR's own run (MeshWeaver.Plugins#2487) every fixture took the
# settlement branch and the self-test went red over the environment, not the code.
saved = {k: os.environ.pop(k) for k in _SELF_TEST_ENV_ISOLATED if k in os.environ}

@systemorph systemorph Bot Sep 28, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question — The failing job ran in MeshWeaver.Plugins (validate / Validate node repos on the settle PR #2487), and that repo carries a vendored scripts/gen-manifests.py with the identical --self-test arm and the same env-reading functions (landed in Plugins#2481; its KNOWN_ARGS includes --self-test) — so the vendored copy has this same leak. The body calls the failing step 'the canonical self-test' and says 'Pairs-with: none', but the stated verification — running main's copy with the settle PR's exported event payload — cannot distinguish the two copies, because they are the same code: it reproduces the failure either way. Can you confirm the validate step fetches and runs this core copy (e.g. through scripts/platform-script.py, which the script's own usage block names) rather than the vendored one? If any Plugins job can reach the vendored --self-test inside a pull-request environment, this isolation needs porting there too, or the settle PR stays red.

@rbuergi rbuergi Sep 28, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed — the failing step runs THIS core copy, not the vendored one. The job was validate / Validate node repos (job 108829103360), failing step "The manifest checker can fail — --resolve, the trunk baseline and the config contract (self-test)", which is node-repo-validate.yml:554: python3 "$RUNNER_TEMP/gen-manifests.py" --self-test — the canonical fetched from Systemorph/MeshWeaver at the lane's scripts ref (main for Plugins), not scripts/gen-manifests.py.

And no Plugins job runs the vendored --self-test: git grep "gen-manifests.py --self-test" over Plugins main's .github/ finds nothing (its repo-gates job, which calls the vendored copy's check_manifests through validate-repos.py, went green on #2487).

The vendored copy still carries the same leak as dead code in CI, so I'll re-sync it to this canonical in the Plugins follow-up anyway: it is kept code-identical to the canonical, and check-resolver-copy.py reports any drift.

try:
return self_test()
finally:
os.environ.update(saved)
if "--list-packages" in args:
return list_packages(root)
if "--resolve" in args:
Expand Down
Loading