Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .claude/skills/new-deployment/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,7 @@ Every one of these is a **prerequisite the Provision REFUSES without**, so do th
## 2. What the operator does — the Provision plan, in order

Composed by `InstanceActionPlan.ProvisionSteps` (MeshWeaver.Plugins, in-mesh — it compiles in the
portal, no CI type-checks it), run by `deploy/aks/operator/bin/run.sh` in a Job in `memex-ops`,
portal, no CI type-checks it), run by `MeshWeaver.Plugins/deploy/hosting-operator/bin/run.sh` in a Job in `memex-ops`,
fail-fast, the failing STEP named on the node:

1. `az postgres flexible-server db create` — control plane, workload identity. (Memex#132 —
Expand Down
26 changes: 4 additions & 22 deletions .github/manual-keyvault.allow
Original file line number Diff line number Diff line change
Expand Up @@ -7,28 +7,10 @@
# 🚨 "TRANSITIONAL" marks a governed verb that still READS a value (or checks existence with a value
# read). Migration step 3 in the design page rewrites each so the writer identity never needs `get`;
# the line goes when that rewrite lands.

# The writer's own verbs: set with tags, status from metadata, lifecycle. None reads a value.
deploy/aks/operator/bin/hosting-kv-set — writer verb: writes through --file, confirms by list-versions. TRANSITIONAL (policy secret-actions-interim-actions-lane): its --generate mode also runs as hosting-operator on the aks-ops lane until the writer identity is provisioned and every secret action runs in the in-cluster Job under it
deploy/aks/operator/bin/hosting-kv-state — writer verb: set-attributes / delete / recover / purge, no value
deploy/aks/operator/bin/hosting-kv-purge — teardown verb: deletes an instance's objects by prefix, no value read

# Governed provisioning verbs that still run as hosting-operator.
deploy/aks/operator/bin/_common.sh — TRANSITIONAL: hosting::pg_password reads the server admin password to compose connection strings
deploy/aks/operator/bin/hosting-kv-ensure — TRANSITIONAL: the admin-password read for the connection string (existence is checked with `list`), and the download of an existing helm-values-<release> to filter it to the record's declared keys (policy one-values-half-per-release)
deploy/aks/operator/bin/hosting-kv-values-set — TRANSITIONAL: the governed key-level writer of helm-values-<release> downloads the half to change named declared keys and keep the rest (a read-modify-write); retired with the vault half
deploy/aks/operator/bin/hosting-kv-copy — TRANSITIONAL: copyFrom reads the source value to write the per-prefix copy
deploy/aks/operator/bin/hosting-kv-rotate — TRANSITIONAL: reads the vaulted registry key to compare before rotating
deploy/aks/operator/bin/hosting-registry-register — TRANSITIONAL: reads the held instance key and the bootstrap key to call the registry
deploy/aks/operator/bin/hosting-registry-coupon — TRANSITIONAL: reads the registry's coupon-mint credential to call the registry; writes the minted coupon through --file, existence checked with `list`
deploy/aks/operator/bin/hosting-signin-app — writes the minted Entra client secret through --file; existence is checked with `list`, no value read
deploy/aks/operator/bin/hosting-image-mirror — TRANSITIONAL: reads the registry publisher password to push the mirror
deploy/aks/operator/bin/hosting-pull-secret — TRANSITIONAL: reads the registry pull key to render the image pull Secret
deploy/aks/operator/bin/hosting-deploy — TRANSITIONAL: downloads the helm values half (helm-values-<release>); retired with the vault half

# Their behaviour suite and its stand-in binaries (they answer the commands, they do not run them).
deploy/aks/operator/test/run-tests.sh — asserts the verbs write through --file, never print a value and never read one back
deploy/aks/operator/test/stubs/*/az — stand-in az binaries for that suite
#
# The hosting operator's governed `hosting-kv-*` verbs moved to Systemorph/MeshWeaver.Plugins with the
# operator ("core slimming"); their entries live in its deploy/hosting-operator/manual-keyvault.allow,
# and its hosting-operator.yml runs this gate over that directory.

# This gate, which has to spell the commands it refuses.
.github/scripts/check-manual-keyvault.py — the gate itself
5 changes: 3 additions & 2 deletions .github/release-channels.json
Original file line number Diff line number Diff line change
Expand Up @@ -321,14 +321,15 @@
},
{
"workflow": "hosting-operator.yml",
"job": "publish",
"repo": "Systemorph/MeshWeaver.Plugins",
"job": "operator",
"repositories": [
"hosting-operator"
],
"registries": [
"meshweaver.azurecr.io"
],
"state": "every merge that touches the operator image"
"state": "every merge there that touches the operator, and hourly when core's chart or drift scripts moved (the operator moved to MeshWeaver.Plugins)"
}
]
},
Expand Down
4 changes: 2 additions & 2 deletions .github/scripts/check-workflow-shell.py
Original file line number Diff line number Diff line change
Expand Up @@ -64,8 +64,8 @@
WORKFLOW_GLOBS = [".github/workflows/*.yml", ".github/workflows/*.yaml"]
SHELL_GLOBS = [".github/scripts/*.sh"]

# shellcheck severity. Matches the level the operator-scripts gate already runs at
# (hosting-operator.yml), so one repo does not carry two different bars for the same tool.
# shellcheck severity. Matches the level memex-local.yml runs at (and the operator scripts' gate, which
# moved to MeshWeaver.Plugins), so one repo does not carry two different bars for the same tool.
SHELLCHECK_SEVERITY = "warning"

# A `${{ … }}` expression is not shell. Substituting a LITERAL for it makes shellcheck
Expand Down
6 changes: 3 additions & 3 deletions .github/scripts/lock-pinned-digests.py
Original file line number Diff line number Diff line change
Expand Up @@ -80,8 +80,8 @@
Axis 2 is a different subject with a different shape (a TAG, in a helm overlay, not a digest in a
workflow) and it necessarily has its own reader. Its scope is deliberately narrow, and the boundary
is measured rather than guessed: `values*.yml` / `values*.yaml` under a `deploy/` or `deployments/`
path segment. Widening it to every YAML/JSON under `deploy/` was tried and rejected — core's
`deploy/aks/operator/test/fixtures/**` carry image references to tags that never existed
path segment. Widening it to every YAML/JSON under `deploy/` was tried and rejected — the hosting
operator's `deploy/hosting-operator/test/fixtures/**` (MeshWeaver.Plugins; core's `deploy/aks/operator` until it moved) carry image references to tags that never existed
(`memex-portal-ai:3.0.0-rc8.ci.5000`), which would red this job nightly over test fixtures, and
`deploy/aks/manifests/observability/log-watcher.yaml` names the FLOATING tag `:latest`, which has
no fixed manifest to protect.
Expand Down Expand Up @@ -5490,7 +5490,7 @@ def check(condition: bool, message: str) -> None:
"ARM 13: Memex's real overlay path was rejected")
check(is_overlay_path("deploy/whisper/helm/values.yaml"),
"ARM 13: this repository's whisper chart values file was rejected")
check(not is_overlay_path("deploy/aks/operator/test/fixtures/audit/clean/manifest.json"),
check(not is_overlay_path("deploy/hosting-operator/test/fixtures/audit/clean/manifest.json"),
"ARM 13: a TEST FIXTURE was accepted as a deployment overlay — those name tags that "
"never existed and would red this job nightly")
check(not is_overlay_path("deploy/aks/manifests/observability/log-watcher.yaml"),
Expand Down
30 changes: 28 additions & 2 deletions .github/scripts/release-channels.py
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,14 @@
PLACEHOLDERS = ("major", "minor", "patch")
REQUIRED_FIELDS = ("id", "tag", "edition", "meaning", "writers", "notation", "status")
BINDING_FIELDS = ("workflow", "job", "repositories")
# The repository whose workflows this file's bindings name by default. A binding may name ANOTHER
# repository's workflow with `"repo": "<owner>/<name>"` — the hosting operator's image is published
# by Systemorph/MeshWeaver.Plugins (its hosting-operator.yml), and its `main` label must stay
# declared, and so protected, here. Such a binding is shape-validated, but its workflow cannot be
# read from this tree, so `check` does not open it; that repository's lane spells the tag and the
# repository itself.
THIS_REPO = "Systemorph/MeshWeaver"
REPO_RE = re.compile(r"^[A-Za-z0-9_.-]+/[A-Za-z0-9_.-]+$")
# A label must look like a tag and must NEVER be SELECTABLE as a version. The self-updater
# (MeshWeaver.Plugins src/Memex.Portal.Shared/SelfUpdate/VersionSelect.cs) keeps every tag matching
# `PlatformVersionTag` = ^\d+\.\d+\.\d+([-+].*)?$ EXCEPT one ending in `-latest` (`MovingPointer`), so
Expand Down Expand Up @@ -186,6 +194,8 @@ def validate(doc: dict) -> list[str]:
problems.append(f"{at}: `repositories` must be a list of repository names")
if binding.get("notation", "any") not in NOTATIONS:
problems.append(f"{at}: notation {binding.get('notation')!r} is not one of {NOTATIONS}")
if "repo" in binding and not (isinstance(binding["repo"], str) and REPO_RE.match(binding["repo"])):
problems.append(f"{at}: `repo` must be '<owner>/<name>', got {binding['repo']!r}")
return problems


Expand Down Expand Up @@ -280,6 +290,8 @@ def check(doc: dict, root: Path, platform_version=None, is_floating=None) -> lis
workflows = root / ".github" / "workflows"
for entry in channels(doc):
for binding in entry["writers"]:
if binding.get("repo", THIS_REPO) != THIS_REPO:
continue # another repository's workflow: shape-validated above, not readable here
wf = binding["workflow"]
path = workflows / wf
if not path.is_file():
Expand Down Expand Up @@ -451,8 +463,22 @@ def line_pointers(version: str) -> list[str]:
# The publishers outside main-cd/release.yml are declared too (regression: they were omitted).
expect("edge is protected (edge-images.yml writes it for portal and migration)",
is_protected(real, "edge") and tags_for(real, "3.1.10050", "edge-images.yml", repository="memex-migration") == ["edge"])
expect("hosting-operator:main is declared (hosting-operator.yml#publish)",
tags_for(real, "3.1.10050", "hosting-operator.yml", job="publish", repository="hosting-operator") == ["main"])
expect("hosting-operator:main is declared (MeshWeaver.Plugins hosting-operator.yml#operator)",
tags_for(real, "3.1.10050", "hosting-operator.yml", job="operator", repository="hosting-operator") == ["main"])
expect("the hosting-operator writer names the repository that publishes it",
any(b.get("repo") == "Systemorph/MeshWeaver.Plugins" and b["workflow"] == "hosting-operator.yml"
for e in real["channels"] if e["id"] == "main" for b in e["writers"]))
# A binding into ANOTHER repository is not opened here; one into THIS repository still is.
elsewhere = json.loads(json.dumps(real))
elsewhere["channels"][3]["writers"][0].update(workflow="no-such-workflow.yml", repo="Systemorph/Elsewhere")
expect("a writer in another repository is not looked for in this tree",
not any("no-such-workflow.yml" in p for p in check(elsewhere, root, FakePV, always_floating)))
here = json.loads(json.dumps(real))
here["channels"][3]["writers"][0].update(workflow="no-such-workflow.yml", repo=THIS_REPO)
expect("a writer that names THIS repository explicitly is still looked for",
any("does not exist" in p for p in check(here, root, FakePV, always_floating)))
expect("validate refuses a malformed `repo`",
bool(broken(lambda d: d["channels"][3]["writers"][0].update(repo="not a repo"))))
expect("memex-portal-ai-base:latest is declared (base-image-acr.yml#base)",
tags_for(real, "3.1.10050", "base-image-acr.yml", job="base", repository="memex-portal-ai-base") == ["latest"])
patterns = {p["id"]: p["repositories"] for p in protected_patterns(real)}
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/homebrew.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
name: Homebrew (memex-local)

# memex-local is delivered through Homebrew — the tap Systemorph/homebrew-memex — and until this
# workflow existed nothing ever INSTALLED the formula: the Linux gate in hosting-operator.yml
# workflow existed nothing ever INSTALLED the formula: the Linux gate in memex-local.yml
# parses the CLI, shellchecks it and runs its behaviour tests against stub tools, which is exactly
# the layer where most of its defects lived, but `brew install` itself, the vendored chart layout,
# the wrapper's MEMEX_LOCAL_SHARE/MEMEX_CHART_DIR plumbing and macOS's bash 3.2 were all asserted by
Expand Down
Loading
Loading