feat(deploy): make the plugin catalog configurable from the chart, and skill the deployment ramp-up

.claude/skills/new-deployment/SKILL.md

@@ -0,0 +1,137 @@
+---
+name: new-deployment
+description: Ramp up a NEW MeshWeaver portal deployment (its own domain, database, sign-in, plugins) on the shared AKS cluster, and register it so it is tracked. Use when standing up a new customer/internal portal, adding a namespace to the cluster, or auditing whether an existing deployment is wired correctly (self-update, plugins, TLS, update policy). Covers the order that matters (federated identity BEFORE deploy, DNS BEFORE TLS), the silent-failure traps (federated-credential subject mismatch, un-templated chart keys, non-idempotent portal-patch, CSI envFrom order), and where the deployment inventory lives — the PRIVATE Systemorph/Memex repo, never the public MeshWeaver repo.
+user-invocable: true
+allowed-tools:
+  - Bash
+  - Read
+  - Edit
+  - Grep
+---
+
+# /new-deployment — stand up a new portal deployment
+
+A deployment is **one namespace** on the shared AKS cluster with its own domain, database and
+sign-in. Everything else — cluster, ingress, Postgres server, Key Vault, ACR, observability — already
+exists and is shared.
+
+> 🔒 **The inventory is private.** Customer domains, namespaces and database names go in
+> [`Systemorph/Memex`](https://github.com/Systemorph/Memex) → `docs/deployments.md` and its
+> **Deployments** tab. The public MeshWeaver repo carries the *mechanism* (chart, scripts, generic
+> docs) and must never carry *who runs what*. Deployment records were deleted from the public repo on
+> 2026-08-06 for this reason — don't recreate them there.
+
+## Ground truth first
+
+Never write a runbook step from memory — read the live cluster:
+
+```bash
+# What exists, and what each namespace actually runs
+az aks command invoke -g memex-aks-rg -n memexaks-cluster --command \
+  "for ns in memex memex-cloud atioz; do echo -n \"\$ns|\"; \
+   kubectl -n \$ns get deploy memex-portal-deployment -o jsonpath='{.spec.template.spec.containers[0].image}'; \
+   echo -n '|'; kubectl -n \$ns get ingress -o jsonpath='{range .items[*]}{range .spec.rules[*]}{.host} {end}{end}'; echo; done" \
+  --query logs -o tsv
+```
+
+The cluster is **private** — every `kubectl` goes through `az aks command invoke`. There is no direct
+kubeconfig path without the P2S VPN.
+
+## The order that matters
+
+Two orderings are not negotiable, and both fail *silently* when you get them wrong:
+
+1. **Federated workload identity BEFORE the first deploy.** Subject must be exactly
+   `system:serviceaccount:<env>:memex-portal-sa`. A mismatch does **not** error — the in-pod
+   Deployment patch still works, only ACR *tag discovery* is blocked, so the deployment simply never
+   self-updates and looks fine.
+2. **DNS BEFORE TLS.** cert-manager uses an HTTP-01 challenge; without a publicly resolving A-record
+   it fails and retries with backoff that gets slow enough to look like a hang.
+
+## The steps
+
+Full runbook with commands: **`docs/new-deployment.md` in `Systemorph/Memex`** (private). Generic
+mechanism: [OnboardingNewEnvironment.md](../../../src/MeshWeaver.Documentation/Data/Architecture/OnboardingNewEnvironment.md)
+and [DEPLOY-RUNBOOK.md](../../../deploy/aks/DEPLOY-RUNBOOK.md).
+
+| # | Step | Gets it wrong how |
+|---|---|---|
+| 1 | Namespace → `portalNamespaces` in `deploy/aks/infra/main.bicep`, re-run infra deploy | subject mismatch → never self-updates |
+| 2 | `az postgres flexible-server db create -d <env>` on `memexaks-pg` | migrating? **reuse the source master key** or every `enc:` provider key is undecryptable |
+| 3 | DNS A-record → the **shared** ingress IP | pointing at a per-env IP that doesn't exist |
+| 4 | Entra app (multi-tenant) + Key Vault secrets `<env>-*` | invitation-only is the gate, not the audience |
+| 5 | `deploy/aks/envs/<env>/` scaffold + `deploy.sh` | see traps below |
+| 6 | **Plugins** → `/plugins` skill | new deployment starts with none |
+| 7 | `tls.sh` | needs step 3 resolving publicly |
+| 8 | Register the GitHub Environment on `Systemorph/Memex` | otherwise it's untracked |
+| 9 | `Admin/UpdatePolicy`: Continuous internal, **Stable** for a customer | |
+
+`values.<env>.yaml` and `secretproviderclass.yaml` are **git-ignored** — they carry hosts, database
+names, Entra ids and KV references. Managed out-of-band on disk. That is deliberate.
+
+## 🚨 Silent-failure traps
+
+Every one of these presents as "it deployed fine" while something is broken.
+
+- **The chart configMap only emits keys it templates.** `deploy/helm/templates/memex-portal/config.yaml`
+  has a fixed key list; anything else in your overlay is **dropped without a warning**. Symptom: the
+  Microsoft button never renders, or the plugin catalog reads "not configured", with the values file
+  looking correct. **If you add a config key, add it to the template.** Verify what actually landed:
+  ```bash
+  helm template t deploy/helm -f <your-values>.yaml | grep -A0 '<YourKey>' || echo "DROPPED"
+  ```
+- **`kubectl set env` is not a deployment.** Config applied by hand lives only on the live
+  Deployment; the next `helm upgrade` reverts it and nothing in git says it existed. If you find
+  yourself patching env vars to make something work, the fix is a chart key, not a patch.
+- **`portal-patch.json` is not idempotent and replaces volumes BY INDEX.** Re-applying can fail on a
+  duplicate volume add, and because the patch is atomic that rejects the *whole* patch — silently
+  dropping the CSI `envFrom`. Adding or reordering a chart volume misaligns every environment.
+- **The Key Vault (CSI) secret must be LAST in `envFrom`.** The chart's `memex-portal-secrets` has an
+  *empty* client secret; the CSI-synced one has the real value; later `envFrom` wins. Symptom:
+  `AADSTS7000218`.
+- **Never emit `""` for an int/bool key.** `Anthropic__Order: ""` fails `Int32` binding → DI throws →
+  the post-onboarding landing page dies.
+- **HTTP 200 proves nothing.** The Blazor shell returns 200 for an error page or a paywall redirect.
+  Verify rendered content, in a loop.
+- **KEDA overrides `kubectl scale`.** `minReplicaCount: 2` silently restores replicas, so the
+  documented scale-to-zero heal never runs. `kubectl get scaledobject -n <env>` first.
+
+## Verifying a fresh deployment
+
+A cold deployment recompiles **every** dynamic NodeType. Expect a window of *"No response received …
+`SubscribeRequest`"*. That is cold-compile, not a bad image.
+
+- **Do not cycle pods while it warms** — that restarts the work from cold.
+- A pod sitting `0/1` during a bake is the readiness gate doing its job.
+- A *steady* ~50% error ratio across many minutes is one failing replica, not warm-up. Warm-up
+  converges toward 100%.
+
+```bash
+NEW=$(kubectl -n <env> get pods -l app.kubernetes.io/component=memex-portal \
+  --sort-by=.metadata.creationTimestamp -o jsonpath='{.items[-1:].metadata.name}')
+kubectl -n <env> logs "$NEW" | grep "DynamicTypePreWarmer: warm-up complete"
+```
+
+## Register it
+
+```bash
+gh api -X PUT repos/Systemorph/Memex/environments/<env>
+jq -n '{ref:"main", environment:"<env>", task:"deploy", auto_merge:false, required_contexts:[],
+        production_environment:true, description:"<purpose>",
+        payload:{namespace:"<env>", cluster:"memexaks-cluster", database:"<env>",
+                 image:"meshweaver.azurecr.io/memex-portal-ai:<tag>"}}' \
+  | gh api -X POST repos/Systemorph/Memex/deployments --input -
+# then a status carrying the live URL
+```
+
+`required_contexts` must be a real JSON array — `-F required_contexts='[]'` is rejected with a 422,
+which is why the body goes through `jq` and `--input -`.
+
+Then add the row to `docs/deployments.md` in that repo. A deployment nobody recorded is a deployment
+nobody will remember to patch.
+
+## Related
+
+- `/plugins` — wire the deployment to the plugin registry
+- `/release` — how images get built and rolled
+- `/delete-user` · `/storm` · `/debug` — operating a live deployment

.claude/skills/plugins/SKILL.md

@@ -0,0 +1,161 @@
+---
+name: plugins
+description: Wire a MeshWeaver deployment to the plugin catalog — as a CONSUMER (installs plugins from a registry over HTTP, no git credential) or as the REGISTRY (holds the git credential, serves private plugin repos at /api/plugins). Use when a deployment's Plugin Catalog says "not configured", when adding a new plugin source repo, when issuing an install its registry token, when a plugin won't appear or won't install, or when auditing who can read the catalog. Covers the credential-encapsulation model, the Helm keys (and the trap that un-templated keys are silently dropped), token issuance, and the anonymous-registry exposure that a missing token list causes.
+user-invocable: true
+allowed-tools:
+  - Bash
+  - Read
+  - Edit
+  - Grep
+---
+
+# /plugins — the plugin catalog: registry and consumers
+
+Plugins are folders of mesh nodes in **private** git repos. The point of the design is that **only
+one install holds a git credential**.
+
+```
+Systemorph/MeshWeaver.Plugins ─┐
+                               ├─→ REGISTRY (memex-cloud)  ── GET /api/plugins ──→ CONSUMER
+Systemorph/education ──────────┘   holds the ONE credential   POST /api/plugins/files   no git access
+```
+
+npm/NuGet-style: the registry has source access, clients just speak HTTP. Model:
+[PluginRegistry.md](../../../src/MeshWeaver.Documentation/Data/Architecture/PluginRegistry.md) ·
+[Plugins.md](../../../src/MeshWeaver.Documentation/Data/Architecture/Plugins.md).
+
+## Which am I configuring?
+
+| | Registry | Consumer |
+|---|---|---|
+| Holds git credential | ✅ | ❌ never |
+| Helm values | `pluginCatalog.sources` | `pluginCatalog.registryUrl` (or `.registries`) |
+| Secret | `PluginCatalog__RegistryTokens` (the list it **accepts**) | `PluginCatalog__RegistryToken` (the one it **sends**) |
+| Today | `memex-cloud` | every other deployment |
+
+A registry is also a consumer of itself — `memex-cloud` sets both.
+
+## Consumer wiring
+
+```yaml
+pluginCatalog:
+  registryUrl: "https://memex.meshweaver.cloud"
+config:
+  memex_portal:
+    # Systemorph-operated deployments track their plugin repos continuously: a fresh install
+    # record is seeded opted-in, so a plugin repo's green build lands with nobody clicking Update.
+    # Install-time SEED only — flipping it later changes nothing for already-installed packages.
+    PluginCatalog__AutoUpdateByDefault: "true"
+secrets:
+  memex_portal:
+    PluginCatalog__RegistryToken: "<the token this install was issued>"
+```
+
+Several registries instead of one:
+
+```yaml
+pluginCatalog:
+  registries:
+    - {name: Plugins,   url: "https://memex.meshweaver.cloud"}
+    - {name: Education, url: "https://<other-registry>"}
+```
+
+## Registry wiring
+
+```yaml
+pluginCatalog:
+  sources:
+    - {name: Plugins,   repoPath: "https://github.com/Systemorph/MeshWeaver.Plugins", ref: main}
+    - {name: Education, repoPath: "https://github.com/Systemorph/education",          ref: main}
+secrets:
+  memex_portal:
+    PluginCatalog__RegistryTokens: ["<token-for-install-a>", "<token-for-install-b>"]
+```
+
+`format` defaults to `node-repo` — a folder appears once it carries a `<Folder>/index.json` **Space**
+root with a `PluginManifest`. `package-json` is the alternative layout. Sources merge in configured
+order; on an id collision the first wins; a failing repo contributes nothing (logged) rather than
+breaking the catalog.
+
+## 🚨 A registry with no tokens serves everyone
+
+`PluginCatalog:RegistryTokens` empty ⇒ the registry answers **anonymously**. That is the local-dev /
+e2e stub mode. On a production registry it means the full catalog **and every package's file
+content** is readable by anyone who knows the URL.
+
+**Audit it — this is a one-line check, run it:**
+
+```bash
+curl -sS https://<registry-host>/api/plugins -o /dev/null -w "%{http_code}\n"   # want 401
+curl -sS -X POST https://<registry-host>/api/plugins/files \
+  -H 'Content-Type: application/json' -d '{"id":"<some-id>"}' -o /dev/null -w "%{http_code}\n"  # want 401
+```
+
+`200` unauthenticated = every private plugin repo behind that registry is public to anyone with the
+URL. Issue tokens and set `RegistryTokens`.
+
+> As of 2026-08-06 `https://memex.meshweaver.cloud/api/plugins` returns **200 with 28 packages** to an
+> unauthenticated caller, including paid course content from `Systemorph/education`. Fixing it means
+> issuing each install a token, setting `RegistryTokens` on the registry, and rolling — in that order,
+> or consumers lose their catalog in the gap.
+
+Issue a token:
+
+```bash
+openssl rand -base64 32
+```
+
+Store it in Key Vault (`<env>-PluginCatalog-RegistryToken`) and surface it through the
+SecretProviderClass — never in a values file.
+
+## Installing
+
+Platform admin → **Settings ▸ Administration ▸ Plugin Catalog** → **Install**. It is an admin *tab*,
+not a browsable Space (a Space partition would correctly deny read to everyone else — that was the
+"Access denied on 'Plugins'" bug). The consumer pulls the package files over HTTP and imports them:
+
+- **Content** package → imports its folder into the target partition.
+- **Code** package → synthesizes its `NodeType` from the manifest, imports `Source/*.cs` as Code
+  children, and requests a release so the mesh compiles the type live. No app rebuild, no NuGet.
+
+Re-installing is an upsert; installing one module never disturbs another in a shared partition.
+
+## Troubleshooting
+
+**"Plugin catalog not configured"** → `EffectiveRegistries` is empty: no `registryUrl` and no
+`registries` entry with a URL reached the pod. Check what actually landed, not what the values file
+says:
+
+```bash
+kubectl -n <env> get deploy memex-portal-deployment \
+  -o jsonpath='{range .spec.template.spec.containers[0].env[*]}{.name}={.value}{"\n"}{end}' | grep -i plugin
+kubectl -n <env> get configmap memex-portal-config -o jsonpath='{.data}' | tr ',' '\n' | grep -i plugin
+```
+
+**The #1 cause: the chart only emits keys it templates.** Before `pluginCatalog` existed in
+`values.yaml`, every plugin key except `AutoUpdateByDefault` was silently dropped — which is why
+`memex-cloud`'s registry config was hand-applied as raw Deployment env vars that no redeploy
+reproduces. If a key isn't in `deploy/helm/templates/memex-portal/config.yaml`, it does not reach the
+pod. Verify by rendering:
+
+```bash
+helm template t deploy/helm -f <values>.yaml | grep PluginCatalog
+```
+
+**401 from the registry** → the consumer's token isn't in the registry's `RegistryTokens`. Both sides
+validate through `PluginRegistryTokens`, one shared contract, so a mismatch is a real value mismatch,
+not a format one.
+
+**A folder in the repo doesn't appear in the catalog** → it has no `<Folder>/index.json` Space root
+with a `PluginManifest`, or its source's `format` is wrong.
+
+**Package installs but the type doesn't work** → it's a Code package; the compile happens live. Check
+the NodeType's compile status before suspecting the install.
+
+## Related
+
+- `/new-deployment` — step 6 wires plugins into a fresh deployment
+- `Systemorph/Memex` → `docs/new-deployment.md` §6 (private, has the real values)
+- [PluginRegistry.md](../../../src/MeshWeaver.Documentation/Data/Architecture/PluginRegistry.md) ·
+  [PluginAuthoring.md](../../../src/MeshWeaver.Documentation/Data/Architecture/PluginAuthoring.md) ·
+  [PluginUpdateOnGreenBuild.md](../../../src/MeshWeaver.Documentation/Data/Architecture/PluginUpdateOnGreenBuild.md)

.gitignore

@@ -410,4 +410,7 @@ memex/aspire/Memex.Portal.Distributed/wwwroot/app/
 
 # The plugins/ dir is a SEPARATE git repo (Systemorph/MeshWeaver.Plugins) nested locally —
 # never tracked here. Content ships via the plugin registry / GitSync, not this repo.
-plugins/
+# 🚨 ANCHORED (leading slash) on purpose: an unanchored `plugins/` matches at EVERY depth, and on
+# macOS's case-insensitive filesystem it also matches `Plugins/` — which silently dropped every NEW
+# file under src/MeshWeaver.AI/Plugins/ and .claude/skills/plugins/. Keep the slash.
+/plugins/

deploy/helm/templates/memex-portal/config.yaml

@@ -219,6 +219,25 @@ data:
   # seed only; a record's own AutoUpdate flag stays the runtime authority thereafter.
   # See Doc/Architecture/PluginUpdateOnGreenBuild.md.
   PluginCatalog__AutoUpdateByDefault: "{{ .Values.config.memex_portal.PluginCatalog__AutoUpdateByDefault | default "true" }}"
+  # ---- Plugin catalog: which registry this install CONSUMES, and (on a registry install) which
+  # git repos it SERVES. Both were previously un-templated, so an env overlay that set them was
+  # silently dropped — the catalog then reads "not configured" with the values file looking right,
+  # and the only way to wire it was a hand-applied `kubectl set env` that no redeploy reproduces.
+  # Tokens are secrets and live in secrets.yaml / the Key Vault CSI mount, never here.
+  # See Doc/Architecture/PluginRegistry.md.
+  PluginCatalog__RegistryUrl: "{{ .Values.pluginCatalog.registryUrl | default "" }}"
+  PluginCatalog__RegistryRef: "{{ .Values.pluginCatalog.registryRef | default "HEAD" }}"
+  {{- range $i, $r := .Values.pluginCatalog.registries }}
+  PluginCatalog__Registries__{{ $i }}__Name: "{{ $r.name }}"
+  PluginCatalog__Registries__{{ $i }}__Url: "{{ $r.url }}"
+  PluginCatalog__Registries__{{ $i }}__Ref: "{{ $r.ref | default "HEAD" }}"
+  {{- end }}
+  {{- range $i, $s := .Values.pluginCatalog.sources }}
+  PluginCatalog__Sources__{{ $i }}__Name: "{{ $s.name }}"
+  PluginCatalog__Sources__{{ $i }}__RepoPath: "{{ $s.repoPath }}"
+  PluginCatalog__Sources__{{ $i }}__Ref: "{{ $s.ref | default "main" }}"
+  PluginCatalog__Sources__{{ $i }}__Format: "{{ $s.format | default "node-repo" }}"
+  {{- end }}
   # ---- System email (Microsoft Graph) — BIDIRECTIONAL. Outbound invitations + notifications
   # (/sendMail, Mail.Send app permission) AND optional inbound email→agent channel (inbox
   # change-notification subscription, Mail.ReadWrite app permission). Non-secret here;

deploy/helm/templates/memex-portal/secrets.yaml

@@ -51,4 +51,16 @@ stringData:
   {{- if .Values.secrets.memex_portal.Authentication__Apple__PrivateKey }}
   Authentication__Apple__PrivateKey: "{{ .Values.secrets.memex_portal.Authentication__Apple__PrivateKey }}"
   {{- end }}
+  # ---- Plugin catalog tokens ------------------------------------------------
+  # CONSUMER: the instance token this install was issued when it registered with the registry,
+  # sent as `Authorization: Bearer`. REGISTRY: the list of tokens it accepts — one per registered
+  # install. 🚨 A registry with an EMPTY token list answers ANY anonymous caller with the full
+  # catalog and every package's file content. Configure tokens on every production registry.
+  # See Doc/Architecture/PluginRegistry.md.
+  {{- if .Values.secrets.memex_portal.PluginCatalog__RegistryToken }}
+  PluginCatalog__RegistryToken: "{{ .Values.secrets.memex_portal.PluginCatalog__RegistryToken }}"
+  {{- end }}
+  {{- range $i, $t := .Values.secrets.memex_portal.PluginCatalog__RegistryTokens }}
+  PluginCatalog__RegistryTokens__{{ $i }}: "{{ $t }}"
+  {{- end }}
 type: "Opaque"