feat(deploy/k8s): ocp-single — single-namespace, no-SCC dev/test target

[pdettori]

What

Adds an optional dev/test install mode (setup.sh --target ocp-single, smoke target ocp-single) for shared OpenShift clusters where the operator holds RBAC on exactly one namespace and cannot be granted SCC or cluster-scoped access. Refs #426.

The default install is unchanged: three namespaces + nonroot-v2 SCC (--target ocp). The new mode is dev/test only and self-contained — the sandbox pool (moca-sandbox StatefulSet, its Role/RoleBinding, the relay attach Secret) is included in the single namespace.

How the three-namespace topology collapses

New overlay deploy/k8s/overlays/ocp-single (namespace moca-single by default, SH_SINGLE_NAMESPACE to rename):

• No SCC grant: strips runAsUser/runAsGroup/fsGroup from all five workloads so the namespace's restricted-v2 SCC assigns UID/GID from the range; runAsNonRoot + seccompProfile kept.
• Writable sandbox dirs under the assigned UID: emptyDir volumes over /workspace and /home/sandbox (image-baked dirs are uid 1001, mode 775). Trade-off: writable-layer data is pod-local, lost on restart — documented in README §12.1.
• In-namespace addressing: cross-namespace env strings rewritten to single-namespace service DNS (relay, control plane, sandbox, credentials).
• Network policies by pod label: namespaceSelectors replaced with pod-label selectors inside the one namespace; DNS egress retargeted to openshift-dns on 5353 (vs kube-system/kube-dns on 53).
• Namespace deletes: the three Namespace objects and duplicated policies are removed via $patch: delete.
• P4 cannot attach in this mode (no Route is rendered) — the mode is container-sandbox only, matching its dev/test scope.

Base and #426 interaction

• Branch is based on upstream/main @ ccc17b0 (post-feat: sandbox tiers and session-to-sandbox affinity — harness, relay, workers (P6.3 PR 1, #425) #442); deploy/k8s was unchanged between the feat: P6 on Kubernetes, slice 2 — external P4 hosts over TLS (#424) #437 merge and this PR, so there is no overlay drift.
• Slice 3 (Sandbox tier selection and session-to-sandbox affinity in selectPoolSandbox (P6-on-K8s slice 3) #425, merged as feat: sandbox tiers and session-to-sandbox affinity — harness, relay, workers (P6.3 PR 1, #425) #442): the shipped tier labels (moca.dev/tier=container, SH_SANDBOX_TIERS unset = no filtering) compose with this mode — verified, no conflict. The overlay relies on slice-3 labels for the sandbox pool's pod selectors.
• Later Epic: P6 on Kubernetes — process manager, relay, Redis and control plane as K8s workloads; only P4 on a VM/bare metal #426 slices are unaffected: P4/remote-worker work (slice 2, feat: P6 on Kubernetes, slice 2 — external P4 hosts over TLS (#424) #437) targets the 3-namespace + Route path, which this overlay does not render.

Validation

• kind with simulated assigned UID 1000670000: smoke 11/11.
• Real shared OCP (ROKS) tenant under restricted-v2 with assigned uid 1002610000: smoke 11/11 with a mock model (experiment branch). On the final PR-branch run against that tenant, all infrastructure claims pass (1, 2, 5, 6, 7, 10, 11); claims 3/4/8/9 fail only for lack of a real model credential — expected per README §12.2.
• kube-API egress caveat: kubernetes.default.svc:443 is reachable from the sandbox on the shared tenant despite the overlay's ipBlock exceptions: the VIP DNATs to ROKS's node-local proxy 172.20.0.1:2040, which the sandbox reaches directly even though it is inside the excepted 172.16.0.0/12, so a cluster-level allowance (Calico; not readable with namespace rights) admits it, not the overlay. The sandbox holds no ServiceAccount token, so the API refuses it. The same re-probe found the tenant's node network (a non-RFC 1918 range) outside the except list (the sandbox reaches the node's kubelet and SSH), documented in README §12.3. Smoke downgrades this claim to a note for ocp-single; defence-in-depth trade-off documented in README §12.3.
• deploy/k8s/tests/setup.test.sh: all passed (5 new ocp-single blocks).
• deploy/k8s/tests/smoke.test.sh: all passed.
• packages/supervisor vitest k8s render suite: 66/66 (new overlay-ocp-single.test.ts, 6 tests: single-namespace collapse, UID stripping, emptyDirs, env rewrite, policy edges, sandbox-pool self-containment).

Docs

README §12 covers the target: §12.1 namespace-scoped comparison and isolation trade-offs, §12.2 smoke with a real model, §12.3 the verified kube-API VIP finding, §12.4 P4 non-attach and SH_SINGLE_NAMESPACE non-stickiness.

Refs #426 (not closed by this PR; later slices remain)

🤖 Generated with Claude Code

[pdettori]

Summary

A well-scoped ocp-single target. The overlay design checks out: the base has exactly two cross-namespace namespaceSelector policies, and patch-policies.yaml rewrites both. Every workload except the control plane keeps automountServiceAccountToken: false. Credential Secret names are hashed (sh-cred-<hash>), so they can't collide with platform Secrets. CI isn't green, and the PR has one unrelated regression in setup.sh's preflight.

• lint fails on prettier for packages/supervisor/test/k8s/render.ts, overlay-ocp-single.test.ts, deploy/k8s/overlays/ocp-single/patch-addressing.yaml and deploy/k8s/README.md. Run pnpm exec prettier --write on those four.
• The PR body says both "Refs #426" and "Closes #426", and also mentions later #426 slices. Merging will auto-close the tracking issue; switch to Refs if that isn't intended.

Author: pdettori (MEMBER — maintainer)
Areas reviewed: Shell (setup/smoke/tests), Kustomize/K8s YAML, NetworkPolicy/RBAC isolation, TypeScript tests, Docs
Agent/IDE config (.claude/.vscode): none
Commits: 1 commit, all signed-off: yes
CI status: failing (check: TS2571 typecheck; lint: prettier)

Verdict: changes requested (inline must-fix #1 and #2, plus lint). Submitted as COMMENT because this is a self-review.

Assisted-By: Claude Code

[pdettori]

must-fix — check fails on TS2571: Object is of type 'unknown' here and at L106. envVar() (render.ts) types valueFrom as unknown, so you can't chain .fieldRef/.secretKeyRef off it. Vitest passes because it doesn't type-check; tsc --noEmit does. Suggested:

expect(envVar(container(ss, 'sandbox'), 'SANDBOX_ID')).toMatchObject({
  valueFrom: { fieldRef: { fieldPath: 'metadata.name' } },
});
expect(envVar(container(ss, 'sandbox'), 'SANDBOX_TOKEN')).toMatchObject({
  valueFrom: { secretKeyRef: { name: 'moca-relay-attach', key: 'SH_RELAY_TOKEN' } },
});

[pdettori]

Fixed in be7496e (casts); e338f59 switches to the suggested toMatchObject form. tsc --noEmit clean, k8s render suite 66/66.

[pdettori]

must-fix — This hunk deletes command -v sha256sum ... || command -v shasum ... || missing="$missing sha256sum|shasum" from preflight for every target, and nothing in the ocp-single scope needs that. sha256() (L494) still needs one of the two. Without the check, a host that has neither fails at write_settings under pipefail, after the Secrets are already applied, with no clear message. It looks accidental; please restore the line.

[pdettori]

Restored in e338f59. It was accidental, and the check runs for every target again.

[pdettori]

suggestion — kc get configmap moca-setup has no -n, so it reads from the kubeconfig's current namespace. You need the namespace to find the ConfigMap that is supposed to tell you the namespace. If the context isn't set to the tenant namespace, smoke quietly falls back to moca-single; with two stacks (README §12.4 allows that) it can smoke the wrong one. Suggest honoring SH_SINGLE_NAMESPACE first:

NS="${SH_SINGLE_NAMESPACE:-$(kc get configmap moca-setup -o jsonpath='{.data.SH_SINGLE_NAMESPACE}' 2>/dev/null || true)}"

[pdettori]

Done in e338f59: an exported SH_SINGLE_NAMESPACE now wins, and the ConfigMap read is only the fallback. The comment says the fallback reads the kubeconfig's current namespace.

[pdettori]

suggestion — This claim list says "kube API … BLOCKED", but for ocp-single smoke.sh turns that claim into a note. The next line ("a custom SH_SINGLE_NAMESPACE needs no extra flag") only holds when the kubeconfig context is already in that namespace (see the smoke.sh comment).

[pdettori]

Fixed in e338f59. §12.2 now lists kube API as a note, not a claim. It also says to export the install-time SH_SINGLE_NAMESPACE for a custom namespace, because the ConfigMap fallback only works when the context is already in it.

[pdettori]

suggestion — One consequence of the collapse isn't listed here. In the base, moca-control-plane-credentials grants get/create/update/patch/delete on Secrets only in moca-credentials. Collapsed into one namespace, the Role covers every Secret: relay attach/exec tokens, the Redis password, the MU1 keys. A compromised control plane can now read and overwrite platform secrets, not just user credentials. That may be acceptable for dev/test, but this paragraph should say so.

[pdettori]

Added to §12.1 in e338f59 as a third consequence. The Role has no resourceNames, so once the namespaces collapse it reaches the relay tokens, the Redis password and the MU1 keys. Those names are fixed, so "no list" doesn't help.

[pdettori]

suggestion — "Cluster-level admission" probably isn't the mechanism; admission doesn't filter traffic. What you observed fits OVN-K evaluating ipBlock after the Service VIP is DNAT'd: the DNS VIP is blocked (backends are pod IPs in excepted ranges), other VIP ports are blocked (no endpoint), and the API port is open because the apiserver endpoint sits outside the except list (e.g. a ROKS master endpoint), so the overlay's own 0.0.0.0/0 rule admits it. If so, the cause is the sandbox internet-egress rule, not the cluster. Worth rewording here and in smoke.sh's note text.

[pdettori]

Reworded in e338f59, in README §12.3 and in smoke.sh's comment and note. The text now names the sandbox's own 0.0.0.0/0 rule matching after DNAT as the most likely cause, and drops "admission". I kept it hedged: I couldn't confirm the apiserver endpoint on that tenant. On ROKS it is often 172.20.0.1:2040, which is inside 172.16.0.0/12, and that would contradict this explanation. The README says the endpoint is unconfirmed. Worth checking kubectl get endpointslices -n default -l kubernetes.io/service-name=kubernetes next time we have the tenant.

[pdettori]

Correction to my reply above. I re-probed the ROKS tenant from moca-sandbox-0, and the theory doesn't hold. f6d4afb rewrites §12.3 and the smoke note to match:

• The CNI is Calico, not OVN-K.
• kubernetes.default.svc → 172.21.0.1:443 → ROKS node-local proxy 172.20.0.1:2040. A direct connection to 172.20.0.1:2040 is OPEN, even though it's inside the excepted 172.16.0.0/12. Other ports on that IP, 10.0.0.1 and the DNS VIP are blocked. So the 0.0.0.0/0 rule isn't what admits it. Some cluster-level allowance is, and I can't read it with namespace-only rights (GlobalNetworkPolicies are forbidden). The original "cluster-level" wording was closer; only "admission" was wrong.

Two more findings, now documented in §12.3:

• The node network is on a non-RFC 1918 range, which is outside the except list. The sandbox reaches its node's kubelet (10250) and SSH (22).
• A preinstalled tenant allow-same-namespace policy widens ingress to every pod. Isolation rests on the sandbox's egress policy, which holds: Redis and relay exec are BLOCKED.

[pdettori]

nit — This cites "README §12.4" for the verification. That's §12.3; §12.4 is "What does not work here".

[pdettori]

Fixed in e338f59 (§12.3).

[pdettori]

nit — This assertion can never fail: the ocp-single access banner doesn't contain the string ocp-single, so 'ocp-single.*https' never matches. Grep the access output for https:// instead.

[pdettori]

Fixed in e338f59: the assertion now greps for https://. Mutation-checked: an https:// line injected into the ocp-single banner makes the test fail; it passes without it.