Repository navigation
Description
User Story
I use OpenShell's Helm chart to deploy managed OpenShell gateways on Kubernetes and OpenShift from NVIDIA NemoClaw, which installs the chart through the OpenTofu Helm provider. I directly need the gateway admitted under OpenShift's default restricted-v2 SCC by the same install as on Kubernetes, so that NemoClaw users don't have to declare that their cluster is OpenShift. On OpenShift Local 4.22.14 with chart 0.1.7, the gateway runs only when the install clears its UID and fsGroup. Downstream tracking: NVIDIA/NemoClaw#13828.
Problem Statement
The chart gives the gateway a fixed UID and fsGroup on every cluster: securityContext.runAsUser: 1000 and podSecurityContext.fsGroup: 1000 (values.yaml). OpenShift's restricted-v2 SCC admits only IDs inside the namespace's assigned range, and the chart doesn't detect OpenShift, so it can't leave the two IDs for OpenShift to assign. The docs also disagree on what an OpenShift install needs: the README's Install on OpenShift clears both, but the OpenShift guide's Install OpenShell clears neither and says "Keep the default restricted security posture".
Impact / Why This Matters
- Without this, every installer has to find out whether the cluster is OpenShift before rendering the chart, then clear both IDs itself. The README passes
--set podSecurityContext.fsGroup=null --set securityContext.runAsUser=null"to allow SCC assignment", and so do the guide's TLS installs (L164-L168, L216-L220). OpenShell's e2e harness detects OpenShift and layersci/values-openshift-scc.yaml, which also clears the shared supervisor's fixed 65534 IDs (fix(helm): allow shared supervisor SCC-assigned IDs #4785). - Tools that drive Helm from their own configuration, such as OpenTofu, Flux, or Argo CD, each need their own OpenShift switch for this.
- Getting it wrong fails at admission, away from the cause. fix(e2e): fail loudly when OpenShift detection is inconclusive #4090 describes OpenShell's own e2e silently misdetecting OpenShift, which "produced security contexts that SCC admission rejects".
- A reader who follows the guide's main install gets the chart's fixed IDs on OpenShift.
- The overrides can't simply be applied everywhere. On Kubernetes without SCC admission, a gateway with
podSecurityContext.fsGroup=nullcan't read its signing key, which the chart mounts withdefaultMode: 0400(_gateway-workload.tpl). With chart 0.1.7 on kind, it crash-looped withcannot read sandbox launch-signing private key from /etc/openshell-jwt/signing.pem: Permission denied.
Proposed Design
On OpenShift, installing the chart needs no security-context overrides:
oc create ns openshell
helm install openshell oci://ghcr.io/nvidia/openshell/helm-chart \
--version <version> \
--namespace openshell- On OpenShift, the rendered gateway pod sets no
runAsUserorfsGroup, sorestricted-v2assigns the namespace range's first UID and group.runAsNonRoot, the dropped capabilities, and the seccomp profile stay. - On other clusters, the gateway keeps UID 1000 and fsGroup 1000, as today.
- Values an operator sets explicitly still take effect on both.
- Rendering without a cluster (
helm template) has a documented way to say that the target is OpenShift, for example--api-versions security.openshift.io/v1.
The shared supervisor (supervisor_topology = "gateway") has the same fixed-ID pattern and could follow the same rule. We haven't run that topology.
Acceptance Criteria
- On OpenShift with the default
restricted-v2SCC, an install without security-context overrides runs the gateway as the namespace range's first UID, with the range's first group as its fsGroup. - On Kubernetes without SCC admission, the same install still runs the gateway as UID 1000 with fsGroup 1000.
- Explicitly set
securityContext.runAsUserandpodSecurityContext.fsGroupvalues render as given on both. - The README's OpenShift install, the OpenShift guide, and
ci/values-openshift-scc.yamlno longer clear the gateway's IDs, and the README and the guide give the same install. - Chart tests render the gateway with and without OpenShift and check both IDs.
Alternatives Considered
- Keep the overrides. They work. On OpenShift Local 4.22.14 with chart 0.1.7 and both nulls,
restricted-v2admitted the gateway, which ran as UID 1000680000 with fsGroup 1000680000 (range1000680000/10000) and didn't restart. But each installer still detects OpenShift itself, and the docs disagree on whether the overrides are needed. - Ship a supported OpenShift values file, for example by promoting
ci/values-openshift-scc.yaml. That gives the overrides one home, but the choice and the detection stay with each installer. - Drop both IDs on every cluster. This works for
runAsUser: the gateway image declares a numericUSER 1000:1000(Dockerfile.gateway), which kubelet accepts forrunAsNonRoot, and the certgen hook already relies on this (fix(helm): render Restricted-compatible security contexts #3340). It doesn't work forfsGroup, because of the signing key above, sofsGroupat least needs a per-cluster choice. - OpenShell extension points (middleware, interceptors, providers) don't apply: this is the chart's pod spec, fixed before the gateway runs.
Agent Investigation
- feat(openshift): supported install path without the privileged SCC or disabled TLS #2812 listed this as an acceptance criterion: "The gateway pod is admitted under
restricted-v2withoutpodSecurityContext.fsGroup=null/securityContext.runAsUser=nulloverrides." It was closed on 2026-08-23 with a pointer to feat(kubernetes): add proxy-pod topology (in-pod process supervisor, out-of-pod proxy) #2885, which closed unmerged in favor of feat(kubernetes): isolate workloads behind a dedicated supervisor #3144. At f3d9765 the gateway's defaults are still 1000 and 1000. - Related work:
- fix(helm): omit podSecurityContext block when value is null #3033 and fix(helm): omit podSecurityContext block when value is null #3034 render
podSecurityContext: nullcleanly. - feat(e2e): make e2e:kubernetes work transparently on OpenShift #3183 added the e2e detection, through the
route.openshift.ioAPI group, and the overlay. - bug: shared supervisor ignores OpenShift SCC security context overrides #4784 and fix(helm): allow shared supervisor SCC-assigned IDs #4785 extended the overlay to the shared supervisor.
- bug(helm): certgen hook is rejected by Restricted Pod Security #3215 and fix(helm): render Restricted-compatible security contexts #3340 left the certgen hook's
runAsUserunset forrestricted-v2. - fix(e2e): fail loudly when OpenShift detection is inconclusive #4090 (open) hardens the e2e detection.
- fix(helm): omit podSecurityContext block when value is null #3033 and fix(helm): omit podSecurityContext block when value is null #3034 render
- The chart already reads the API server's served versions: agent-sandbox-preflight.yaml checks
.Capabilities.APIVersions.Has "agents.x-k8s.io/v1beta1". A check onsecurity.openshift.io/v1is one possible signal. Because values.yaml defaults both IDs to 1000, telling an explicit 1000 from the default may need the default to move into the template.
Checklist
- I've reviewed existing issues and the published docs
- This is a design proposal, not a "please build this" request
Activity