Skip to content

feat(helm): let OpenShift assign the gateway's UID and fsGroup without value overrides #5483

Open

Description

@gaveezy

User Story

I use OpenShell's Helm chart to deploy managed OpenShell gateways on Kubernetes and OpenShift from NVIDIA NemoClaw, which installs the chart through the OpenTofu Helm provider. I directly need the gateway admitted under OpenShift's default restricted-v2 SCC by the same install as on Kubernetes, so that NemoClaw users don't have to declare that their cluster is OpenShift. On OpenShift Local 4.22.14 with chart 0.1.7, the gateway runs only when the install clears its UID and fsGroup. Downstream tracking: NVIDIA/NemoClaw#13828.

Problem Statement

The chart gives the gateway a fixed UID and fsGroup on every cluster: securityContext.runAsUser: 1000 and podSecurityContext.fsGroup: 1000 (values.yaml). OpenShift's restricted-v2 SCC admits only IDs inside the namespace's assigned range, and the chart doesn't detect OpenShift, so it can't leave the two IDs for OpenShift to assign. The docs also disagree on what an OpenShift install needs: the README's Install on OpenShift clears both, but the OpenShift guide's Install OpenShell clears neither and says "Keep the default restricted security posture".

Impact / Why This Matters

  • Without this, every installer has to find out whether the cluster is OpenShift before rendering the chart, then clear both IDs itself. The README passes --set podSecurityContext.fsGroup=null --set securityContext.runAsUser=null "to allow SCC assignment", and so do the guide's TLS installs (L164-L168, L216-L220). OpenShell's e2e harness detects OpenShift and layers ci/values-openshift-scc.yaml, which also clears the shared supervisor's fixed 65534 IDs (fix(helm): allow shared supervisor SCC-assigned IDs #4785).
  • Tools that drive Helm from their own configuration, such as OpenTofu, Flux, or Argo CD, each need their own OpenShift switch for this.
  • Getting it wrong fails at admission, away from the cause. fix(e2e): fail loudly when OpenShift detection is inconclusive #4090 describes OpenShell's own e2e silently misdetecting OpenShift, which "produced security contexts that SCC admission rejects".
  • A reader who follows the guide's main install gets the chart's fixed IDs on OpenShift.
  • The overrides can't simply be applied everywhere. On Kubernetes without SCC admission, a gateway with podSecurityContext.fsGroup=null can't read its signing key, which the chart mounts with defaultMode: 0400 (_gateway-workload.tpl). With chart 0.1.7 on kind, it crash-looped with cannot read sandbox launch-signing private key from /etc/openshell-jwt/signing.pem: Permission denied.

Proposed Design

On OpenShift, installing the chart needs no security-context overrides:

oc create ns openshell
helm install openshell oci://ghcr.io/nvidia/openshell/helm-chart \
  --version <version> \
  --namespace openshell
  • On OpenShift, the rendered gateway pod sets no runAsUser or fsGroup, so restricted-v2 assigns the namespace range's first UID and group. runAsNonRoot, the dropped capabilities, and the seccomp profile stay.
  • On other clusters, the gateway keeps UID 1000 and fsGroup 1000, as today.
  • Values an operator sets explicitly still take effect on both.
  • Rendering without a cluster (helm template) has a documented way to say that the target is OpenShift, for example --api-versions security.openshift.io/v1.

The shared supervisor (supervisor_topology = "gateway") has the same fixed-ID pattern and could follow the same rule. We haven't run that topology.

Acceptance Criteria

  • On OpenShift with the default restricted-v2 SCC, an install without security-context overrides runs the gateway as the namespace range's first UID, with the range's first group as its fsGroup.
  • On Kubernetes without SCC admission, the same install still runs the gateway as UID 1000 with fsGroup 1000.
  • Explicitly set securityContext.runAsUser and podSecurityContext.fsGroup values render as given on both.
  • The README's OpenShift install, the OpenShift guide, and ci/values-openshift-scc.yaml no longer clear the gateway's IDs, and the README and the guide give the same install.
  • Chart tests render the gateway with and without OpenShift and check both IDs.

Alternatives Considered

  • Keep the overrides. They work. On OpenShift Local 4.22.14 with chart 0.1.7 and both nulls, restricted-v2 admitted the gateway, which ran as UID 1000680000 with fsGroup 1000680000 (range 1000680000/10000) and didn't restart. But each installer still detects OpenShift itself, and the docs disagree on whether the overrides are needed.
  • Ship a supported OpenShift values file, for example by promoting ci/values-openshift-scc.yaml. That gives the overrides one home, but the choice and the detection stay with each installer.
  • Drop both IDs on every cluster. This works for runAsUser: the gateway image declares a numeric USER 1000:1000 (Dockerfile.gateway), which kubelet accepts for runAsNonRoot, and the certgen hook already relies on this (fix(helm): render Restricted-compatible security contexts #3340). It doesn't work for fsGroup, because of the signing key above, so fsGroup at least needs a per-cluster choice.
  • OpenShell extension points (middleware, interceptors, providers) don't apply: this is the chart's pod spec, fixed before the gateway runs.

Agent Investigation

Checklist

  • I've reviewed existing issues and the published docs
  • This is a design proposal, not a "please build this" request
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    state:triage-neededOpened without agent diagnostics and needs triage

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions