You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[Bug]: a partially specified NVIDIADriver startupProbe renders zeros instead of operator defaults #3132
When an NVIDIADriver CR sets spec.startupProbe but leaves some fields unset, the GPU Operator renders explicit zeros for those fields into the driver DaemonSet, so the driver startup window collapses to the generic Kubernetes probe defaults instead of the operator's intended values.
getDriverSpec (internal/state/driver.go) applies getDefaultStartupProbe only when spec.StartupProbe == nil:
If the CR specifies the probe partially (for example startupProbe: {initialDelaySeconds: 120}), the unset fields stay zero, and manifests/state-driver/0500_daemonset.yaml renders failureThreshold: 0, periodSeconds: 0, timeoutSeconds: 0, successThreshold: 0. The API server then substitutes the generic defaults (failureThreshold 3, periodSeconds 10, timeoutSeconds 1) rather than the operator defaults (failureThreshold 120, timeoutSeconds 60). A driver that compiles for more than ~30s, or whose nvidia-smi probe takes longer than 1s, is then killed and restarted in a loop.
The ClusterPolicy path does not have this problem: setContainerProbe (controllers/object_controls.go) overlays only the non-zero user fields on top of the defaults. The two paths disagree.
To Reproduce
Apply an NVIDIADriver CR whose spec.startupProbe sets only initialDelaySeconds (any single field), and inspect the rendered driver DaemonSet: the other probe fields are 0.
Reproduced with a unit test over getDriverSpec: a CR with StartupProbe{InitialDelaySeconds: 120} yields failureThreshold 0 / periodSeconds 0 / timeoutSeconds 0 / successThreshold 0 instead of the operator defaults.
Expected behavior
A partially specified startupProbe keeps the fields the user set and falls back to the operator defaults for the rest, as the ClusterPolicy path already does.
Environment (please provide the following information):
GPU Operator Version: present in v26.7.2 and on main (internal/state/driver.go has no overlay for a partial probe); reproduced on main
OS: N/A for the reproduction (unit test, macOS arm64, Go 1.26)
Kernel Version: N/A (not run on a cluster)
Container Runtime Version: N/A (not run on a cluster)
Kubernetes Distro and Version: N/A (not run on a cluster; reproduced with the internal/state unit tests)
Activity