Skip to content

Bound Slurm evidence collector resources and add a liveness probe - #114

Merged
RohanAdwankar merged 3 commits into
mainfrom
fix/pr111-resource-test-cleanup
Nov 17, 2026
Merged

RohanAdwankar merged 3 commits into
mainfrom
fix/pr111-resource-test-cleanup

Conversation

@RohanAdwankar

@RohanAdwankar RohanAdwankar commented Nov 16, 2026 •

Copy link
Copy Markdown
Member

Include the implementation from PR #111 together with the reviewed test cleanup. Add configurable collector resource requests and limits and an HTTP liveness probe. Test explicit resource overrides and resources:null without pinning chart defaults, and describe the render test accurately. Validation: three targeted Slurm render tests, two collector behavior tests and Helm lint passed; the branch merges cleanly with current main.

Summary by CodeRabbit

  • New Features
    • The Slurm evidence collector now has a liveness check that periodically verifies its health.
    • Default CPU and memory requests and limits are provided for the collector, and these settings can be customized. Setting resources to null omits the resource configuration while keeping the liveness check enabled.

SelvanayakiGurusamy and others added 2 commits November 14, 2026 15:04
The Slurm evidence collector DaemonSet set no resources, so it ran as
BestEffort on the same nodes as Slurm jobs, and it had only a readiness
probe, so a hung collector was marked unready but never restarted.

Add slurmEvidenceCollector.resources with conservative defaults
(requests 50m CPU / 64Mi, limits 500m CPU / 256Mi), rendered when set
and omitted when set to null, and a livenessProbe on the existing
/healthz endpoint. Add a chart-render test for the defaults, overrides,
and omission.

Refs #109

Signed-off-by: Selvanayaki Gurusamy <354650042+SelvanayakiGurusamy@users.noreply.github.com>
Assert only explicitly supplied resource quantities, retain null-resource coverage, and name the test for the rendered liveness configuration it checks.

Signed-off-by: Rohan Adwankar <39285979+RohanAdwankar@users.noreply.github.com>
@RohanAdwankar

Copy link
Copy Markdown
Member Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Nov 17, 2026 •

Copy link
Copy Markdown
Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@coderabbitai

coderabbitai Bot commented Nov 17, 2026

Copy link
Copy Markdown

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

The Slurm evidence collector now has an HTTP liveness probe at /healthz and configurable container resources. Default requests and limits are set in chart values. Render tests cover custom resources and the behavior when resources are set to null.

Changes

Slurm evidence collector configuration

Layer / File(s) Summary
Collector probe and resource configuration
helm/mosaic-stack/values.yaml, helm/mosaic-stack/templates/slurm-evidence.yaml, helm/mosaic-stack/tests/chart-render.test.mjs
The chart sets default CPU and memory requests and limits, renders configured resources, and adds a liveness probe at /healthz. Render tests check custom resource values and confirm that resources: null omits the resource block while retaining the probe.

Estimated code review effort: 2 (Simple) | ~8 minutes

Change: Feature

Possibly related issues

  • Issue 109 requests configurable resources and a /healthz liveness probe for the Slurm evidence collector. This change adds both; the requested Node.js image update is not included.
Merge Risk: Minimal · up to b7235

This adds a liveness probe and default CPU and memory bounds to the Slurm evidence collector. Setting resources to null removes the bounds. No concrete defects were found and the change is covered by render tests, so it is ready to merge.

Security architecture risk: Minimal · up to b7235

The collector gains resource bounds and health-based restarts without changing its credentials, host access, or API. No introduced security or persistent-state recovery risk was established.

Retained concerns
No architecture-level concerns identified.

Security review details

Security Blast Radius

  • inferred — The changed failure domain is the collector container on eligible Slurm nodes. The PR bounds its CPU and memory consumption without expanding the configured credential, host mount, evidence roots, or network listener. Cluster-wide reachability was not assessed.

Trust Boundaries and Controls

  • observed — Evidence routes retain timing-safe Bearer-token verification and canonical allowed-root checks for requested paths. The unauthenticated health endpoint predates this PR and was already targeted by readiness; the new probe does not grant access to evidence routes.

Resilience and Maintainability Implications

  • inferred — The probe checks HTTP responsiveness, not progress of an evidence operation. A pending asynchronous operation can retain the busy gate while health checks succeed. This existing limitation is not a newly introduced recovery regression, and the inspected helpers use asynchronous filesystem operations without durable writes.
Pre-merge checks: Passed 4

Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title names the two main changes: bounded resources for the Slurm evidence collector and a new liveness probe. The raw summary shows both changes in the Helm template, values.yaml, and render test…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Title check

Explanation

The title names the two main changes: bounded resources for the Slurm evidence collector and a new liveness probe. The raw summary shows both changes in the Helm template, values.yaml, and render tests. The title is specific and matches the changeset.

Finishing Touches

Generate unit tests (beta)

  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

A rabbit checks the probe at dawn
At /healthz, a green light stays on
Small requests and limits fit
Null means no resource block is writ
The collector hops along its way

Recent review info

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Enterprise
  • Run ID: 94ca096c-f276-4468-a448-95a91cc761c9
Commits

Reviewing files that changed from the base of the PR and between 0509477 and b7235c1.

Files selected for processing: 3

  • helm/mosaic-stack/templates/slurm-evidence.yaml
  • helm/mosaic-stack/tests/chart-render.test.mjs
  • helm/mosaic-stack/values.yaml

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 10 remain after this review.

Remove unvalidated default requests and limits and the unsupported workload co-location claim. Preserve resource passthrough and the HTTP liveness probe, and verify that default renders omit resource sizing.

Signed-off-by: Rohan Adwankar <39285979+RohanAdwankar@users.noreply.github.com>
@RohanAdwankar
RohanAdwankar merged commit fa7a554 into main Nov 17, 2026
4 checks passed
@RohanAdwankar
RohanAdwankar deleted the fix/pr111-resource-test-cleanup branch November 17, 2026 20:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants