Skip to content

fix(onboard): record managed vLLM serving-profile provenance - #12346

Closed
harjothkhara wants to merge 2 commits into
NVIDIA:mainfrom
harjothkhara:codex/nemoclaw-12289-2026-09-27
Closed

harjothkhara wants to merge 2 commits into
NVIDIA:mainfrom
harjothkhara:codex/nemoclaw-12289-2026-09-27

Conversation

@harjothkhara

@harjothkhara harjothkhara commented Sep 27, 2026 •

Copy link
Copy Markdown
Contributor

Outcome

Fresh onboarding with an explicit managed vLLM serving profile now preserves the catalog-derived profile provenance through selection, the onboarding session, sandbox registration, and status reporting. Previously, the managed vLLM selection result omitted that provenance and the provider handler replaced the valid session value with null.

Reason

NemoClaw needs the exact selected serving profile in the sandbox registry so status and later lifecycle operations can identify the configuration that produced the managed vLLM deployment.

Related issues

Fixes #12289

Changes

  • Add catalog-derived provenance to the existing requested serving-profile model consumed by managed vLLM selection.
  • Reuse the resolved profile model during selection and return its provenance only after this run installs and selects managed vLLM. Existing endpoints, retry, failed, unprofiled, and session-only paths return no new provenance.
  • Add regression coverage through command resolution, managed vLLM selection, session update, registry creation, and status projection.

Verification

Latest published commit: 1b4c9c2c2c09f70488884370236bd3f826e4300f.

  • npx vitest run --project cli src/lib/onboard/setup-nim-flow-serving-profile.test.ts src/lib/onboard/machine/handlers/provider-inference-managed-llama-resume.test.ts src/lib/inference/serving/requested-profile-model.test.ts src/lib/onboard/setup-nim-flow-vllm-resume.test.ts --no-file-parallelism passed: 4 files, 26 tests.
  • CLI and plugin builds passed. Normal pre-commit and commit-message hooks passed.
  • Normal pre-push publication validation and plugin, JavaScript, and CLI TypeScript checks passed with NODE_OPTIONS=--max-old-space-size=8192.
  • GitHub verified the published commit signature. The diff contains no credentials or secrets.

Original contributor verification follows:

  • npx vitest run --project cli src/lib/onboard/machine/handlers/provider-inference-managed-llama-resume.test.ts --no-file-parallelism — T1 RED before the production change: exit 1; the new [DGX Spark][Sandbox] serving profile provenance is never recorded, so status cannot report it before or after rebuild #12289 regression was the sole failure among 8 tests.
  • npx vitest run --project cli src/lib/onboard/machine/handlers/provider-inference-managed-llama-resume.test.ts src/lib/onboard/setup-nim-flow-serving-profile.test.ts src/lib/inference/serving/requested-profile-model.test.ts --no-file-parallelism — exit 0; 3 files and 17 tests passed.
  • npx vitest run --project cli src/lib/onboard/command.test.ts src/lib/onboard/setup-nim-flow.test.ts src/lib/onboard/machine/handlers/provider-inference-managed-llama-resume.test.ts src/lib/onboard/sandbox-registration.test.ts src/lib/onboard/setup-nim-flow-serving-profile.test.ts src/lib/inference/serving/requested-profile-model.test.ts --no-file-parallelism — exit 0; 6 files and 141 tests passed.
  • npm --prefix nemoclaw run build && npm run typecheck:cli — exit 0.
  • npm run checks:repository — exit 0; all 18 checks passed.
  • node scripts/lib/openshell-sdk-install.mts check — exit 0; the vendored OpenShell SDK imports successfully.
  • git diff --check — exit 0.
  • npm run validate:pr — exit 0 on signed commit 1d9f58d3713ac2ea52e958ad59c46ea0e18b33dd with a clean worktree.
  • Repository-wide symbol search found no integration-project consumer requiring another test project.
  • The complete diff was inspected and contains no secrets, API keys, or credentials.
  • No DGX or live-hardware proof was performed, and none is claimed; the regression is deterministic and does not require hardware.

Review notes

All five changed files match the canonical contributor sensitive-path patterns under src/lib/inference/** or src/lib/onboard/**. An independent routine Claude review inspected the complete NVIDIA/NemoClaw diff at commit 1d9f58d3713ac2ea52e958ad59c46ea0e18b33dd and returned CLEAN/READY with no findings. Receipt: /Users/harjothk/Documents/cowork/.pr-review/results/4894249efb4d50cb92a7fe8e8f3dd0f127557443812c1f528ea27d22ca1de227/receipt.json. This review context does not authorize approval or merge.


Signed-off-by: harjoth harjoth.khara@gmail.com

Signed-off-by: Rebecca Sliter 571084+rsliter@users.noreply.github.com

Summary by CodeRabbit

  • Improvements
    • Serving profile provenance is now preserved when a requested profile is used for a managed vLLM installation, carried into the saved provider setup, and available in sandbox status reporting.
    • Matching serving profiles now include provenance when their served name is valid.
    • Profiles for already-running vLLM servers do not include managed-install provenance.

Preserve catalog-derived serving-profile provenance after a managed vLLM
profile is selected. Fresh onboard now carries the selected profile through
session, registry, and status.

Fixes NVIDIA#12289

Agent-Id: agent-c5819b3a-f2af-4bee-a591-193fc379dcf4
@copy-pr-bot

copy-pr-bot Bot commented Sep 27, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@coderabbitai

coderabbitai Bot commented Sep 27, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Repository: NVIDIA/NemoClaw/.coderabbit.yaml
  • Review profile: CHILL
  • Plan: Enterprise
  • Run ID: b2573869-187b-4477-9c02-cbefc06fd1aa
📥 Commits

Reviewing files that changed from the base of the PR and between 1d9f58d and 1b4c9c2.

📒 Files selected for processing (3)
  • src/lib/onboard/machine/handlers/provider-inference-managed-llama-resume.test.ts
  • src/lib/onboard/setup-nim-flow-serving-profile.test.ts
  • src/lib/onboard/setup-nim-flow.ts

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 10 remain after this review.


📝 Walkthrough

Walkthrough

Onboarding now records provenance for an explicitly requested managed vLLM profile. The resolved profile model includes provenance, which the onboarding flow carries into the session, sandbox registry entry, and status report. Existing vLLM selections do not receive this provenance.

Changes

Serving Profile Provenance

Layer / File(s) Summary
Resolve serving profile provenance
src/lib/inference/serving/requested-profile-model.ts, src/lib/inference/serving/requested-profile-model.test.ts
The requested serving profile model now includes provenance for its selected preset. The test checks the returned provenance.
Propagate provenance through onboarding
src/lib/onboard/setup-nim-flow.ts, src/lib/onboard/setup-nim-flow-serving-profile.test.ts, src/lib/onboard/machine/handlers/provider-inference-managed-llama-resume.test.ts
The vLLM flow resolves the requested profile once and records its provenance for managed installs. Tests check provenance in the session, registry entry, and status report, and check that an already-running vLLM server has no provenance.

Priority: ➖ Normal

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Bug fix · Severity of issue fixed: Medium

Sequence Diagram(s)

sequenceDiagram
  participant ProfileResolver
  participant Onboarding
  participant Session
  participant SandboxRegistry
  participant StatusReport
  ProfileResolver->>Onboarding: Resolve requested profile provenance
  Onboarding->>Session: Persist provenance for managed install
  Session->>SandboxRegistry: Supply inference selection
  SandboxRegistry->>StatusReport: Supply provenance for status
Loading

Suggested reviewers: cv

Merge Risk: ⚪ Minimal · up to 1b4c9

No concrete issue identified here blocks merging. The reported checks passed; live-hardware behavior remains unverified.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 66.67% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 6 functions across 5 files. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed The PR addresses the coding requirements in [#12289]. servingProfileModel now adds catalog provenance for the selected preset. The managed vLLM flow keeps that provenance only after a successful ins…
Out of Scope Changes check ✅ Passed The production changes are limited to serving-profile provenance resolution and successful managed vLLM selection. The test changes cover the linked onboarding and status path and refactor shared regi…
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the main change: recording managed vLLM serving-profile provenance during onboarding.
  • Fix all pre-merge checks with AI
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create a new PR

Comment @coderabbitai help to get the list of available commands.

@harjothkhara
harjothkhara marked this pull request as ready for review September 27, 2026 02:21
@wscurran wscurran added area: onboarding Onboarding FSM, provider setup, sandbox launch, or first-run flow area: providers Inference provider integrations and provider behavior bug-fix PR fixes a bug or regression provider: vllm vLLM local or hosted provider behavior labels Oct 6, 2026
Signed-off-by: Rebecca Sliter <571084+rsliter@users.noreply.github.com>

@rsliter rsliter left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed commit 1b4c9c2c2c09f70488884370236bd3f826e4300f.

The change now preserves managed vLLM profile provenance through the session, sandbox registry, and status output. During review, I repaired one edge case: reusing an existing endpoint must not claim a catalog recipe, image, or model revision that onboarding did not install. The updated regression verifies that boundary. All 26 focused tests, builds, and local publication/type checks passed. CodeRabbit completed without actionable findings. I found no remaining correctness or security issue in the reviewed diff; required CI is still outstanding.

Thank you @harjothkhara for tracing the missing provenance and contributing the fix and lifecycle regression coverage.

I am opening a replacement PR from a branch in NVIDIA/NemoClaw, then closing this PR with a link to it. Our required reviewed OpenShell SDK job currently rejects fork PRs. The replacement will preserve your original commit and authorship and credit you as the original author in its description.

@rsliter

rsliter commented Oct 7, 2026

Copy link
Copy Markdown
Collaborator

Continued in #12764, now open from a branch in NVIDIA/NemoClaw so the required reviewed OpenShell SDK CI can run.

Thank you @harjothkhara for the contribution. The replacement preserves your original signed commit and authorship and credits you as the original author. Closing this PR in favor of that linked replacement.

@rsliter rsliter closed this Oct 7, 2026
rsliter added a commit that referenced this pull request Oct 8, 2026
## Outcome

Fresh onboarding with an explicit managed vLLM profile preserves its
catalog provenance in the session, sandbox registry, and status report.
Reusing an existing vLLM endpoint does not claim that NemoClaw installed
its recipe or image.

Original author: **@harjothkhara (Harjoth Khara)**. The original signed
commit `1d9f58d3713ac2ea52e958ad59c46ea0e18b33dd` is preserved
unchanged. Rebecca Sliter supplied the managed-install guard and
maintainer follow-up.

## Reason

Provider selection previously erased valid profile provenance before
sandbox registration. This same-repository replacement also enables the
required reviewed OpenShell SDK CI path, which rejects fork PRs.

### Related issues

Fixes #12289. Replaces #12346 with the contributor's history intact.

## Changes

- Carry catalog-derived provenance in the requested serving-profile
model.
- Return that provenance only after this invocation successfully
installs and selects managed vLLM. The named gate keeps this rule out of
the selection function's nested control flow.
- Cover fresh installation through session, registry, and status, and
reject new provenance for an existing endpoint. Retain resume and
session-only denial coverage.
- Clarify that reused vLLM endpoints do not record the requested profile
provenance; preserve the status behavior for managed vLLM and llama.cpp
installations.

## Verification

Candidate: `032b321a4ee88141198fe41eedb767d4c5d49907`.

- `npx vitest run --project cli
src/lib/onboard/setup-nim-flow-serving-profile.test.ts
src/lib/onboard/machine/handlers/provider-inference-managed-llama-resume.test.ts
src/lib/inference/serving/requested-profile-model.test.ts
src/lib/onboard/setup-nim-flow-vllm-resume.test.ts
--no-file-parallelism`: 4 files and 26 tests passed. The
existing-endpoint regression failed before the guard.
- CLI and plugin builds, normal commit hooks, publication validation,
and the CLI TypeScript check passed. Canonical changed-path selection
skipped plugin and JavaScript typechecks. The checked-in compiler runner
uses an 8 GB Node heap.
- `npm run docs`: passed with zero errors; OpenClaw, Hermes, and Deep
Agents generated command references inspected. Fern reported the
existing optional authenticated redirect check and light-theme contrast
warnings.
- All PR commits are GitHub Verified. The inspected diff contains no
secrets, API keys, or credentials.

## Review notes

The five changed code and test files are sensitive inference or
onboarding paths. The additional command-reference clarification
received independent local documentation review. The [maintainer review
on the original
PR](#12346 (review))
covers `NVIDIA/NemoClaw` commit
`1b4c9c2c2c09f70488884370236bd3f826e4300f`, records the provenance
repair and regression evidence, and thanks the original contributor. The
current candidate awaits its required CI, GPU qualification, and
automated reviews.

---

Signed-off-by: harjoth <harjoth.khara@gmail.com>

Signed-off-by: Rebecca Sliter <571084+rsliter@users.noreply.github.com>


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Improvements**
* Managed vLLM setup retains the selected serving profile’s provenance
through setup and sandbox status reporting. Profile details are recorded
only when setup installs a managed server; reusing an already-running
vLLM server does not assign managed-profile details.
* Sandbox status displays serving-profile details for sandboxes created
with managed vLLM or llama.cpp. JSON status includes the complete
secret-free profile provenance record.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

Signed-off-by: Rebecca Sliter <571084+rsliter@users.noreply.github.com>
Co-authored-by: harjoth <harjoth.khara@gmail.com>
Co-authored-by: Rebecca Sliter <571084+rsliter@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: onboarding Onboarding FSM, provider setup, sandbox launch, or first-run flow area: providers Inference provider integrations and provider behavior bug-fix PR fixes a bug or regression provider: vllm vLLM local or hosted provider behavior

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[DGX Spark][Sandbox] serving profile provenance is never recorded, so status cannot report it before or after rebuild

3 participants