feat(runtime): route host-local inference by ericksoa · Pull Request #8066 · NVIDIA/NemoClaw

ericksoa · 2026-08-01T15:10:20Z

Summary

This PR connects the dormant host-local inference runtime contract to the provider-neutral onboarding and application-routing boundary. Ollama, NIM, and vLLM remain injected capabilities; no central Podman switch or production activation is added.

Changes

Add provider-neutral startup requests and route results for host Ollama and managed NIM/vLLM.
Bind returned receipts to the injected provider and exact runtime authority.
Translate provider-native endpoints to the canonical gateway registration host while keeping https://inference.local/v1 identical for OpenClaw, Hermes, and Deep Agents Code.
Reject provider, service, and authority mismatch before runtime or gateway mutation.
Carry immutable managed probe-image authority through routing fixtures.
Prove retry after provider-registration failure reuses exact managed ownership and publishes one final route reservation.
Keep the resolver dormant until later activation and protected qualification slices.

Quality Gates

Tests added or updated for changed behavior
Docs not applicable: no supported runtime surface is activated
Sensitive paths changed
Exact-head advisor, CI, and protected E2E refresh completed

Documentation Writer Review

Documentation writer subagent reviewed the completed changes
Result: pending exact-head refresh
Evidence: review requested for exact head b588f8b468c8a6982efff82dd57790325176e5dc.
Agent: Codex Desktop

Verification

Signed commits and DCO
Focused provider/routing suite: 5 files, 40 tests passed
CLI typecheck, repository checks, growth guard, and normal commit/push hooks passed
Exact-head GitHub qualification is running

Exact head: b588f8b468c8a6982efff82dd57790325176e5dc.

Signed-off-by: Aaron Erickson aerickson@nvidia.com

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>

coderabbitai · 2026-08-01T15:10:27Z

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: c78456fc-9d9d-4e03-bbaf-ff67118edb28

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

🔍 Trigger review

_{Comment @coderabbitai help to get the list of available commands.}

github-code-quality · 2026-08-01T15:10:59Z

Code Coverage Overview

Languages: TypeScript

TypeScript / code-coverage/plugin

The overall coverage in commit b588f8b in the feat/host-local-infe... branch remains at 96%, unchanged from commit d5b64a7 in the feat/podman-host-loc... branch.

Show a code coverage summary of the most impacted files.

File	`feat/podman-host-loc...` d5b64a7	`feat/host-local-infe...` b588f8b	+/-
`nemoclaw/src/se...ntial-filter.ts`	100%	0%	-100%
`nemoclaw/src/sh...er-boundary.cts`	95%	0%	-95%
`nemoclaw/src/se...ot-sanitizer.ts`	95%	0%	-95%
`nemoclaw/src/co...test-fixture.ts`	81%	0%	-81%
`nemoclaw/src/co...ration-state.ts`	93%	94%	+1%

TypeScript / code-coverage/cli

The overall coverage in commit b588f8b in the feat/host-local-infe... branch remains at 81%, unchanged from commit d5b64a7 in the feat/podman-host-loc... branch.

Show a code coverage summary of the most impacted files.

File	`feat/podman-host-loc...` d5b64a7	`feat/host-local-infe...` b588f8b	+/-
`src/lib/onboard...trap/adapter.ts`	0%	63%	+63%
`src/lib/onboard...strap/docker.ts`	0%	64%	+64%
`src/lib/onboard...ne-lifecycle.ts`	0%	83%	+83%
`src/lib/onboard...cker-journal.ts`	0%	83%	+83%
`src/lib/onboard...al-inference.ts`	0%	85%	+85%
`src/lib/onboard...trap-journal.ts`	0%	86%	+86%
`src/lib/actions...ne-providers.ts`	0%	86%	+86%
`src/lib/onboard...-replacement.ts`	0%	87%	+87%
`src/lib/onboard...-transaction.ts`	0%	88%	+88%
`src/lib/onboard...test-fixture.ts`	0%	96%	+96%

_{Updated August 02, 2026 15:43 UTC}

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>

github-actions · 2026-08-01T15:34:00Z

PR Review Advisor — Blocking findings reported

Advisor assessment: Blockers require maintainer review
Next action: Review the blockers below.
Findings: 1 blocker · 0 warnings · 0 suggestions

Model lanes

GPT-5.6 Terra (primary): Completed · high confidence · 1 blocker · 0 warnings · 0 suggestions
Nemotron 3 Ultra (second opinion): Completed · high confidence · 0 blockers · 0 warnings · 0 suggestions
Model comparison: normalized findings differ; normalized terminology decisions differ; normalized E2E selections differ; Nemotron reported 1 fewer blocker, the same number of warnings, the same number of suggestions.

8 terminology differences from the second opinion

Advisory only. These are normalized differences from the primary terminology receipt.

provider-native at src/lib/onboard/runtime-provider/host-local-inference-routing.ts:59: primary classified it as justified; the second opinion classified it as established.
dormant at src/lib/onboard/machine/handlers/provider-inference.ts:69: selected only by the second-opinion lane as established.
host.openshell.internal at src/lib/onboard/runtime-provider/host-local-inference-routing.ts:12: selected only by the second-opinion lane as established.
authority drift at src/lib/onboard/runtime-provider/host-local-inference-routing.test.ts:116: selected only by the second-opinion lane as established.
inference.local at src/lib/onboard/runtime-provider/host-local-inference-routing.ts:13: selected only by the second-opinion lane as established.
HostLocalInferenceRuntime at src/lib/onboard/runtime-provider/host-local-inference-routing.ts:7: selected only by the second-opinion lane as established.
HostLocalInferenceStartupRequest at src/lib/onboard/runtime-provider/host-local-inference-routing.ts:23: selected only by the second-opinion lane as justified.
HostLocalInferenceStartupRoute at src/lib/onboard/runtime-provider/host-local-inference-routing.ts:33: selected only by the second-opinion lane as justified.

3 additional E2E selections from the second opinion

Advisory only. The primary lane did not select these E2E jobs or targets.

full-e2e: The completed second-opinion lane identified E2E coverage that the primary lane omitted.
ubuntu-repo-cloud-openclaw: The completed second-opinion lane identified E2E coverage that the primary lane omitted.
ubuntu-repo-cloud-langchain-deepagents-code: The completed second-opinion lane identified E2E coverage that the primary lane omitted.

Second-opinion terminology and E2E selections are advisory. They do not change the primary assessment or E2E / PR Gate.

3 semantic terminology decisions

Terminology decisions are advisory. They affect the assessment only when a separate finding identifies concrete semantic impact.

established — host-local inference at src/lib/onboard/runtime-provider/host-local-inference-routing.ts:53: Keep the established term for this routing path.
justified — provider-native at src/lib/onboard/runtime-provider/host-local-inference-routing.ts:59: Keep the modifier because it identifies the host that must not be exposed to central orchestration.
justified — canonical host alias at src/lib/onboard/runtime-provider/host-local-inference-routing.ts:61: Keep the phrase with the adjacent host constant as its concrete definition.

E2E guidance

Advisory only. E2E / PR Gate selects and runs jobs independently.

Recommended E2E: cloud-onboard, inference-routing, network-policy, onboard-repair, onboard-resume

1 optional E2E recommendation

ollama-auth-proxy

Blockers

`PRA-1` Blocker — Wire host-local startup into the onboarding state machine

Location: src/lib/onboard/machine/handlers/provider-inference.ts:770
Category: correctness
Problem: The state machine creates inference options without hostLocalInference in both the resume and normal setup paths. Consequently, ordinary onboarding cannot start the host-local runtime, validate its receipt authority, or register its canonical route; those operations occur only in direct setupInference tests.
Impact: The new routing and authority-validation behavior is unreachable from ordinary onboarding, so host-local inference cannot be configured through this integration.
Fix: Pass a validated HostLocalInferenceStartupRequest and the sandbox-bound runtime resolver from the selected runtime-provider surface through both state-machine setup paths, or remove this dormant integration until a current production consumer exists.
Verification: Read both deps.setupInference call sites in provider-inference.ts and confirm that each options object carries the selected host-local startup request when that runtime is selected.
Test coverage: Add a state-machine integration test that selects a host-local runtime and proves setupInference receives the startup request and resolves the runtime in both a normal onboarding path and the applicable resume path.
Evidence: src/lib/onboard/machine/handlers/provider-inference.ts:770-789 constructs inferenceOptions for the resume branch without hostLocalInference. src/lib/onboard/machine/handlers/provider-inference.ts:985-1001 constructs inferenceOptions for the normal branch without hostLocalInference. src/lib/onboard/setup-inference.ts:404-415 executes host-local startup only when options.hostLocalInference is present. Repository search found no production assignment of hostLocalInference or resolveHostLocalInferenceRuntime.

Workflow run details

This automated review informs maintainers. Warnings and suggestions do not require a response. A maintainer decides whether to merge.

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>

cv

Reviewed commit b588f8b468 in bottom-up stack order. This PR depends on #8061, which is not approval-ready. Although this child reports green checks, its parent is conflicting and can change the routing contract, so the green child result cannot authorize approval. Review remains blocked until the parent is reconstructed or repaired, this branch is restacked on the accepted parent commit, and current CI, security, and documentation evidence passes for the resulting commit. No approval is granted for the current branch.

## Summary Reconstructs B4-D of #7744 on current `main` and consolidates the former 4.5A–4.5D review slices into one dormant provider-owned Podman GPU and host-local inference batch. Ollama, NVIDIA NIM, and vLLM now share exact operation authority, durable secret-free lifecycle receipts, and provider-neutral `inference.local` startup integration for OpenClaw, Hermes, and LangChain Deep Agents Code. Podman remains absent from production registration and support claims. ## Related Issue Refs #7744. Consolidates the reviewed source intent from #8059, #8060, #8061, and #8066 into this existing PR. ## Changes - Bind each host-local inference operation to the exact Podman executable and safe parent chain, socket endpoint, server authority, provider-owned network, and physical NVIDIA CDI UUID inventory. Revalidation rejects executable, endpoint, CDI, launch-spec, or authority drift before mutation. - Translate only the bounded Podman inference command surface. The translator rejects Docker-named helper execution, raw mounts, host IPC, `--rm`, ambiguous GPU requests, mutable images, and secret-bearing argv. - Add canonical secret-free Ollama, NIM, and vLLM receipts bound to provider, engine, endpoint, immutable images, runtime identity, launch digest, CDI authority, provider-native HTTP/inference proof, and publication transaction. - Add fail-closed start, inspect, stop, preserve, resume, interrupted recovery, rollback, and destroy behavior through an injected runtime provider. Evidence is redacted before fallback or rollback; missing, reused, ambiguous, or indeterminate cleanup identity is rejected. - Route the three agents through the same `https://inference.local/v1` application contract while direct provider proof uses the inspected provider gateway. Startup and policy tests cover success, drift, rollback, stopped/running restoration, cleanup ambiguity, recovery, route allow/deny, and cross-engine exclusion. - Preserve only the two bounded OpenClaw continuity fields needed to prevent `missing-meta-vs-last-good` rollback; backup routes, providers, credentials, and arbitrary metadata never cross into the regenerated managed config. - Preserve Docker behavior and the socket-free MXC-style provider seam. `CURRENT_RUNTIME_PROVIDER_BUNDLES` remains Docker/Kubernetes only, the production host-local resolver remains null, and hidden `--temp-managed-runtime` behavior remains default-off and undocumented. The operation-scoped provider-owned selection is carried through the provider inference handler and all three startup flows, with current-provider null-path regressions. Per #7744's 4.5C/4.5D boundary, the production root intentionally injects no Podman selection in B4-D; durable ownership follows in B4-E and production Podman registration remains reserved for 4.6D. ## Reconstruction Safety - The original PR and donor heads were preserved before reconstruction under `backup/b4d-20260811/*` refs. - The public branch update is a fast-forward from the preserved #8061 head; no commit was rewritten and no force push was used. - Current `main` is included through signed merge commits, and the B4-D patch was reconstructed path-by-path instead of cherry-picking stale cross-cutting donors. ## Type of Change - [x] Code change (feature, bug fix, or refactor) - [ ] Code change with doc updates - [ ] Doc only (prose changes, no code sample modifications) - [ ] Doc only (includes code sample changes) ## Quality Gates - [x] Tests added or updated for changed behavior - [ ] Existing tests cover changed behavior — justification: - [ ] Tests not applicable — justification: - [ ] Docs updated for user-facing behavior changes - [x] Docs not applicable — justification: Podman remains dormant, unregistered, default-off, and intentionally undocumented; no supported product workflow changes. - [x] Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging) - [x] Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: Independent exact-tree architecture and security audits found no P0/P1/P2 issue. They verified credential redaction, exact endpoint/CDI/executable authority, evidence-before-rollback ordering, cleanup fail-closed behavior, Docker/current-provider preservation, and dormant registration. - [ ] Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue: ## Documentation Writer Review - [x] Documentation writer subagent reviewed the completed changes - Result: `no-docs-needed` - Evidence: Codex Desktop reviewed the complete 57-file exact diff for `a4fbaf82e` against `06ac446dd` after reconciling #8724 review recovery. The diff changes dormant injected Podman paths and internal OpenClaw continuity metadata only; production registration remains Docker/Kubernetes, and no documented command, flag, default, configuration, or supported workflow changes. - Agent: Codex Desktop   ## DGX Station Hardware Evidence - [ ] Tested on DGX Station - Tested commit: Protected exact-head E2E pending. - Station profile/scenario: Podman physical NVIDIA CDI and local-inference qualification. - Result: Pending protected workflow evidence; no local synthetic result is claimed. - Supporting evidence: Pending exact-head workflow URL. ## Verification - [x] PR description includes a `Signed-off-by:` line and every newly published reconstruction commit appears as `Verified` in GitHub (`valid`). - [x] Pre-commit and pre-push hooks passed. Local whole-range commitlint additionally reports one preserved historical donor `style(...)` commit; the PR title conforms and GitHub's commit-lint workflow validates the title. - [x] Targeted behavior tests pass for the current change set — the prior B4-D changed-file suite passed 444 tests; the exact continuity fix additionally passed its 8-test regression file, all 128 generator tests, and 57 managed-startup tests. Typechecks, repository checks, catalog checks, test-structure gates, Biome, pre-commit, and pre-push hooks pass. - [x] Exact-head post-#8724 reconciliation passes 250 focused CLI tests across 14 files. The affected integration set passed 132/133 in the host environment; the sole Homebrew trust leak reproduced on exact base, and the isolated test passed on both head and base with host Homebrew excluded from `PATH`. Both typechecks, repository architecture checks, test size/title gates, Biome, diff hygiene, and pre-commit/pre-push hooks pass. Exact-head CodeRabbit found an assertion typing weakness; the shared mocks now use the real dependency signatures and the negative checks inspect strongly typed call tuples without casts. The four affected handler files pass 78 tests, and CLI typecheck passes on `a4fbaf82e`. After integrating current main, canonical catalog generation plus the merged-tree CLI set pass 132 tests; adjacent Docker authority, Ollama cleanup, vLLM catalog, platform, runner, and dispatch integration passes 143 tests, with both typechecks green. - [ ] Applicable broad gate passed — exact-head canonical CI, Advisor, CodeRabbit, managed-image/startup lanes, and protected E2E are pending. The local all-project run was non-diagnostic under host oversubscription; relevant failures passed serially, including the current-provider integration with Homebrew excluded from the child `PATH`. - [x] Quality Gates section completed with required justifications or waivers - [x] No secrets, API keys, or credentials committed - [x] `npm run docs:prepare && npm run docs:validate` passed with zero errors and two standard Fern warnings. - [ ] Doc pages follow the style guide (doc changes only) - [ ] New doc pages include SPDX header and frontmatter (new pages only) Additional exact-head local gates: CLI build, standard and strict CLI typechecks, repository checks, config/catalog validation, source-shape budget, test-size budget, semantic E2E phase coverage (125 tests across 81 files), Biome, gitleaks, and docs validation. ## Exact Slice Identity - Head: `a4fbaf82e618c6f09afa30033560cb19c9be6b85` - Base: `06ac446ddaa8f4ebd790da463757f3ffd52a764b` - Delta: 57 files, +13,010/-344 --- Signed-off-by: Aaron Erickson <aerickson@nvidia.com>  ## Summary by CodeRabbit * **New Features** * Added host-local inference support for Ollama, NIM, and vLLM through managed Podman runtimes. * Added startup recovery, resume, rollback, canonical routing, GPU support, and provider-neutral inference checks. * Added secure, operation-scoped environment forwarding. * Preserved valid OpenClaw version metadata during configuration regeneration. * **Bug Fixes** * Prevented stale local-inference policies and unauthorized direct host access. * Detects executable, endpoint, runtime, network, GPU, and configuration changes. * Improved rollback, cleanup verification, redaction, and fail-closed behavior. * OpenClaw health checks now accept authenticated responses. * Fixed recovery for containers caught in restart loops.  --------- Signed-off-by: Aaron Erickson <aerickson@nvidia.com> Signed-off-by: Apurv Kumaria <akumaria@nvidia.com> Signed-off-by: Julie Yaunches <jyaunches@nvidia.com> Co-authored-by: Apurv Kumaria <akumaria@nvidia.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Julie Yaunches <jyaunches@nvidia.com>

ericksoa added 2 commits August 1, 2026 07:59

feat(runtime): route host-local inference

91eec72

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>

merge(stack): bind host inference parent

0410dbe

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>

This was referenced Aug 1, 2026

[Epic] Support native Podman with buildless managed onboarding #7744

Open

feat(runtime): add native installer qualification receipts #8067

Merged

test(runtime): inventory host-local routing boundary

3961da6

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>

ericksoa added 6 commits August 2, 2026 06:55

merge(stack): restack routing on inference readiness

c176e0d

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>

docs(runtime): align inference route terminology

895d0e5

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>

merge(stack): restack routing on test-only harness

d62c13b

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>

merge(stack): restack routing on managed readiness

38f670e

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>

test(runtime): carry managed readiness authority

68991cc

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>

test(runtime): cover managed route retry

b588f8b

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>

ericksoa mentioned this pull request Aug 2, 2026

feat(runtime): complete host-local inference lifecycle #8076

Closed

10 of 13 tasks

apurvvkumaria assigned ericksoa Aug 4, 2026

cv reviewed Aug 4, 2026

View reviewed changes

cv mentioned this pull request Aug 4, 2026

feat(runtime): add durable host-local inference lifecycle #8069

Closed

10 of 12 tasks

ericksoa merged commit 1539456 into feat/podman-host-local-inference-runtime Aug 5, 2026
72 of 73 checks passed

ericksoa deleted the feat/host-local-inference-routing-integration branch August 5, 2026 06:04

ericksoa mentioned this pull request Aug 11, 2026

feat(runtime): add dormant Podman GPU and local inference #8061

Merged

13 of 24 tasks

Navigation Menu

Repository navigation

feat(runtime): route host-local inference - #8066

feat(runtime): route host-local inference#8066
ericksoa merged 9 commits into
feat/podman-host-local-inference-runtimefrom
feat/host-local-inference-routing-integration

ericksoa commented Aug 1, 2026 •

edited

Loading

Uh oh!

coderabbitai Bot commented Aug 1, 2026 •

edited

Loading

Review skipped

Uh oh!

github-code-quality Bot commented Aug 1, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Aug 1, 2026 •

edited

Loading

Uh oh!

cv left a comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

ericksoa commented Aug 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

Quality Gates

Documentation Writer Review

Verification

Uh oh!

coderabbitai Bot commented Aug 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review skipped

Uh oh!

github-code-quality Bot commented Aug 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code Coverage Overview

TypeScript / code-coverage/plugin

TypeScript / code-coverage/cli

Uh oh!

github-actions Bot commented Aug 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Review Advisor — Blocking findings reported

Model lanes

E2E guidance

Blockers

PRA-1 Blocker — Wire host-local startup into the onboarding state machine

Uh oh!

cv left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

ericksoa commented Aug 1, 2026 •

edited

Loading

coderabbitai Bot commented Aug 1, 2026 •

edited

Loading

github-code-quality Bot commented Aug 1, 2026 •

edited

Loading

github-actions Bot commented Aug 1, 2026 •

edited

Loading

`PRA-1` Blocker — Wire host-local startup into the onboarding state machine