[senthilr-nv]

Outcome

Hermes recovery now waits for the gateway process to become observably running after NemoClaw starts a stopped sandbox. A transient indeterminate process observation no longer advances immediately to gateway verification and causes sandbox start or connect --probe-only to fail.

Reason

Live Docker/Hermes testing of the fix from #11922 showed that the first post-start process observation can be indeterminate while the same container continues starting normally. The prior predicate accepted that state, so recovery reached gateway inspection too early and failed.

Related issues

Follow-up to #11922

Changes

• Add a shared, bounded Hermes gateway-process settlement check that accepts only an observed running process.
• Apply the settlement check after sandbox start and when connect --probe-only starts a stopped ordinary Hermes sandbox.
• Preserve immediate behavior for already-running, paused, unresolved, failed-start, portable, OpenClaw, and terminal-agent paths.
• Add coverage for transient indeterminate observations, persistent non-running observations, successful recovery, and paths that must not receive post-start grace.

Verification

• ./node_modules/.bin/vitest run --project cli src/lib/actions/sandbox/start.test.ts src/lib/actions/sandbox/connect-probe-observe.test.ts — PASS, 2 files and 32 tests.
• node --max-old-space-size=8192 node_modules/typescript/bin/tsc -p tsconfig.cli.json — PASS after CLI and plugin builds.
• Normal pre-commit, commit-msg, and pre-push hooks — PASS.
• Live Brev Docker/Hermes reproduction on merged fix(hermes): wait for gateway after start #11933 commit b259c14c33f20f18a8ae2c995aaeb5e8242b14d9 — installation, created state, status, managed inference, route isolation, and official cleanup passed; explicit start and stop-to-probe recovery failed on the same running container after the transient process observation, confirming this change's target boundary.
• Diff review — no secrets, API keys, credentials, or private endpoint data included.

Review notes

This is a narrow lifecycle correction. It does not change credentials, network policy, routes, privileges, container identity, or external mutation controls. A PR-head live Docker/Hermes recovery run remains the final real-runtime validation.

---

Signed-off-by: Senthil Ravichandran senthilr@nvidia.com

Summary by CodeRabbit

• Bug Fixes
  • Improved sandbox recovery after starting a stopped container by waiting for its gateway process to become available.
  • Added retry handling for gateway startup checks to improve recovery reliability.
  • Sandbox connections now report a clear process-stage failure when the gateway does not become available, rather than incorrectly signaling readiness.
  • Preserved existing behavior for already-running containers and terminal error scenarios.

[coderabbitai]

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

• @coderabbitai resume to resume automatic reviews.
• @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

• ▶️ Resume reviews
• 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 599b4f28-69e0-4c65-aa97-3c42d808c19a

📥 Commits

Reviewing files that changed from the base of the PR and between 76bd1d9 and 5227620.

📒 Files selected for processing (1)

• src/lib/actions/sandbox/connect-probe-observe.test.ts

Included review availability: Your plan provides up to 12 included reviews per hour; 10 remain after this review.

---

📝 Walkthrough

Walkthrough

Hermes startup settlement now uses a shared retry helper. Connect probes wait for a started gateway before process recovery, classify failed settlement, and stop before readiness publication.

Changes

Hermes gateway settlement

Layer / File(s)	Summary
Shared settlement helper src/lib/actions/sandbox/process-recovery.ts, src/lib/actions/sandbox/start.ts, src/lib/actions/sandbox/status/process-recovery.ts	The shared helper performs up to three Hermes gateway probes with two-second delays between false results. Sandbox startup uses the helper and the status module re-exports it.
Connect probe startup recovery src/lib/actions/sandbox/connect.ts, test/support/connect-flow-test-harness.ts, src/lib/actions/sandbox/connect-probe-observe.test.ts	Connect probes identify stopped-container startup, wait for the Hermes gateway, record the processes stage, and exit on failed settlement. The harness and tests cover successful, failed, terminal-error, unavailable-gateway, and already-running-container cases.

Priority: ⬇️ Low

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Bug fix

Sequence Diagram(s)

sequenceDiagram
  participant ConnectProbe
  participant HermesSettlement
  participant GatewayProcess
  participant ProcessRecovery
  ConnectProbe->>HermesSettlement: wait after starting stopped Hermes container
  HermesSettlement->>GatewayProcess: probe gateway process
  GatewayProcess-->>HermesSettlement: return settlement result
  HermesSettlement-->>ConnectProbe: return true, false, or null
  ConnectProbe->>ProcessRecovery: continue recovery when settlement succeeds

Loading

Merge Risk: ⚪ Minimal · up to 52276

The change addresses Hermes startup settlement with bounded recovery behavior, and no remaining merge-blocking issue is identified.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 28.57% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 7 functions across 7 files.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly identifies the Hermes fix and the main behavior change: waiting for the gateway to become observable after startup.

• Fix all pre-merge checks with AI

✨ Finishing Touches 💡 1

📝 Generate docstrings 💡

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch codex/11922-hermes-indeterminate-start

---

_{Comment @coderabbitai help to get the list of available commands.}

[github-code-quality]

Code Coverage Overview

Languages: TypeScript

TypeScript / code-coverage/plugin

The overall line coverage in commit 5227620 in the codex/11922-hermes-i... branch remains at 96%, unchanged from commit b10f3a2 in the main branch.

TypeScript / code-coverage/cli

The overall line coverage in commit 5227620 in the codex/11922-hermes-i... branch remains at 83%, unchanged from commit 1eb370f in the main branch.

Show a line coverage summary of the most impacted files.

File	main 1eb370f	codex/11922-hermes-i... 5227620	+/-
src/lib/state/r...e-generation.ts	82%	64%	-18%
src/lib/onboard...ort-listener.ts	98%	93%	-5%
src/lib/onboard...ess-identity.ts	100%	97%	-3%
src/lib/adapter...shell/client.ts	92%	91%	-1%
src/lib/actions...dbox/connect.ts	85%	85%	0%
src/lib/actions...ess-recovery.ts	65%	65%	0%
src/lib/onboard...uild-context.ts	87%	87%	0%
src/lib/onboard...eway-process.ts	89%	89%	0%
src/lib/onboard...eway-runtime.ts	94%	96%	+2%
src/lib/agent/dashboard-ui.ts	85%	91%	+6%

---

_{Updated September 17, 2026 07:26 UTC}

[rsliter]

Blocking on exact head 1463a24: waitForStartedHermesGatewayProcess retries null, but null includes non-transient supervisor failures, not only a gateway that is still starting. That delays the failure and replaces the existing classified diagnostic with a generic settlement timeout. Please retry only the exact stopped-process result (false / SUPERVISOR_NOT_RUNNING) and let null continue immediately to the existing gateway verification or recovery path. Add a start-path regression test proving one probe and immediate classified failure for null; add the equivalent probe-only connect handoff test if that path can express it. The exact-head Advisor gate independently reports this as a P1 behavior blocker.

[senthilr-nv]

@rsliter Your blocking comment is addressed in 09fd42d0. NemoClaw now retries only false (SUPERVISOR_NOT_RUNNING); null continues immediately to the existing classified verification or recovery path. The start and probe-only connect regression tests cover this behavior, and the focused suite passes 32/32 tests. Please review the latest PR commit.

[github-actions]

PR Review Advisor finished for commit 5227620. Include the Advisor findings in the complete PR feedback collection. Verify and group valid findings before repair.

Request review only when Require no Advisor blockers is green.

All previous runs

[rsliter]

Approved at exact head 5227620 after reviewing the complete exact-head PR Review Advisor output. All nine specialists are clear, required CI passes, the settlement failure regression is covered, and local focused tests, repository checks, typecheck, and full PR validation pass. The Advisor's Hermes E2E recommendation is optional for this revision.