[kesslerio]

Related: #55

TL;DR

Each LLM card lists harness sessions from one occupancy helper on that machine. The dashboard reads the helper's cache on a timer. It does not open an ACP connection for each session or each poll. This is the forward-port of #55 onto current main, rebuilt around that single helper.

What Problem This Solves

Polling each harness from the dashboard would start a process per session, and an ACP client can slip into calls that create or resume a session. Hermes' ACP session/list can also write while it backfills a directory, and it returns no usable rows. The card still needs those sessions, with a finite age, and the dashboard should not run a shell on the box.

Why This Change Was Made

One helper process per host does the discovery. OpenClaw, OpenCode, and oh-my-pi answer only ACP initialize and session/list. Hermes and DeepSeek Harness keep private read-only adapters. The dashboard stores the helper URL and bearer token, pins each fetch to an allowlisted address, and drops a poll that finishes after disable or reconfigure. An ACP timestamp never becomes Generating or Stalled.

The helper now contains adapter startup failures, excludes its HTTP bearer from subprocess environments, and bounds ACP stdout frames and diagnostic history. Dashboard snapshots use cached configuration; malformed config retains the last valid copy, while mutations refuse to overwrite it. Helper response disconnects, timeouts and malformed envelopes fail the refresh.

User Impact

Settings, the command palette, and + Harness open a wizard: pick harnesses, one helper URL and token, then a route onto a Spark and port. Check shows coverage and a count, not titles. The card shows title, age, and context. Generating and Stalled appear only for a Hermes or DeepSeek Harness session the adapter marks running. Age is 1 hour to 14 days, default 12 hours.

Risk and Rollout

Nothing is listed until a helper is saved. The helper refuses 0.0.0.0 and ::, and it requires a bearer token off loopback. The dashboard rejects userinfo, redirects, and DNS answers outside the allowlist (loopback, plus SPARKDASH_HELPER_HOSTS). Helper tokens are stored apart from SSH passwords and LLM API keys. Spark monitor generation is unchanged. A helper HTTP error, including 401, comes back as a helper failure so the dashboard access-token dialog stays closed.

The maintainer still needs to decide whether to accept helper-token storage and operator-set outbound URLs. This update does not deploy or merge the change.

Evidence

• Integrated upstream main df6a22371d0bd7d25f9dd7486f7b8c5702a70fcd with an ordinary merge at 08442fd819ade404bb0b3513ccc1d8d59b484b7b. The previously published head ad6a51978353a88aaf89356976588c1c1cc20143 remains an ancestor. Conflicts in CHANGELOG.md, README.md, and package.json are resolved with current-main changes and the occupancy docs and test globs retained. Forge readback confirms this head on kesslerio/sparkDash:fm/sparkdash-pr55-harness-observability, based on df6a223, with mergeable: true (clean). No CI checks are configured.
• node --test scripts/occupancy-helper/*.test.js server/occupancy/__tests__/*.test.js: 70/70 passed. Coverage includes allowed loopback reads, denied bad bearers and all-interface binds, LAN HTTP bearer refusal, DNS pinning and redirect rejection, helper mutation rate limits, corrupt configuration, a failed helper beside a healthy one, response deadlines and body caps, child secret isolation, ACP bounds and owned-child cleanup, valid and malformed helper replies, Hermes empty and cancellable database reads, and DSH timeouts and cancellation. Allowed and denied tests include loopback may run without a token and reads do not spawn another bridge, a wrong bearer is denied and the body is not served, all-interface and unauthenticated reachable binds are refused before any spawn, a bearer is refused over LAN HTTP before any request and loopback HTTP remains allowed, forbidden DNS answers are rejected and a pinned connection ignores a later lookup, and helper PUT and DELETE share a principal mutation budget before writes or reconfiguration. Lifecycle and isolation tests include a missing adapter binary reports unavailable without crashing the helper, DSH aborts stalled headers and bodies within its total refresh budget, DSH rejects oversized streamed JSON and cancels the reader, real DSH stalled sockets close on deadline and a later list succeeds, a stalled DSH slot cannot block later ACP refreshes and close aborts its request, optional model lookups share the list deadline instead of restarting it, an existing empty Hermes directory is unavailable, a pending Hermes database read yields the event loop, a real Hermes database child yields to timers and is killed on cancellation, and oversized stdout terminates the owned real child and permits a fresh child.
• npm run typecheck, npm run build, and TZ=UTC npm run test:frontend passed. The frontend suite passed 655/655 tests in 85 files. The build still warns about large chunks.
• npm test stopped after the server suite returned 1,178 passed, 4 failed, and 16 skipped. Its && prevents the frontend command from running after that failure; the frontend suite passed separately above. The 4 failures are in test files unchanged from current main: the Intel GPU fixture removes a directory without recursive, two GPU-clock fake-sudo fixtures call /usr/bin/install (absent on this host), and Tool Eval's missing-uv message assertion fails. The broad server suite is not green.
• I did not run an independent wizard/list interaction and screenshot review for this integration. The click-through notes and images below are earlier implementer evidence, not independent QA. The browser CLI's chrome-devtools-mcp runtime is unavailable here, and I did not install a package to obtain it. No live profiles or helper credentials were used. Remote HTTPS deployment and descendant trees beyond the owned-child fixtures remain untested.
• Earlier implementer click-through covered the Spark page in light and dark themes at 1440px and 390px. It rejected empty harness selection and a URL with a username. A blank token showed "Helper answered HTTP 401" without opening the dashboard token dialog. A valid check showed "3 sessions" plus coverage without titles. Save listed the helper; the card showed ages for ACP rows and Stalled for a native Hermes row without working-directory text. Disable cleared the card and Enable restored it. Settings "1h" persisted as 3600000 ms. The palette command "Set up harness sessions" opened the wizard, and submitting the connection form advanced to Routes. No second person reviewed those flows, and there is no continuous screen recording.
• Browser shots of the wizard denial, the check step, and the session list:

[MiaAI-Lab]

Reviewed at cfde9dc. It merges cleanly and typecheck, build and tests pass, and I found no shell or argv injection. I am not merging it because it stores a new bearer token (helperTokens in the encrypted store) and makes the server fetch an operator-set URL, so it needs an explicit maintainer decision on that surface. Defects to fix first: (1) scripts/occupancy-helper/acpBridge.js startChild has no error handler, so a missing adapter binary (for example OMP_ACP_COMMAND unset or wrong) crashes the whole helper at start; (2) server/occupancy/poller.js:136 projection() runs from orderedSnapshots() every 2 s and reads and parses the helper config synchronously, and sources.js:25 throws on a corrupt file, so a bad occupancy file can break core snapshots; cache the projection and fall back to the last good config; (3) dshAdapter.js:50-60 fetch has no timeout, so one hung call blocks every adapter while the payload still looks fresh; add an abort signal and mark stale; (4) acpBridge.js:189 has no cap on the stdout buffer and mapRow does not truncate fields; (5) hermesAdapter.js:33-35 reports "supported" with zero rows when the state directory is empty, and spawnSync blocks the event loop; (6) server/occupancy/policy.js:16-60 allows any port on loopback and plain http to LAN hosts, so require https off loopback and consider allowlisting host:port; (7) the child process inherits the helper token through process.env; filter it.

(Review notes prepared with Claude Code on behalf of the maintainer.)

[MiaAI-Lab]

Re-verified at c4d0b68. It does not merge into main (conflicts in CHANGELOG.md, README.md, package.json, server/settings.js, src/api/types.ts) and it rewrites the same part of server/secretsStore.js as #164. Fixed since the first review: redirects are rejected, the connection is pinned to checked addresses, responses are size-capped, hosts are allowlisted and the token is cleared when the origin changes. Still open: acpBridge.js:164-186 has no error handler on the child (a missing harness binary crashes the helper); poller.js:400-407 and sources.js:30-43 re-read and parse the config on every snapshot and throw on a corrupt file, which stops the WebSocket broadcast; dshAdapter.js:302 fetches without a timeout; the ACP stdout buffer is uncapped (acpBridge.js:189); hermesAdapter.js:36-38 reports "supported" with no state and blocks on spawnSync (:202); policy.js:10,25 allows any loopback port and plain http to LAN hosts; the child inherits the helper token (acpBridge.js:303); and any JSON object counts as a healthy helper reply (policy.js:55-86). Because it stores a new bearer token and makes the server fetch an operator-set URL, it needs an explicit maintainer decision after those fixes. Leaving it open.

(Review notes prepared with Claude Code on behalf of the maintainer.)

[MiaAI-Lab]

Re-reviewed at 4672e2e. It is a rebase of c4d0b68 and merges cleanly with main now (typecheck and build pass; server 1102/1103 with one load-related timeout that passes alone, frontend 798), but the occupancy code itself is unchanged, so the earlier findings still apply: acpBridge.js:404-426 has no error handler on the adapter child, so a missing binary crashes the helper; poller.js:132-140 and sources.js:296-311 re-read and parse the config on every snapshot and throw on a corrupt file, which stops polling; pinnedFetch.js:213-245 never settles when the helper drops the connection mid-body, so one hung helper stalls every helper's refresh; the adapter child inherits OCCUPANCY_TOKEN, the ACP stdout buffer and sent list are unbounded, and a bare {} reply is accepted as a healthy helper (policy.js:55-86). Because it stores a new bearer token (helperTokens) and makes the server fetch an operator-set URL, it is not merged without an explicit maintainer decision even after these are fixed. Leaving it open for the author.

(Review notes prepared with Claude Code on behalf of the maintainer.)

[kesslerio]

The six failures in #208 (comment) are repaired at de49bba1a3850ef6a03fe18c31cbddf39e68d771.

• Adapter startup and stdin errors settle the refresh and close the owned child. A missing executable reports unavailable instead of crashing the helper.
• Snapshots use cached configuration. Corrupt config retains the last valid copy; mutations refuse to overwrite the damaged file.
• Interrupted response bodies settle, release abort listeners and close sockets. Requests have a whole-response deadline, including trickled bodies. A real healthy helper still refreshes beside the disconnecting fixture.
• Both ACP and Hermes database subprocesses exclude OCCUPANCY_TOKEN while keeping legitimate configuration. Tests inspect booleans without exposing the secret.
• ACP frames have a byte cap, RPC history retains 64 method names, and failed bridges preserve stale rows. Real disposable children exit after failure and stop; a subsequent child can list successfully.
• Health requires the existing version-1 envelope and valid adapter summaries. Real helper payloads remain accepted; {} and malformed replies fail.

The original published commit remains an ancestor. Current main 9437ce86396e053f77f8dc12c7ed0a95a90752ad was integrated with an ordinary merge, and the branch was pushed without rewriting history.

Validation: occupancy tests 59/59; UTC frontend 815/815; typecheck and build pass. The broad server run is not green: 1,117 passed, 33 failed, 1 skipped, with failures in unchanged collector/GPU-clock/tool fixtures and a WebSocket timing test that passes alone. The host-timezone frontend run has one unchanged UTC-date expectation failure. Commands and limits are in the updated PR body.

This repair leaves the wizard/list presentation unchanged. Independent UI interaction and screenshot QA remains unverified. Live profiles, HTTPS deployment and signal-resistant descendants were not tested. The decision on helper-token storage and operator-set URLs remains with the maintainer; nothing was deployed or merged.

[MiaAI-Lab]

Re-reviewed at de49bba; it merges cleanly with main, typecheck passes and the occupancy and helper tests pass (59/59). Fixed since the last review: the adapter child has error handlers (acpBridge.js startChild), the poller uses a cached config that keeps the last good copy (poller.js:24,137, sources.js:44-60), pinnedFetch settles on every path with a whole-response deadline (pinnedFetch.js:47-61), OCCUPANCY_TOKEN is removed from the child env (adapterEnv.js:1-6), ACP lines and sent are capped, and a bare {} is no longer healthy (policy.js:55-80). Still open: dshAdapter.js:61-70 rpc() calls fetchFn with no timeout or abort signal (and res.json() is unbounded), and the dsh slot sits inside refresh()'s Promise.all, so one hung DSH endpoint stalls every adapter's refresh while the cached payload still looks fresh; also killProcess sends SIGTERM only, so an adapter that ignores it leaks a process per failed refresh (add SIGKILL after a grace period), hermesAdapter.js:36-38 reports 'supported' for an empty state directory and :203 blocks on spawnSync, policy.js:9-34 allows plain http to LAN hosts so the bearer travels in clear, and PUT/DELETE /api/occupancy/helpers have no rate limiter. I found no SSRF, redirect or DNS-rebinding problem in the pinned fetch. Because it stores a new bearer token (helperTokens) and makes the server fetch an operator-set URL, it is not merged in this pass; leaving it open for the author and the maintainer's decision.

(Review notes prepared with Claude Code on behalf of the maintainer.)

[kesslerio]

The six remaining findings are fixed at ad6a51978353a88aaf89356976588c1c1cc20143.

I merged upstream main be35be9f594c65a3324f1d97ace58668d3a22068 with an ordinary merge and pushed fast-forward. Both previously published heads remain ancestors. A fresh forge read reports this head, the expected fork/branch, and mergeable: true (clean).

• DSH headers, streamed bodies and optional model RPCs share an 8-second refresh budget; each response is capped at 1 MB. Tests: DSH aborts stalled headers and bodies within its total refresh budget, DSH rejects oversized streamed JSON and cancels the reader, real DSH stalled sockets close on deadline and a later list succeeds, and optional model lookups share the list deadline instead of restarting it.
• Each adapter owns its refresh guard. Tests: a stalled DSH slot cannot block later ACP refreshes and close aborts its request and DSH timeout preserves stale rows and a later successful refresh replaces them.
• ACP cleanup escalates its owned process group from SIGTERM to SIGKILL after 250 ms. The existing oversized stdout terminates the owned real child and permits a fresh child regression now uses real SIGTERM-resistant children for failure, recovery and stop.
• An empty Hermes directory is unavailable; a real empty session store is supported. SQLite reads run asynchronously with output/time bounds and explicit cancellation. Tests: an existing empty Hermes directory is unavailable, a pending Hermes database read yields the event loop, and a real Hermes database child yields to timers and is killed on cancellation. The last test exposed that Node's automatic abort could leave a resistant child alive; explicit owned-child SIGKILL fixes that path.
• A bearer cannot travel over HTTP to any resolved non-loopback address. Test: a bearer is refused over LAN HTTP before any request and loopback HTTP remains allowed. Allowlisting, DNS pinning and redirects retain their existing tests.
• PUT and DELETE share a 30/minute principal/IP budget, enforced before persistence or reconfiguration. Test: helper PUT and DELETE share a principal mutation budget before writes or reconfiguration; auth tests also deny both mutations without the dashboard token.

Validation on this tree:

• node --test scripts/occupancy-helper/*.test.js server/occupancy/__tests__/*.test.js: 70/70 passed. New regressions failed before the corrections. Existing missing-binary, secret isolation, malformed-envelope, age, epoch and disconnect cases still pass.
• npm run typecheck and npm run build: passed; the build still warns about large chunks.
• TZ=UTC npm run test:frontend: 827/827 passed in 104 files.
• timeout -k 5s 120s npm run test:server: exit 124; 1,144 passed, 10 failed, 1 cancelled, 1 skipped. Failures are in unchanged Intel/GPU-clock/tool fixtures; the VulnHunter suite was interrupted at the deadline. This broad run is not green or complete. No validation descendants remain in the task worktree.

This repair does not redesign the wizard/list presentation. Independent wizard/list interaction and screenshot QA remains unverified, as previously disclosed. Live harness profiles, remote HTTPS deployment, and descendant trees beyond the owned child fixtures were not exercised. The decision on helper-token storage and operator-set outbound URLs stays with the maintainer. Nothing was deployed or merged.

[MiaAI-Lab]

Status at ad6a519: no new commits since the 2026-10-19 re-review; the author's latest comment lists the 8-second DSH refresh budget, the 1 MB response cap and per-adapter refresh guards. The code is not merged in this pass because it stores a new bearer token (helperTokens in the encrypted store) and makes the server fetch an operator-set URL, which is a secrets and network-exposure decision for the maintainer. The author's note that the broad server run is not green (10 failures in unchanged Intel/GPU-clock/tool fixtures) should be re-run against main (448944c) before any decision. Leaving it open.

(Review notes prepared with Claude Code on behalf of the maintainer.)

[MiaAI-Lab]

The current helper and occupancy tests pass 70/70 in a scratch worktree, while this branch has three conflicts with main 00151b5. The HTTPS policy, bounded whole-response fetch, and shared adapter deadline address earlier findings (server/occupancy/policy.js:40, server/occupancy/pinnedFetch.js:98, scripts/occupancy-helper/dshAdapter.js:29). Helper bearer storage and operator-configured outbound URLs meet this triage pass's secrets/network stop conditions. Leaving this open for a maintainer decision and rebase. (Review notes prepared with Claude Code on behalf of the maintainer.)

[MiaAI-Lab]

The updated branch now reconciles cleanly with current main, and all 70 focused helper/occupancy tests pass. The HTTPS policy, bounded response handling and shared adapter deadline address the earlier findings (server/occupancy/policy.js:40, server/occupancy/pinnedFetch.js:98, scripts/occupancy-helper/dshAdapter.js:29). Helper bearer storage and operator-configured outbound URLs still meet this pass's secrets/network stop conditions. Leaving this open for a maintainer decision. (Review notes prepared with Claude Code on behalf of the maintainer.)

[MiaAI-Lab]

Now on main as 94cef5e (merge of the branch, authorship preserved; CHANGELOG credits @kesslerio). Approved for merge by the maintainer after the secrets/network review: helper URLs pass the allowlist and are pinned to the resolved addresses (server/occupancy/policy.js:40-60), redirects are refused (pinnedFetch.js:95), bearer tokens need HTTPS outside loopback and live in the encrypted store. The conflicts with the MQTT, i18n and GPU-clock work were resolved by hand; the Settings test moved to SettingsDialog.occupancy.test.tsx and the shared fixtures gained the new fields. Typecheck, build and the full suites pass (server 1302 pass / 0 fail, frontend 716 pass). Closing as completed. (Review notes prepared with Claude Code on behalf of the maintainer.)