feat: Set GPU cap across fleet in Settings + show on dash by danieljohnmorris · Pull Request #218 · MiaAI-Lab/sparkDash

danieljohnmorris · 2026-10-14T18:58:16Z

Adds fleet-wide GPU clock control in Settings, extending the existing GpuClockManager and generated on-unit programs. Ready for maintainer review; not merged or deployed.

Changes

Settings offers Full capacity and 1800–2400 MHz presets. Save applies GPU changes before storing settings. Failed GPU operations retain the requested state for retry, including Keep after reboot at an unchanged frequency. A failed settings write after GPU success retries settings without reapplying hardware.
Persistence uses the systemd service where sudo reaches a root shell, otherwise a tagged user @reboot job. Cron persistence requires the exact NVIDIA command to work without a password or cached sudo authentication. Authorized NVIDIA rejection retains its clock-error category.
Apply, Remove and newly installed reboot jobs share a per-user util-linux flock, serializing token checks, GPU commands and marker updates. Unreadable crontabs fail status and prevent unverified reboot jobs from changing clocks or markers.
Turning off Keep after reboot restores the cap marker after service teardown and re-applying the cap, fixing Mia’s reported defect. Failed service-file removal restores the live cap and marker or reports the specific restoration failure. Inaccessible old markers refuse changes before GPU mutation. Unrelated cron entries are preserved; all-skipped batches return success:false/applied:0.
The cron boot command now fits Debian cron’s command-field limit without dropping locking, token checks, retries or failure reporting. Sandbox crontab installs enforce that limit. Zero completed fleet requests with errors warn that changes may already have applied, rather than claiming nothing changed.
GPU-clock shell fixtures isolate HOME and PATH in every subprocess, including manual and password-mode commands.
Fixed the launcher-output failures exposed by the full suite: Start and live Output now await GNU tail --pid draining after the process group exits, rather than killing the follower after a fixed delay. Final unterminated UTF-8 output and the real exit code are preserved, including marker-like script output.

Published head and merge state

Head b624d89 includes current upstream main 458cd06. GitHub REST reports mergeable:true / mergeable_state:clean for those exact SHAs. This supersedes the earlier conflict warning and blocked full-server results in prior comments. No force-push or PR close/reopen was used. The PR remains open; it has not been merged into upstream.

Verification at this head

Isolated non-root Linux arm64 Node22.23.3 container, with zsh installed and init reaping detached children:

npm run typecheck — passed.
npm run build — passed; existing >500kB chunk warning remains. No lint script is configured.
npm run test:server and npm run test:frontend — passed at default concurrency: 1168/1168 server tests, 688/688 frontend tests across 84 files, 0 skipped. No narrowed glob or concurrency override. One stale wording-focused test and one obsolete copy assertion were removed rather than re-pinned; retained tests cover retry and settings-write behavior. React act warnings remain.
Real Debian cron3.0pl1-162 smoke — the old 1189-byte command was rejected as “command too long”; the new 834-byte command installed and listed unchanged (exit0, roundTrip:true).
Final actual-component browser smoke — a cap applied on both units before hook-install failure; the summary warned that changes may already have applied, Apply returned false, pending remained true, Keep stayed selected, and refreshed status showed 1800MHz on both units. No live fleet writes.
Direct generated-command runtime smoke — Start and a concurrently attached live Output both delivered the final unterminated UTF-8 line before exit code 4; status reported STOPPED and removed the stale PID file. The delayed-follower regression was observed failing before the fix and passing afterward.
Actual browser smoke for the unchanged GPU/UI paths — same-frequency persistence changes and failed-hook retry; GPU success plus a failed settings PUT kept Save enabled, and retry completed with one GPU operation and two settings requests. API fixtures were isolated; no live fleet writes.
Generated GPU shell sandboxes cover marker restoration, hook failures and stale/racing reboot jobs. A real sudo1.9.13 policy smoke rejected password-only persistence despite cached authentication, then accepted a narrow NOPASSWD rule and executed the generated boot command and Reset.

Review and documentation

All three independent review perspectives returned and reviewed the final affected deltas: code correctness, silent failure, and consumer-visible test coverage. Two confirmed P2 findings from the final review (cron command length and misleading zero-completion summary) are fixed; no confirmed findings remain. Astra-backed read-only workers substituted after reviewer/generic Anthropic agents failed with provider refusal/authentication errors. Results are recorded in the handoff comment. TypeScript-specific review/JSDoc skills were unavailable; type/runtime contracts and public API/JSDoc are reviewed explicitly. No separate adversarial pass was requested.

README, CHANGELOG and CODEBASE document the behavior, prerequisites and verification conventions, including util-linux flock and GNU tail --pid.

Limits and maintainer decision

No physical GPU, real systemd boot or physical reboot verification. This still changes privileged and boot-time behavior, so Mia’s approval is required. Existing low-priority limitations remain: reboot-failure detail is shown in the unit dialog, and custom exact-list sudoers rules must allow the documented 2400MHz status probe. Existing dependency advisories were not changed. No merge or deployment performed.

danieljohnmorris · 2026-10-14T20:56:46Z

Pushed a fix for the case the reviewer flagged as the real-world blocker on this fleet: keeping the cap after reboot demanded NOPASSWD: ALL, so on a box whose sudoers rule covers only nvidia-smi -lgc/-rgc (the shape sparkLimiter installs) the whole apply was refused before the GPU was touched — nothing capped, error in the row.

What changed

persistKind (unit | cron | none) is computed on the unit and reported through status/apply, alongside both hooks' state (service.*, cron.*).
Where sudo reaches a root shell, the oneshot spark-gpu-clock-limit.service is still used. Where it does not but cron runs, the cap is kept by an @reboot line in the SSH user's own crontab (tagged # sparkdash-gpu-clock-cap) that runs the very same sudo -n nvidia-smi -lgc 0,MAX the cap itself ran — no more privilege than capping needs — and refreshes the marker so the row still reads the cap after a boot.
Both surfaces (Settings → GPU clock cap, Edit unit → GPU clock cap) offer and default the tick from that real capability and name the hook on Save (kept after reboot by the boot service / by the @reboot cron job, per unit). A unit with neither hook disables the tick with the reason inline instead of refusing on Apply.
A refused cap still leaves an existing service/cron line/marker untouched; reset removes both hooks and the marker.

Verified

tsc clean, 543/543 vitest, 40/40 server/gpuclock node tests (the two unrelated tooleval failures reproduce on pristine main on macOS — mapfile is bash 4+).
Live on a 3-unit Spark fleet whose only passwordless rule is the narrow nvidia-smi one: apply 1800 MHz + keep after reboot → { ok: true, capMHz: 1800, persistKind: "cron" } on all three, with the crontab line installed; the line was then executed exactly as cron would (sh -c "$(crontab -l | sed 's/^@reboot //')") → cap applied, marker written, exit 0; then the fleet was restored and the test hooks removed.

MiaAI-Lab · 2026-10-15T20:17:02Z

Re-verified at fd88023. It no longer merges: GpuClockManager.js and commands.js conflict (plus CHANGELOG.md and README.md), because main added the sudo-password mode and the manual command to server/gpuclock after this PR's base; a rebase must keep that mode working with the new can_sudo_root probe. Request values reach the unit only as validated integers, the fleet route sits behind the auth middleware with the destructive limiter, and the cron hook edits only its own tagged line and refuses to write an unreadable crontab. To fix: values substituted into the @reboot line from the unit environment ($HOME, the sudo path) should be validated like the fixed paths; a boot-time failure of that job is silent apart from a marker mismatch; the fleet route reports success: true for units skipped as unsupported, so a fleet where nothing was capped can look successful; and the documented narrow sudoers rule nvidia-smi -lgc * should be an anchored pattern instead of a wildcard. It changes privileged behaviour, so it needs a maintainer decision after the rebase; leaving it open.

(Review notes prepared with Claude Code on behalf of the maintainer.)

Settings → GPU clock cap applies a preset (1800–2400 MHz, every 100 MHz) or Remove cap to every Spark in turn, reusing the per-unit GpuClockManager and its GET/PUT/DELETE /api/sparks/:id/gpu-clock semantics, so the sudo and boot-service behaviour is identical to the Edit unit dialog. POST /api/sparks/gpu-clock-all takes { maxMHz, persist? } or { reset: true }, charges the destructive limiter once for the batch, looks up a single shared manager so a fleet apply serialises against an in-flight per-unit change, and returns one result per unit. Units that cannot take a cap at all (hosts, Windows/macOS units, no nvidia-smi) come back as skipped with a reason rather than as errors, and a failure names the unit instead of being counted as a success; the Settings row reports capped/skipped/failed counts and the reasons.

A status read with no deadline left the row on 'Reading the fleet…' for good after one transient hang (a restarting dev server, an unreachable Spark), so nothing was ever marked and the states could not be used. Each unit read now has its own deadline; the ones that answered mark the spoken-for state and the rest are counted on the same line ('2 of 3 unreadable'), so a partial fleet is never shown as the whole truth. The Settings help line no longer mentions the removed 'Remove cap' button.

@reboot

Installing the boot service needs a root shell, so on a fleet whose passwordless rule covers only `nvidia-smi -lgc`/`-rgc` the whole apply was refused before the cap was touched: nothing was capped and the row showed an error. Where sudo cannot reach a root shell, keep the cap with an `@reboot` line in the SSH user's own crontab that runs the very same `sudo -n nvidia-smi -lgc` the cap itself ran - no more privilege than capping needs. cron also refreshes the marker, so the row still reads the cap after a boot. The unit's status now reports which hook it can take (`persistKind`: unit | cron | none, plus the state of both hooks), Save and the fleet row name the hook that took (`kept after reboot by the boot service` / `by the @reboot cron job`), and both surfaces offer - and default - the tick from that real capability instead of a bare "kept after reboot". A unit with neither hook disables the tick with the reason inline rather than refusing on Apply. A refused cap still leaves an existing service, cron line and marker exactly as they were; reset removes both hooks.

@reboot

… a time - crontab is read once; only 'no crontab for' counts as empty, any other read failure refuses before anything changes (apply and reset) - a persist apply removes the other hook (cron line under the service, refuses when a service it cannot remove would keep re-applying its cap) - status offers the service only where systemd runs; root without systemd falls back to the @reboot job - fleet reset never reports 'kept after reboot'; fleet success is false when any unit failed; stale fleet reads cannot overwrite a newer one

… and Settings save gate

danieljohnmorris · 2026-10-16T14:01:53Z

@MiaAI-Lab Hi, merge conflicts have been addressed. Please re-review

MiaAI-Lab · 2026-10-16T17:41:49Z

Re-reviewed at 71de711. It now merges cleanly with main, and typecheck, the server/gpuclock tests (63) and the Settings/fleet frontend tests pass. Because it changes sudo/root and boot-time behaviour it will not be merged by us in this pass; these are the things to fix first. (1) routes.js:163 returns success: true when every unit was skipped as unsupported, and FleetGpuClock.tsx:186-199 then shows a green "Capped" message with 0 units capped; report success: false (or an applied count) when nothing was capped and add an all-skipped test. (2) A failure of the @reboot job is silent (commands.js:38): no log, and the status only shows a stale marker while cron.installed stays true; log through logger or write a failure marker that status reports. (3) The README's narrow rule (README.md:692, nvidia-smi -lgc *) is a sudoers wildcard that also matches extra words, so other nvidia-smi options can ride along; document an anchored pattern (sudoers 1.9.10+ regex form) or the exact per-preset strings, and adjust the sudo -l probe at commands.js:220 to match. (4) Hardening: values taken from the unit environment ($HOME, the sudo path, $ALT_DIR at commands.js:200-201,415-417) go into the crontab line without the absolute-path check the fixed paths get, and the grep -v "$CRON_TAG" at :259,:419 is unanchored; validate them and fail with cron-failed. (5) With persist: false the service cleanup can fail after the cap was applied, leaving a stale boot unit, and the fleet loop has no overall deadline. Leaving it open for the author.

(Review notes prepared with Claude Code on behalf of the maintainer.)

MiaAI-Lab · 2026-10-18T20:47:33Z

Status at 71de711: no new commits since the 2026-10-16 review, and it still merges cleanly with main. It changes sudo/root and boot-time behaviour on the units, so it is not merged in this pass. Still to fix: routes.js:163 reports success: true when every unit was skipped and FleetGpuClock.tsx:186-199 then shows a green 'Capped' for zero units; a failure of the @reboot job is silent (commands.js:38); and values substituted into the @reboot line from the unit environment must be validated or quoted. Leaving it open for the author.

(Review notes prepared with Claude Code on behalf of the maintainer.)

@reboot

Merge: keep both sides of the CHANGELOG conflict. Review fixes (MiaAI-Lab, 2026-10-16 and 10-18): - Fleet route: success is false when no unit was actually capped or reset (all skipped), answers with an applied count, and has a 3 minute budget; the fleet row reads 'Nothing changed' instead of a green 'Capped'. - @reboot job: three tries, then a failure marker plus a logger line that status reports (cron.failed) and the unit dialog shows. Cleared by the next Apply or Remove. - Values from the unit that go into the crontab line (sudo path, HOME) must be plain absolute paths or the change is refused before the GPU is touched. - The crontab line is matched by the tag at its end, not anywhere. - A cap without keep-after-reboot is recorded before old hooks are removed, and a hook that cannot be removed is named in the error. - README: anchored sudoers rule (regex or exact commands) replaces the 'nvidia-smi -lgc *' wildcard; checked against sudo 1.9.17. The sudo -l probe asks -lgc 0,2400, which both forms allow.

danieljohnmorris · 2026-10-18T21:30:15Z

@MiaAI-Lab hi, ready for re-review

MiaAI-Lab · 2026-10-19T19:55:51Z

Re-reviewed at 27ad9ea; it merges cleanly with main, typecheck passes, server/gpuclock tests 70/70 and the frontend suite pass. The earlier findings are fixed: an all-skipped fleet now reports success: false with an applied count (routes.js:175-178, FleetGpuClock.tsx:186-202), a failed @reboot job leaves a failure marker and a logger entry that status reports (commands.js:41-47, :314), the README sudoers rule is anchored (README.md:699-712), and the paths substituted into the cron line are validated by safe_path (commands.js:223, :368-374). One new defect: in the persist: false branch write_marker runs at commands.js:393, then drop_service (:398) stops the unit, whose ExecStopPost=/bin/rm -f marker (:150) deletes it, and the re-applied cap at :399 is not recorded again, so after turning off 'Keep after reboot' on a unit that had the service the GPU is capped but status shows no cap; write the marker again after that second lgc and add a sandbox test where the fake systemctl stop removes it. It still changes sudo/root and boot-time behaviour on the units, so it is not merged in this pass and stays open for the maintainer's decision; remaining low points are that the boot-failure alert shows only in the unit dialog (not the fleet row or Activity) and that a -lgc 0,2400 probe reads a custom exact-list sudoers rule without 2400 as no-sudo.

(Review notes prepared with Claude Code on behalf of the maintainer.)

MiaAI-Lab · 2026-10-21T16:14:27Z

Status at 27ad9ea: no new commits since the 2026-10-19 re-review. It changes sudo/root and boot-time behaviour on the units, so it is not merged in this pass. One defect remains to fix: in the persist: false branch write_marker (commands.js:393) is wiped by drop_service's ExecStopPost (:150), so the re-applied cap at :399 has no marker and status shows no cap; write the marker again after the second lgc and add a sandbox test for it. Leaving it open for the author and the maintainer's decision.

(Review notes prepared with Claude Code on behalf of the maintainer.)

danieljohnmorris

Reviewed published head 77dd9ce against base 9437ce8. The requested service-stop marker-loss defect is fixed; the regression models both ExecStop clock reset and ExecStopPost marker deletion and would fail without the new post-lgc marker write. The full PR is not clear of findings.

Confirmed findings

P2 — Reset can unlock the GPU but leave status reporting the old cap. commands.js:474–475. Trigger: an existing current-boot root-owned marker, no remaining boot service, and sudo narrowed to the nvidia-smi commands. Reset runs -rgc, then cannot remove the root marker. Reproduced in an isolated Linux non-root sandbox: RESULT=error / CODE=marker-failed; GPU unlocked=true; subsequent status CAP=2100; root marker survives. Correction must prevent stale status while accurately reporting partial completion, or refuse before changing clocks when cleanup is known to be impossible. Suppressing the error alone would leave the status lie unresolved.
P2 — Failed boot-hook changes can lose the Save/retry state. FleetGpuClock.tsx:136–138. If applying the live cap succeeds but installing/removing its boot hook fails, the status refresh can match the selected cap. pending then becomes false despite the unsuccessful operation, so Save cannot retry it (or, with other dirty settings, skips retry). Reproduced through the actual component harness: applySucceeded=false, refreshed cap=1800, boot-service failure displayed, pendingForSave=false. Preserve unsuccessful operation intent independently of live-cap equality until retried successfully or cancelled. Existing partial-failure tests leave refreshed status at the old cap and miss this transition. Temporary reproduction file removed.

P3 presentation: FleetGpuClock.tsx:94–108 counts unsupported Darwin/no-NVIDIA units as unreadable. Windows is not a trigger because registry normalization forces kind=host. Unverified risk: duplicate registry IDs targeting the same machine can bypass per-ID operation exclusion.

Reconciliation

Three separate read-only reviewers covered code, failure handling, and test coverage, using the available reviewer agent instead of the reference vendor-specific agents. Duplicate findings merged and P1 ratings reduced to P2 for the bounded triggers above. Rejected the suggestion to let all-skipped batches unblock Save as successful: the maintainer explicitly required zero applied units not to count as success. Also rejected a claimed missing error explanation (the fleet row displays it) and a password-capability finding (the fleet operation has no password input). TypeScript-specific review skills were unavailable; API/runtime types and public JSDoc were covered in the code pass. No separate adversarial pass was requested.

Verification and limits

Node 22: npm run typecheck; npm run build; npm run test:frontend — pass, frontend 830/830. Build has the existing >500 kB chunk warning; no lint script configured.
node --test server/gpuclock/tests/*.test.js — 70/70 pass. Separate generated-unit shell smoke executes the actual stop hooks and confirms persistent 1800 -> temporary 2000 with CAP=2000 and no service. Seven generated shell variants pass /bin/sh -n.
Full Linux non-root server suite: 1117 passed, 1 failed, 1 skipped. The launcher exit-marker assertion also fails on base9437ce8; zsh unavailable. This is not an all-green full suite. Earlier macOS/root-container runs were unsuitable for several integration paths.
Interactive/visual Settings smoke blocked by browser runtime frame/screenshot failures; only DOM/accessibility rendering, current-fleet state, and initial Save-disabled state observed with isolated API fixtures. Existing frontend tests do not replace visual proof.
No live GPU/reboot verification, sudo changes, merge, or deployment. GitHub currently reports merge conflicts. CHANGELOG and PR description document the marker fix and verification.

The two broader P2 findings remain unchanged rather than expanding the requested marker-only fix.

MiaAI-Lab · 2026-10-22T19:38:28Z

The new head 77dd9ce repairs the service-stop marker loss (server/gpuclock/commands.js:397-400), and the focused clock/fleet/route suites pass 70/70. The latest submitted review still identifies reset/status consistency and failed-operation retry problems (server/gpuclock/commands.js:474-475, src/components/FleetGpuClock.tsx:136-138) that need correction and regression coverage. This also changes sudo/root and boot behavior, meeting this pass's privilege stop condition. Leaving this open for those repairs and the maintainer's privilege decision. (Review notes prepared with Claude Code on behalf of the maintainer.)

… jobs

danieljohnmorris

Reconciled published head 689b69a. This supersedes my review at77dd9ce.

All confirmed PR findings are fixed: pre-mutation marker permissions; failed operation retry/persistence intent; unreadable matching selections; per-unit unavailable hook unticking; root/fallback marker migration; Full capacity removing stranded hooks; delayed reboot-job invalidation. Three independent review perspectives rechecked the final changes with no remaining actionable findings. README/CHANGELOG and the PR description now reflect the implementation and evidence.

GitHub reports MERGEABLE / CLEAN after merging upstream00151b5. Typecheck/build pass, frontend 667/667. Generated shell/manual-command runtime smoke and actual Settings browser failure/retry/reset smoke pass with isolated fixtures.

Not an all-green server suite: Linux non-root server run is 1108 passed,1 failed,1 skipped. The sole failure is the launcher output-forgery assertion. Its whole subsystem is byte-identical to latest upstream, but current upstream passes that assertion and its full-suite comparison timed out later; latest-base reproduction is not established. zsh unavailable. No real GPU/reboot verification, merge, or deployment. These verification limits and the maintainer decision on privileged behavior remain; no blanket approval is asserted.

Available reviewer agents replaced vendor-specific review roles; TypeScript skill files unavailable, contracts/types/JSDoc reviewed explicitly. No separate adversarial pass requested.

danieljohnmorris

Final published revision b3fb48b, merged with newly advanced upstream df6a223. The fresh changelog conflict is resolved preserving both sides. GitHub reports MERGEABLE / CLEAN.

All three independent reviewers reaffirmed no remaining actionable findings in the PR integration delta. All confirmed GPU-clock findings are fixed; no GPU-clock runtime/UI/test changes were needed in this final upstream merge. README/CHANGELOG and PR description are current.

Re-run results at b3fb48b: typecheck/build pass; frontend 678/678 in84files. Linux non-root server suite: 1142 passed,2 failed,16 skipped. Failures are the unchanged launcher output assertions at llmlaunch/tests/launcher.test.js:360 and api.test.js:117; skips require zsh. Prior generated-shell and actual Settings browser smokes exercise unchanged GPU-clock code. This is not an all-green server-suite claim; latest-base failure reproduction was not established, and no unrelated launcher fix was added. Full suite qualification, hardware/reboot limits and maintainer approval remain explicit in the description. No merge or deployment.

MiaAI-Lab · 2026-10-23T10:50:01Z

The updated head addresses the previously reported reset/status and retry issues, including marker preflight and keeping persistence changes pending (server/gpuclock/commands.js:509, src/components/FleetGpuClock.tsx:163, src/components/FleetGpuClock.tsx:235). The focused clock/fleet/route suites pass 76 tests, and the focused UI suites pass 45 tests. This still changes sudo/root and boot behavior, meeting this pass's privilege stop condition, and has a CHANGELOG conflict with current main. Leaving this open for the maintainer's privilege decision and reconciliation. (Review notes prepared with Claude Code on behalf of the maintainer.)

danieljohnmorris · 2026-10-23T13:17:03Z

@MiaAI-Lab Ready for your review at b624d89. GitHub reports mergeable:true / clean against main 458cd06; the CHANGELOG conflict is resolved preserving upstream changes.

Your marker/reset/status and persistence-retry findings are addressed. Final independent code-correctness, silent-failure and test-coverage reviews are reconciled, including review of the final affected deltas; no confirmed findings remain. Astra-backed read-only workers substituted after Anthropic reviewer/provider authentication failures. TypeScript-specific review/JSDoc skills were unavailable; type/runtime contracts and public JSDoc were reviewed explicitly. No separate adversarial pass was requested.

Two further P2 defects found in that final review are fixed:

Debian cron rejected the old 1189-byte boot command; the shortened 834-byte command installs and lists unchanged in real cron3.0pl1-162, retaining locking, install-token checks, retries and failure reporting. The sandbox fixture now enforces the command-length limit.
A fleet-wide hook failure after applying caps no longer claims “Nothing changed”. Actual browser exercise confirmed the warning, live 1800MHz caps, failed Apply result and retained pending/Keep state.

Verification of the final code in isolated non-root Linux arm64 / Node22.23.3, at default test concurrency: npm run typecheck, npm run build, npm run test:server (1168/1168, zero skips), npm run test:frontend (688/688,84 files). Frontend count decreased by one because a stale copy-focused test was removed, not re-pinned; Save/retry/settings-write behavioral coverage remains. Build chunk and React act warnings remain; no lint script is configured. Launcher Start/live Output smoke also preserves final unterminated UTF-8 output before exit4 and reports STOPPED with PID cleanup. README, CHANGELOG, CODEBASE and the PR description are updated.

MiaAI-Lab · 2026-10-24T13:05:21Z

Re-checked the new head b624d89 after your update: it merges cleanly with current main, typecheck passes, and the full suites pass (server 1152 pass / 0 fail / 16 skipped, frontend 688 pass). The marker, reset/status and persistence-retry fixes from the earlier review are present (server/gpuclock/commands.js, src/components/FleetGpuClock.tsx). The change still adds sudo/root and boot (cron) behaviour for the clock cap, which is a privilege stop condition, so it is not merged in this pass. Leaving it open for the maintainer's privilege decision. (Review notes prepared with Claude Code on behalf of the maintainer.)

MiaAI-Lab · 2026-10-24T13:43:16Z

Now on main as edfce58 (merge of the branch, so every commit and its authorship is preserved; CHANGELOG credits @danieljohnmorris). The merge needed two small hand resolutions, in the CHANGELOG and in the realManager test helper shared with #264. It was approved for merge by the maintainer after the privilege review (server/gpuclock/commands.js, src/components/FleetGpuClock.tsx). Typecheck, build and the full suites pass on top of main (server 1190 pass / 0 fail, frontend 702 pass). Closing as completed. (Review notes prepared with Claude Code on behalf of the maintainer.)

The drain change from #218 passes GNU-only --pid to tail, so on macOS the follower exited at once and a Start printed nothing and never emitted the exit marker, undoing the #264 fix. Probe for --pid and otherwise use the earlier timed shutdown. Regression test uses a tail that rejects --pid. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

78 upstream commits: MQTT publishing (MiaAI-Lab#248), harness occupancy helpers (MiaAI-Lab#208), GPU clock cap from Settings (MiaAI-Lab#218), settings i18n foundation (MiaAI-Lab#226), configurable GPU temperature limits (MiaAI-Lab#273), Locals/Splash engines, TensorFold cache-counter fixes (MiaAI-Lab#252/MiaAI-Lab#260/MiaAI-Lab#272) and the zsh-safe remote probes (MiaAI-Lab#255/MiaAI-Lab#257). Conflict resolutions: - server/collectors/LlmProbe.js + its TensorFold test: took main's implementation, which supersedes this fork's /health `live` + /metrics work with the cache-counter precedence, running/waiting aliases, histogram validation and prefill holds. This drops the fork's parallel /metrics fetch (micro-optimisation only). - SystemCollector: main's zsh-safe find probes merged into the fork's build/parse refactor, keeping the fork-only RAPL package-power reads; _getRemoteNetwork now uses the injected executor (the merge had left the fork body calling sshExec, which made main's zsh-glob corpus test run real SSH). - OverviewPage: FleetKpis keeps the fork's placement but takes all sparks, so hidden cluster workers count in the KPI tiles (main's fix). - SparkPage: main's column layout (History in the hardware column, storage/network in the right column) over the fork's full-width History panel. - SettingsDialog: fork VulnHunter/REA opt-in toggles kept, metrics-history toggle switched to main's i18n keys. - index.js: fork vulnHunt/rea spark cleanup kept alongside MQTT removeUnit/applySettings. - CHANGELOG/README/package.json: both sides' entries kept; test globs unioned so the fork-only models/autopower/vulnhunt suites still run.

danieljohnmorris mentioned this pull request Oct 14, 2026

feat: GPU clock cap — show it on the cards, control it fleet-wide in Settings #217

Closed

danieljohnmorris closed this Oct 14, 2026

danieljohnmorris reopened this Oct 14, 2026

danieljohnmorris changed the title ~~feat: cap the GPU clock on the whole fleet from Settings~~ feat: Set GPU cap across fleet in Settings + show on dash Oct 14, 2026

danieljohnmorris force-pushed the pr/clock-cap branch 2 times, most recently from f45ff64 to bc71640 Compare October 14, 2026 19:23

danieljohnmorris changed the title ~~feat: Set GPU cap across fleet in Settings + show on dash~~ feat(gpu): cap the GPU clock on the whole fleet from Settings Oct 14, 2026

danieljohnmorris changed the title ~~feat(gpu): cap the GPU clock on the whole fleet from Settings~~ feat: Set GPU cap across fleet in Settings + show on dash Oct 14, 2026

danieljohnmorris force-pushed the pr/clock-cap branch 2 times, most recently from fb8bf32 to b865351 Compare October 14, 2026 19:51

MiaAI-Lab mentioned this pull request Oct 15, 2026

feat: clock-cap addon v2 — rebased onto main, review fixes in #209

Closed

danieljohnmorris added 6 commits October 16, 2026 00:21

docs(gpu-clock): one boot hook at a time; unreadable crontab refuses

267281b

test(gpu-clock): cover hook refusals, cron read-back, partial batches…

71de711

… and Settings save gate

danieljohnmorris force-pushed the pr/clock-cap branch from fd88023 to 71de711 Compare October 15, 2026 23:42

fix(gpu-clock): restore cap marker after removing boot service

77dd9ce

danieljohnmorris commented Oct 22, 2026

View reviewed changes

fix(gpu-clock): preserve reset and retry invariants; merge upstream main

a59248e

fix(gpu-clock): reconcile boot hooks and invalidate superseded reboot…

689b69a

… jobs

danieljohnmorris commented Oct 22, 2026

View reviewed changes

merge: integrate latest upstream and preserve GPU clock release notes

b3fb48b

danieljohnmorris commented Oct 22, 2026

View reviewed changes

danieljohnmorris added 2 commits October 23, 2026 10:23

Merge upstream main into GPU clock PR

24fcfce

fix(gpuclock): preserve persistence intent and serialize boot jobs

ba1366b

danieljohnmorris added 4 commits October 23, 2026 11:58

fix(gpuclock): retain clock errors and settings retry state

39751f2

Merge upstream main and reconcile changelog

bd19dda

fix(llmlaunch): drain log follower before reporting completion

cee3d7f

fix(gpuclock): fit cron boot hook and report partial changes

b624d89

MiaAI-Lab merged commit edfce58 into MiaAI-Lab:main Oct 24, 2026

MiaAI-Lab mentioned this pull request Oct 24, 2026

feat(mqtt): publish events, availability and metrics to an MQTT broker #248

Merged

4 tasks done

Conversation

danieljohnmorris commented Oct 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Published head and merge state

Verification at this head

Review and documentation

Limits and maintainer decision

Uh oh!

danieljohnmorris commented Oct 14, 2026

Uh oh!

MiaAI-Lab commented Oct 15, 2026

Uh oh!

danieljohnmorris commented Oct 16, 2026

Uh oh!

MiaAI-Lab commented Oct 16, 2026

Uh oh!

MiaAI-Lab commented Oct 18, 2026

Uh oh!

danieljohnmorris commented Oct 18, 2026

Uh oh!

MiaAI-Lab commented Oct 19, 2026

Uh oh!

MiaAI-Lab commented Oct 21, 2026

Uh oh!

danieljohnmorris left a comment

Choose a reason for hiding this comment

Confirmed findings

Reconciliation

Verification and limits

Uh oh!

MiaAI-Lab commented Oct 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

danieljohnmorris left a comment

Choose a reason for hiding this comment

Uh oh!

danieljohnmorris left a comment

Choose a reason for hiding this comment

Uh oh!

MiaAI-Lab commented Oct 23, 2026

Uh oh!

danieljohnmorris commented Oct 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

MiaAI-Lab commented Oct 24, 2026

Uh oh!

MiaAI-Lab commented Oct 24, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

danieljohnmorris commented Oct 14, 2026 •

edited

Loading

MiaAI-Lab commented Oct 22, 2026 •

edited

Loading

danieljohnmorris commented Oct 23, 2026 •

edited

Loading