Skip to content

feat: opt-in read-only API Keys page (LiteLLM virtual keys via a host-side bridge) - #314

Closed
travellingsoldier85 wants to merge 8 commits into
MiaAI-Lab:mainfrom
travellingsoldier85:feat/api-keys-readonly
Closed

travellingsoldier85 wants to merge 8 commits into
MiaAI-Lab:mainfrom
travellingsoldier85:feat/api-keys-readonly

Conversation

@travellingsoldier85

Copy link
Copy Markdown
Contributor

Refs #297. This is the first of the two PRs suggested there: the bridge and the read-only views. Creating, deleting, blocking and changing budgets come in PR #2, so that review can focus on the write call list and the lockout.

Summary

  • New Settings toggle API Keys page (apiKeysEnabled, default off).
  • New module server/apiKeys/ registers GET /api/apikeys/status, /keys and /keys/:id behind the existing auth and a 60/min rate limit.
  • The module talks only to a host-side bridge over a unix socket, with a fixed list of GET calls: /v1/health, /v1/keys and /v1/keys/<64-hex>.
  • Reference bridge in scripts/apikeys-bridge/ (Node, no dependencies), with a systemd unit example.
  • Frontend page /api-keys: key table plus a detail view, using the existing tk-* page/card/table styles and theme variables.

Maintainer points from #297, one by one

  1. Optional module, MQTT shape. Off by default behind apiKeysEnabled, which sits next to mqttEnabled in settings.js. The feature lives in its own server/apiKeys/ module with its own routes and its own README section. While off there is no sidebar, palette or mobile entry, every /api/apikeys/* route answers 404, and a deep link to /api-keys lands on Overview. There is no disabled page.
  2. Bridge only, no direct mode. Nothing in the server can reach LiteLLM or hold a master key. bridgeClient.js refuses any path outside the call list before it opens a socket, and the module has no write method at all.
  3. Admin token not in settings.json. The token is read from SPARKDASH_APIKEYS_BRIDGE_TOKEN (and the socket from SPARKDASH_APIKEYS_BRIDGE_SOCKET) once at startup, like SPARKDASH_MQTT_PASSWORD. A test checks that settings.json gets no bridge field.
  4. File credentials via secretsStore. PR1 has no file credential, so secretsStore is not touched. If PR feat: add power controls and safe daily self-updates #2 needs one, it will go through server/secretsStore.js.
  5. Unknown, not zero. Missing or non-numeric spend and limits normalise to null on the server and render as "unknown" (spend) or "—" (limits) on the page. There are tests on both sides.
  6. Audit log retention. PR1 keeps no audit log because nothing changes state. PR feat: add power controls and safe daily self-updates #2 will add a bounded log and state the bound.
  7. Lockout / backoff.
    • In sparkDash: a 401/403 from the bridge stops all bridge calls for 30 s, doubling on each further rejection up to 15 min. The first accepted call resets this. A bridge 429 is honoured with its Retry-After (capped at 15 min).
    • In the reference bridge: 10 rejected tokens within 10 min means every request gets 429 until the window clears, even with the right token. The bridge stores only the token's sha256 and compares in constant time.
    • Both behaviours are tested.
  8. Conventions. Plain ESM server code, TS/React 19/Tailwind v4, existing components and classes, kebab-case CSS, no inline styles. Server fixtures run in temp dirs, and the new test globs are added to test:server.

Security model / call list

sparkDash route Bridge call LiteLLM call (reference bridge)
GET /api/apikeys/status GET /v1/health GET /health/readiness
GET /api/apikeys/keys GET /v1/keys GET /key/list?return_full_object=true (paged, max 50×100)
GET /api/apikeys/keys/:id GET /v1/keys/<64-hex> GET /key/info?key=
  • The bridge answers 405 to any other method and 404 to any other path, without calling LiteLLM.
  • The reference bridge reads at most 8 MB per LiteLLM reply and refuses (502) a list of more than 5000 keys rather than returning part of it.
  • Replies carry an allow-listed set of fields; key material is never forwarded, and sparkDash drops any field it doesn't know.
  • The master key reaches the bridge through systemd LoadCredential, so the key file can stay root-only.

How to test

  • npm run typecheck && npm run build && npm run i18n:check
  • node --test server/apiKeys/__tests__/*.test.js scripts/apikeys-bridge/*.test.js server/__tests__/settings-apikeys.test.js
  • npx vitest run
  • Manual: run the reference bridge against a LiteLLM proxy, set the two env vars, enable Settings → API Keys page, then open API Keys. Switch the setting off: the entry disappears and /api/apikeys/keys answers 404.

Scope

Read-only views only. Destructive actions (create with one-time reveal, block/unblock, kill, delete, limits and budgets) come in PR #2, with the bounded audit log and a browser-side admin gate.

…ix socket

Fixed GET-only call list (health, key list, key info). Bridge socket and token come from
SPARKDASH_APIKEYS_BRIDGE_SOCKET / SPARKDASH_APIKEYS_BRIDGE_TOKEN, never settings.json.
404 while apiKeysEnabled is off. Unreadable values stay null (unknown), never 0. A rejected
token backs off 30 s doubling to 15 min. Refs MiaAI-Lab#297
… read-only calls

Node, no dependencies; holds the LiteLLM master key via systemd LoadCredential, checks the
sha256 of the dashboard token, locks out after 10 bad tokens in 10 min. Example unit included.
…s is on

No nav entry, palette command or mobile Stats entry while the setting is off; a deep link
to /api-keys then lands on Overview. Unreported spend renders as unknown, never $0.
Refs MiaAI-Lab#297
@MiaAI-Lab

MiaAI-Lab commented Oct 31, 2026 •

Copy link
Copy Markdown
Owner

First review, at bccc4254 against main (5e7be05): the sparkDash side is port-ready. The bridge call list is enforced before any socket is opened (server/apiKeys/bridgeClient.js), key ids are hex-validated at three independent layers, the token comes only from SPARKDASH_APIKEYS_BRIDGE_TOKEN with a test pinning that settings.json gains no bridge field, the routes are registered behind the auth middleware with a 60/min rate limit, and everything answers 404 while apiKeysEnabled is false. Two fixes are needed in the reference bridge itself: the advertised "at most 8 MB per LiteLLM reply" bound is applied only after the whole body has been buffered (scripts/apikeys-bridge/bridge.js:67-68) — count bytes incrementally and destroy the response the way bridgeClient.js:39-47 already does; and a reply without total_pages silently truncates the key list at 100 (bridge.js:93) — fail closed with 502 unless total_pages is a positive number, matching the existing over-5000 behaviour. With those two addressed this is ready to land. Leaving it open for your follow-up.

…t total_pages

Review follow-up for the reference bridge:

- LiteLLM replies are read from Response.body chunk by chunk. Bytes
  are counted as they arrive and the body is cancelled (undici then
  closes the connection) as soon as the total passes 8 MB, or before
  reading when Content-Length already says it will, the way
  bridgeClient.js destroys the request. Such a reply is still a failure
  (status 0), but it is no longer buffered whole first.
- /v1/keys answers 502 when a /key/list page has no total_pages that is
  a positive whole number, instead of treating that page as the last
  one. The one exception is LiteLLM's own reply for an empty key table
  (keys [], total_count 0, total_pages 0, from the ceiling division in
  list_key_helper), which stays an empty list. More than 50 pages (5000
  keys) is now refused from the first page instead of after 50 fetches.

Tests: a streamed over-limit body is cancelled at most one chunk past
the limit; a Content-Length over the limit is refused with nothing
read; missing, null, 0, negative, fractional and string total_pages,
and non-JSON NaN/Infinity, give 502; the empty-table reply is an empty
list; a 3-page list sent in 1-byte chunks (multi-byte characters split
across reads) comes back whole.
Resolves the three conflicts with 92ac62c (kiosk review follow-ups) in
src/constants.ts, src/App.tsx and src/components/shell/AppSidebar.tsx by
keeping both entries: KIOSK_ID and API_KEYS_ID (constant, /kiosk and
/api-keys paths, and the imports).
@travellingsoldier85

Copy link
Copy Markdown
Contributor Author

Thanks for the review. Both bridge fixes are in 6be543a:

  1. 8 MB bound. LiteLLM replies are now read from Response.body chunk by chunk. Bytes are counted as they arrive, and the body is cancelled as soon as the count passes 8 MB, or before anything is read when Content-Length already says it will. Cancelling makes undici close the connection, which is the fetch version of the req.destroy() in bridgeClient.js:39-47. The 15 s timeout still covers reading the body. Tests check that an oversized stream is cancelled at most one chunk past the limit and that an over-limit Content-Length is refused with nothing read.

  2. total_pages. /v1/keys now answers 502 unless each page has a total_pages that is a positive integer (missing, null, 0, negative, fractional, string and non-finite values are tested). More than 50 pages is now refused on the first page instead of after 50 fetches.

One exception you may want to veto: for an empty key table LiteLLM itself sends total_pages: 0, because list_key_helper computes -(-total_count // size). That exact reply (page 1, keys: [], total_count: 0, total_pages: 0) still gives an empty list, so a LiteLLM with no virtual keys doesn't show up as an error. total_pages: 0 with any key, or without total_count: 0, is still a 502. If you'd rather keep it strict, it's a two-line change and I'll make it.

Since 92ac62c the branch conflicted with main in src/App.tsx, src/components/shell/AppSidebar.tsx and src/constants.ts (KIOSK_ID and API_KEYS_ID added in the same places). 63cd127 merges main and keeps both entries. It changes nothing else. Typecheck, build, the i18n check, the API keys/bridge/settings tests and vitest pass on the merged tree.

(Optional) Main now localises every Settings row, so the new API Keys toggle is the only English-only one left. I can move it into the settings catalog and add /api-keys to docs/I18N-COVERAGE.md, either here or in a follow-up, whichever you prefer.

@MiaAI-Lab

Copy link
Copy Markdown
Owner

I read the bridge client and service closely: unix socket only, a fixed allowlist of GETs with a 64-hex id check (isBridgeCall), a 4 MiB response cap, bearer token from env, backoff after repeated auth rejections (server/apiKeys/routes.js), all behind the apiKeysEnabled flag, read-only throughout — the dashboard never holds the LiteLLM master key. That's a sound design. asks before merge: document the bridge token rotation next to the socket path in the README, and linearize the branch (it carries a merge of main, which noise up the diff). The new page/nav is a product sign-off, so leaving it open for a maintainer pass.

(Review notes prepared with Claude Code on behalf of the maintainer.)

@travellingsoldier85

Copy link
Copy Markdown
Contributor Author

Thanks for the close read. Both asks are done, in #365:

  1. Token rotation. The API Keys section of the README now has a Rotating the bridge token paragraph directly under the socket and token variables. Both sides read the token once at startup and the reference bridge accepts a single token, so the steps are: make a new token and its hash, restart the bridge first, then sparkDash (under Docker docker compose up -d --force-recreate, because docker compose restart keeps the old environment). It also says why that order matters (sparkDash's backoff, and the socket directory that systemd recreates on every bridge restart, which a running container would not see) and how to clear a bridge lockout.

  2. Linear branch. The seven commits are rebased onto current main (7c2b963) with no merge commit. The kiosk conflict is resolved inside the UI commit the way the merge did, and the commit messages are unchanged. The tree after the seventh commit is exactly git merge-tree of main and this PR's head, so the code is this PR merged with main; the README paragraph is the only addition.

Tests on the new head (Node 24): typecheck, build and both i18n checks pass; vitest 109 files / 823 tests pass (main: 108 / 818, the extra ones are this PR's); the server suite, run file by file, gives the same result per file as main plus this PR's 21 tests, which pass (the few failures on my host are GPU/sudo environment tests that fail the same way on main).

Rewriting this branch would have meant force-pushing over this PR's history, so the linear version comes from a new branch as #365, and this branch stays as it was. I'm closing this PR in favour of #365; the review history stays here. Understood that the new page and nav need your product sign-off; #365 is ready for that pass.

MiaAI-Lab pushed a commit that referenced this pull request Nov 4, 2026
Both sides read the token once at startup and the reference bridge accepts
a single token hash, so a rotation is: new token and hash, restart the bridge
first, then sparkDash (under Docker recreate the container, because
docker compose restart keeps the old environment). The paragraph says why
that order (sparkDash's backoff, and the socket directory systemd recreates
on every bridge restart) and how to clear a bridge lockout.

Asked for in the review of #314.

(cherry picked from commit 03963a9)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants