Repository navigation
feat: promote config bundles through the control plane from mocactl (ADR-0038) - #458
Conversation
…plane Adds ADR-0038 and its design spec: promote a skills directory from mocactl through a new POST /v1/config-bundles control-plane endpoint instead of a direct kubectl/Redis tunnel, so P6-on-Kubernetes users with no cluster credentials can promote and use their own skills. Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Paolo Dettori <dettori@us.ibm.com>
- Reuse knative-server's existing configRef_invalid error code verbatim instead of minting a differently-spelled invalid_configRef sibling for the same concept. - Note that getSession's echoed configRef is not a liveness signal: it reflects what was recorded at creation, not whether the digest still exists in Redis past its 30-day TTL. - Fix ADR index table column-1 padding for the new 0038 row to match the rest of the table. Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Paolo Dettori <dettori@us.ibm.com>
…ding The first version assumed /v1/turn already applied configRef; it does not (handleTurn parses only sessionId and prompt). It also had mocactl send configRef on every turn while rejecting exactly that design, missed the control plane's 64 KiB body cap, and relied on a dependency ADR-0036 forbids. The harness now receives configRef through the credential exchange and applies it with the overlay code shared with run-leaf; uploads get a per-route body limit and a bundle cap; mocactl takes a named, test-enforced exception for @moca/config-bundle. Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Paolo Dettori <dettori@us.ibm.com>
…romotion Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Paolo Dettori <dettori@us.ibm.com>
…ane can use it Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Paolo Dettori <dettori@us.ibm.com>
Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Paolo Dettori <dettori@us.ibm.com>
…onfig-bundles Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Paolo Dettori <dettori@us.ibm.com>
…it to the harness via the exchange Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Paolo Dettori <dettori@us.ibm.com>
…of run-leaf into a shared helper Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Paolo Dettori <dettori@us.ibm.com>
…path Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Paolo Dettori <dettori@us.ibm.com>
…he exchange; 410 when it expired Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Paolo Dettori <dettori@us.ibm.com>
… the ADR-0038 dependency exception Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Paolo Dettori <dettori@us.ibm.com>
…aped directory Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Paolo Dettori <dettori@us.ibm.com>
Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Paolo Dettori <dettori@us.ibm.com>
Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Paolo Dettori <dettori@us.ibm.com>
…ning New Session Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Paolo Dettori <dettori@us.ibm.com>
…docs at the mocactl path Also rewraps three lines in the config-bundle promotion design spec so no inline code span crosses a line break; prettier otherwise reflows them and breaks the list indentation (pre-commit run --all-files failed on it). Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Paolo Dettori <dettori@us.ibm.com>
Each /v1/turn attach now claims a per-turn <sid>.<nonce> ref on the digest; the session link stays keyed by the session id and is removed only once no ref of that session remains, and the cache only once the digest has none. The /runs leaf passes no refId and keeps the single-ref script unchanged. A failing detach no longer replaces the turn's outcome. Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Paolo Dettori <dettori@us.ibm.com>
Each successful POST /v1/config-bundles appends config_bundle_uploaded or config_bundle_unchanged with the subject, the digest (configRef) and the decoded byte count. The audit entry gains two optional fields for them. ADR-0038 records the upload quota and pod digest-cache LRU as follow-ups. Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Paolo Dettori <dettori@us.ibm.com>
… the bundle The CLI exits 2 on an empty or blank --config, and cmdRun forwards any configRef that is not undefined, so a programmatic empty string reaches the server and gets configRef_invalid. The --config tests now assert the new messages rather than just the flag name. Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Paolo Dettori <dettori@us.ibm.com>
…TTP 410 For both the sync and SSE /turn paths: the exchange's configRef reaches executeTurn, a body-supplied configRef does not, and a BundleNotFoundError from the turn is a plain 410 config_bundle_not_found JSON response. Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Paolo Dettori <dettori@us.ibm.com>
mocactl sessions are interactive, and building them unattended warned "Use --mode attended" for dialogue skills, a flag mocactl does not have. Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Paolo Dettori <dettori@us.ibm.com>
…ory restores it Digests are deterministic, so promoting the unchanged directory again reproduces the digest and revives the session; only a changed directory needs a new session. Spec §3 carries the same text and reason. Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Paolo Dettori <dettori@us.ibm.com>
…he rollout order The harness must be upgraded before the control plane: an older harness ignores configRef, so turns would run without the skills and no error. ADR-0038 Consequences records the same. Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Paolo Dettori <dettori@us.ibm.com>
The live smoke on kind failed every turn of a session started with a config bundle: "config overlay failed (exit 127)". The overlay unpacks the bundle with `base64 -d | tar -x -z` under flock, and the worker image had tar only in its ripgrep fetch stage. Install tar in both runtime images, list gzip and util-linux-core (flock) explicitly since they were only inherited from the base, and add a parity test mapping every overlay binary to a package the runtime stage must install. Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Paolo Dettori <dettori@us.ibm.com>
Brings in P6.3 sandbox tiers (rossoctl#454) and 80 other commits. Every conflict was additive: SessionRecord, the exchange response, createSession, TurnAuth, ExecuteTurnInput and the OpenAPI document now carry both configRef (ADR-0038) and sandboxTier (P6.3). One ordering decision in executeTurn: placement is recorded first, then the config bundle is attached, then the P6.3 reset frame is emitted. The reset frame flushes the SSE headers, so attaching before it keeps a missing bundle a plain 410 rather than an error frame. Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Paolo Dettori <dettori@us.ibm.com>
`setup.sh --build` rebuilt and `kind load`ed an image under the same `:local` tag, so the apply saw unchanged pod templates and rolled nothing: the pods kept running the old image. Found when the live config-bundle smoke needed a manual `kubectl rollout restart`. On kind, ensure_images now reads each loaded image's ID (on --skip-build too) and write_overlay stamps it as moca.dev/image-id on the control plane, supervisor, relay and sandbox pod templates. A new image is now a template change; an unchanged one rolls nothing. OCP targets pull by ref and carry no ID. Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Paolo Dettori <dettori@us.ibm.com>
ingpaolodettori-dev
left a comment
There was a problem hiding this comment.
This is a well-structured slice. configRef is fixed at session creation, comes only from the control-plane exchange, and is never taken from the turn body. The per-route body limit is enforced while streaming. Each turn holds its own overlay ref, and detach runs on every exit path. The 410 is mapped before any SSE bytes are sent. The run-leaf refactor preserves behaviour. CI is green and all 26 commits are signed off.
Blocking:
POST /v1/config-bundlesexposes the lenientuntarto every API-token holder. A negative octal size in a tar header makes it loop forever, synchronously, so one small request hangs the control plane.- Nothing bounds uploads into the shared 256Mi Redis (no quota, no
maxmemory, AOF on), so a few dozen 8 MiB uploads can take Redis down for all tenants. - The PR adds a
.claude/test fixture, which the agent-config inspection gate requires a maintainer to confirm. It is benign; see inline.
The suggestions and nits inline are non-blocking.
| import { gunzipSync, gzipSync } from 'node:zlib'; | ||
| import { contentDigest, untar } from '@moca/config-bundle'; | ||
| import { contentDigest } from './build.js'; | ||
| import { untar } from './tar.js'; |
There was a problem hiding this comment.
must-fix (DoS): putBundle now runs untar() on client-supplied bytes from POST /v1/config-bundles. untar (tar.ts:79) parses the size with parseInt(…, 8), which accepts a leading -, and then advances with off += Math.ceil(size / BLOCK) * BLOCK (tar.ts:89) without checking that it moved forward. A header with size -1000 (octal for −512) cancels the off += BLOCK before it, so the loop re-reads the same header forever and pushes an entry on every pass. The loop is synchronous, so one sub-64 KiB request from any API-token holder blocks the control-plane event loop and eventually OOMs it. tar.ts is unchanged in this PR, but before it the parser could only be reached with cluster/Redis access. Fix: accept only sizes matching ^[0-7]+$ (or reject a negative/NaN size), require off to strictly increase and stay within tar.length, and add a hostile-header test in handlers-config-bundles.test.ts. (Anchored on the untar import because putBundle's body at store.ts:58 is unchanged by the rename and so isn't part of the diff.)
There was a problem hiding this comment.
Fixed in 0119f2d. untar now accepts the size and mode fields only as unsigned octal (^[0-7]*$, empty = 0), throws on an entry that runs past the end of the archive, and requires the offset to advance on every pass. Through putBundle that surfaces as 400 digest_mismatch. New tests: a negative size, a non-octal size and mode, and a truncated entry, each under a 1 s vitest timeout. handlers-config-bundles.test.ts also has a hand-built 512-byte header with size -1000 and type 0. Before the fix, that test OOMed the vitest worker, which is exactly the loop you described.
| } | ||
| let uploaded: boolean; | ||
| try { | ||
| ({ uploaded } = await putBundle(deps.bundles, digest, tar)); |
There was a problem hiding this comment.
must-fix (shared-Redis exhaustion): Any API user can store any number of distinct bundles of up to 8 MiB each, with a 30-day TTL, in the Redis that also holds sessions, ownership and sandbox records. Random content doesn't compress, so each bundle is about 10.7 MiB once gzipped and base64-encoded. deploy/k8s/base/redis.yaml limits Redis to 256Mi, sets no maxmemory, and setup.sh enables appendonly yes. Roughly two dozen uploads should therefore OOM-kill Redis, and the AOF would reload the keys on restart. ADR-0038 lists the quota as a follow-up, but a cross-tenant outage that outlasts the request seems worth closing before this route ships. Options: a per-subject byte/count budget, a separate or maxmemory-bounded store for bundles, or a much lower cap.
There was a problem hiding this comment.
Fixed in 336f269 and 212ef3e. The control plane now enforces a per-subject budget (SH_BUNDLE_SUBJECT_BYTES, default 32 MiB) and a deployment-wide one (SH_BUNDLE_TOTAL_BYTES, default 64 MiB). A bad value refuses to boot. The charge is the stored value's length with a 4 KiB per-bundle floor for key and index overhead. A NEW digest over either budget gets 429 bundle_quota_exceeded, with a message naming which budget and its size. Re-uploading a stored digest is free and refreshes it. Entries live in expiry-scored zsets (sh:cp:bundles:all, sh:cp:bundles:owner:<subjectHash>) plus a size/owner hash, so they age out with the bundle's 30-day TTL. A digest is charged to the subject that first stored it. Concurrent uploads are serialized in-process: admit, charge and SET for a new digest run under one lock. That relies on the control plane running as a single replica, which is what deploy/k8s renders. The charge is recorded before the SET, and the whole record is rolled back if the record or the SET fails partway (e44dfa0). mocactl maps the code to a one-line message, and the ADR-0038 follow-up bullet is replaced by the implemented budget.
| const configRoot = resolveConfigRoot(dir, env.home ?? homedir(), env.cwd ?? process.cwd()); | ||
| let result; | ||
| try { | ||
| result = buildBundle({ |
There was a problem hiding this comment.
suggestion: promoteDirectory exposes @moca/config-bundle's filesUnder (resolve.ts) to every mocactl user. That function only bounds directory symlinks with isWithin. A file symlink inside a skill (e.g. skills/x/creds -> ~/.aws/credentials) is followed, read and uploaded, and so are dotfiles such as .env, .npmrc or a nested .git/. Only the structural secret patterns stand in the way. Consider applying the isWithin check to the file branch too (and to markdownFiles), and/or excluding dotfiles.
There was a problem hiding this comment.
Fixed in 0aa9ffa. filesUnder now applies the same realpath + isWithin check to files as to directories, and so do the prompts and memory .md listings (including MEMORY.md). An escaping file is skipped with the existing skill_symlink_escaped warning. Dotfiles still travel: they can be legitimate skill content, and the secret scan plus the report, which is now shown before upload (see 92778b2, --dry-run), cover them.
| try { | ||
| const r = await promoteDirectory(arg, rt.cp); | ||
| setPendingBundle(r); | ||
| notify(`promoted ${r.skills.length} skills — ${r.uploaded ? 'uploaded' : 'unchanged'}`); |
There was a problem hiding this comment.
suggestion: Scan warnings never reach the user before the upload. The CLI prints r.report only after putConfigBundle has stored the bundle (headless.ts:121). The TUI never shows the report at all, only promoted N skills — uploaded. That hides possible_secret ("verify before promoting"), skill_symlink_escaped and dropped commands. Consider showing the report, or adding a confirm/--dry-run step, before uploading, and at least a warning count in this toast. Also, the toast says promoted 0 skills for a commands-only directory.
There was a problem hiding this comment.
Fixed in 92778b2 and 389b1f6. promoteDirectory gained an onBuilt hook, called after preflight and before putConfigBundle, and a dryRun option. The CLI (mocactl promote DIR) and --dry-run show the summary and report (stderr) before anything is uploaded; --dry-run uploads nothing and needs no login. The TUI shows a warning count after the upload: "promoted 1 skill, 2 commands — uploaded; 1 warning — run mocactl promote <dir> --dry-run to see it", naming the directory you passed. A commands-only directory no longer reads "promoted 0 skills" alone.
| !descriptors.some((d) => d.consumer === 'inference'); | ||
| const credentialName = fallback ? '' : resolveInferenceName(descriptors, requested); | ||
|
|
||
| let configRef: string | null = null; |
There was a problem hiding this comment.
nit: configRef is validated after deps.credentials.list()/resolveInferenceName. A request with a malformed configRef and no credential therefore gets credential_required, not configRef_invalid, and costs a store round trip first. Validating it alongside the other body-shape checks fixes both.
There was a problem hiding this comment.
Fixed in 035938a. configRef is validated with the other body-shape checks, before the credential lookup. A malformed configRef with no credential now gets configRef_invalid, and credentials.list is never called; a spy test asserts that.
| if (typeof body.tar !== 'string' || body.tar.length === 0) { | ||
| throw new CpError('invalid_request', 'tar must be a non-empty base64 string'); | ||
| } | ||
| const tar = Buffer.from(body.tar, 'base64'); |
There was a problem hiding this comment.
nit: Buffer.from(s, 'base64') silently skips invalid characters, so a malformed tar comes back as digest_mismatch, not invalid_request. A strict base64 check (e.g. /^[A-Za-z0-9+/]*={0,2}$/) would return the more accurate error.
There was a problem hiding this comment.
Fixed in 1cb8dc0. tar must be strict base64 (alphabet, at most two =, length a multiple of 4), otherwise invalid_request.
|
|
||
| describe('an expired config bundle', () => { | ||
| it('is 410 config_bundle_not_found, not a 500', () => { | ||
| const err = Object.assign(new Error('config bundle not found: sha256:' + 'a'.repeat(64)), { |
There was a problem hiding this comment.
nit: server.ts matches on err.name, and its comment on the NO_CAPACITY mapping relies on the paired test constructing the real class. This test builds a plain Error with the name set by hand, so renaming BundleNotFoundError wouldn't fail it. Consider constructing the real class (re-exported via @moca/harness if needed).
There was a problem hiding this comment.
Fixed in 037f518. @moca/harness/run-turn re-exports BundleNotFoundError, and the test constructs the real class.
| throw new CpError('configRef_invalid', 'configRef must be a string'); | ||
| } | ||
| try { | ||
| configRef = assertValidDigest(body.configRef); |
There was a problem hiding this comment.
suggestion: configRef is only shape-checked here. A well-formed digest that was never uploaded (a typo, or a digest from another deployment) creates a session whose every turn returns 410 config_bundle_not_found. mocactl then shows "this session's config bundle has expired — promote the same directory again", which is misleading in this case. deps.bundles is already wired in. An exists(bundleKey(configRef)) check here would fail fast at creation. Adding an expire() would also refresh the 30-day TTL. As far as I can see, nothing on the turn path refreshes it (getBundle is read-only), so a bundle in daily use still expires 30 days after its last upload.
There was a problem hiding this comment.
Fixed in 035938a. createSession checks the bundle exists. A missing digest gets 404 config_bundle_not_found ("no config bundle with that digest — promote the directory first") and creates nothing. A stored one has its 30-day TTL and budget entry refreshed. Every exchange of a session with a configRef refreshes them too, best-effort, so a bundle in daily use no longer expires. mocactl's message now fits both sources: "this session's config bundle is gone (expired or never uploaded) — …".
| configRef: digest, | ||
| }), | ||
| ).rejects.toThrow(); | ||
| expect(attachMock).toHaveBeenCalledWith( |
There was a problem hiding this comment.
nit: Every case here relies on the core throwing right after the attach point. None asserts that attached.promotedConfig actually reaches executeTurnCore (the { ...input, promotedConfig: attached.promotedConfig } swap in run-turn.ts). A spy on the loader construction or on promotedLoaderOptions would cover that wiring in unit tests. Today only the live smoke does.
There was a problem hiding this comment.
Covered in 305b872. A spy on promotedLoaderOptions asserts that the loader is built from attached.promotedConfig when configRef is set. Reverting the swap in run-turn.ts makes the test fail.
ingpaolodettori-dev
left a comment
There was a problem hiding this comment.
Re-review at 5531473.
The new commit (fix(deploy/k8s): a rebuilt kind image rolls the pods that run it) looks correct and well tested. The moca.dev/image-id stamp covers every workload that runs the harness or sandbox image (control plane, supervisor, relay, sandboxes). It merges into the existing settings-hash annotation map without adding a second add, and it leaves OCP renders unchanged. Nothing new to raise on it.
None of the files from the previous review (5447237048) changed, and none of its 17 comments has a reply yet, so those findings still stand:
- #1 (must-fix, DoS):
untaraccepts a negative octal size, and the offset never advances, so one small upload toPOST /v1/config-bundleshangs the control-plane event loop. - #2 (must-fix, shared-Redis exhaustion): there is no per-subject quota on bundle uploads in front of the 256Mi, AOF-backed Redis that also holds sessions and ownership.
- #3 (agent-config gate): a maintainer still needs to confirm inspection of
packages/mocactl/test/fixtures/promote-skills/.claude/skills/moca-smoke-marker/SKILL.md.
The suggestions and nits from that review are still open too.
Author: pdettori (MEMBER — maintainer)
Areas reviewed: Shell, K8s/kustomize overlays, TypeScript test, Docs (incremental); prior-review status across the full PR
Agent/IDE config (.claude/.vscode): FLAGGED (unchanged) — packages/mocactl/test/fixtures/promote-skills/.claude/skills/moca-smoke-marker/SKILL.md
Commits: 27 commits, all signed-off: yes
CI status: 14 passing, k8s-kind-e2e pending at review time
…ders A size field of -1000 cancelled the header advance, so untar re-read the same header forever and one small POST /v1/config-bundles hung and OOMed the control plane. Size and mode now accept only unsigned octal, an entry that runs past the archive throws, and the offset must advance. The throw surfaces as 400 digest_mismatch through putBundle. Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Paolo Dettori <dettori@us.ibm.com>
…fig bundles Any API user could store unbounded 8 MiB bundles with a 30-day TTL in the shared Redis. A NEW digest is now charged its stored size (the gzip+base64 value putBundle writes, passed to a new admit hook) to the subject that first stored it, and refused with 429 bundle_quota_exceeded over SH_BUNDLE_SUBJECT_BYTES (32 MiB) or SH_BUNDLE_TOTAL_BYTES (64 MiB). A re-upload is free and refreshes the entry's expiry. Entries live in zsets sh:cp:bundles:all and sh:cp:bundles:owner:<hash> scored by expiry ms, plus hash sh:cp:bundles:meta (digest -> bytes:owner); expired members are dropped before each sum. No lock: two concurrent uploads can overshoot by one bundle each. mocactl maps the code to a one-line message. Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Paolo Dettori <dettori@us.ibm.com>
Buffer.from(s, 'base64') silently skips invalid characters, so a garbled upload came back as digest_mismatch. The tar must now be strict base64 (alphabet, at most two '=' and a length divisible by 4). Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Paolo Dettori <dettori@us.ibm.com>
Only successful uploads left an audit row. A refusal (digest_mismatch, an oversize or non-base64 tar, bundle_quota_exceeded) now writes decision config_bundle_refused with the reason code through the same best-effort path. A non-mismatch store error is logged with the route and digest (message only, never the body) before it becomes 503 redis_unavailable, so a programming bug no longer looks like an outage. Also makes the budget re-upload test assert ownership directly: a freshly built second bundle can differ in size by a byte. Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Paolo Dettori <dettori@us.ibm.com>
… not stored createSession validated configRef after the credential lookup, so a malformed configRef with no credential got credential_required after a store round trip. It is now validated with the other body-shape checks. A well-formed digest nobody uploaded created a session whose every turn answered 410. Creation now checks the bundle exists (404 config_bundle_not_found, 'no config bundle with that digest -- promote the directory first') and refreshes its 30-day TTL and budget entry. The exchange refreshes both best-effort, so a bundle in daily use no longer expires 30 days after its last upload. mocactl's message fits both sources. Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Paolo Dettori <dettori@us.ibm.com>
putBundle verified only the content digest, which excludes lockfile.json and ignores non-regular or trailing tar data, then stored the raw upload, so the first uploader of a digest set an unchecked lockfile for everyone. It now stores canonicalTar of the verified regular-file entries without lockfile.json; nothing reads the in-bundle lockfile at run time, so the stored bytes are a function of the digest alone. Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Paolo Dettori <dettori@us.ibm.com>
filesUnder bounded only directory symlinks, so skills/x/creds -> ~/.aws/credentials was read and uploaded. Files now get the same realpath + isWithin check, and so do the prompts and memory .md files (including MEMORY.md); an escaping one is skipped with the existing skill_symlink_escaped warning. Dotfiles inside a skill still travel: they can be legitimate skill content, and the secret scan plus the pre-upload report (promote --dry-run) cover them. Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Paolo Dettori <dettori@us.ibm.com>
… exchange An exchange configRef of '' passed the type check and was then dropped, so the turn ran without its bundle (fail open), unlike /runs. A present configRef must now match sha256:<64 lowercase hex>, or the turn fails as credential_unavailable (a control-plane fault). Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Paolo Dettori <dettori@us.ibm.com>
…mapping server.ts matches on err.name, and the test built a plain Error with the name set by hand, so renaming the class would not fail it. run-turn now re-exports BundleNotFoundError beside the sandbox errors it already re-exports, and the test constructs the real class. Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Paolo Dettori <dettori@us.ibm.com>
…nishes The turn finally cleared the lease renewal before attached.detach(), a remote exec with a 60s timeout against a 60s default lease TTL, so a slow cleanup could let the lease lapse and the sandbox be handed out while the overlay was still being removed. Detach now runs first, then the interval is cleared, then the lease released. run-leaf.ts has the same order but predates this PR and is left as is. Also covers the wiring the review found untested: a spy on promotedLoaderOptions proves the attached bundle's promotedConfig, not the raw input, reaches the turn core's resource loader (verified by reverting the swap: the test fails). Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Paolo Dettori <dettori@us.ibm.com>
… uses A control-plane session id is also its leafSessionId, so a /runs leaf and a /v1/turn turn of one session share /workspace/leaves/<sid>/.sh-config on a shared pod. The single-ref cleanup removed that link unconditionally, from under the live turn. Both modes now release it through sessionLinkReleaseLines, inside the lock: it goes only once no <sid> or <sid>.* ref is left. The id checks are unchanged (sessionId and refId are both still validated), and run-leaf-promoted.test.ts passes unmodified. Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Paolo Dettori <dettori@us.ibm.com>
The CLI printed the preflight report only after putConfigBundle had stored the bundle, and the TUI never showed it, hiding possible_secret, skill_symlink_escaped and dropped commands. promoteDirectory now takes an onBuilt hook, called after preflight and before the upload, and a dryRun option that returns the summary (uploaded: false, dryRun: true) without a control plane. mocactl promote prints the summary and the report (stderr) through onBuilt; --dry-run builds and prints only and needs no login. The TUI toast reads 'promoted N skills, M commands -- uploaded', plus a warning count pointing at --dry-run, so a commands-only directory no longer says 'promoted 0 skills'. Docs (README, spec 2.5): /promote starts the session at once when there is exactly one inference credential and no presets, and promote's exit codes are 2 = refused before upload (usage, not logged in, preflight errors), 3 = structural credential match, 1 = anything else. Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Paolo Dettori <dettori@us.ibm.com>
… at 4 KiB Concurrent uploads all passed admitBundle before any was recorded (the check and the charge were separated by a gzip and a ~10 MB SET), so ~30 parallel 8 MiB uploads from one token could still OOM Redis. Admit, charge and SET for a new digest now run under an in-process promise-chain lock, which assumes the single control-plane replica deploy/k8s renders; a stored digest is re-checked inside the lock and stays free. A tiny bundle was charged ~100 B but costs Redis ~700-800 B in key, TTL, meta field and zset members, and every upload sums all live entries. The charge is now max(stored length, MIN_BUNDLE_CHARGE_BYTES = 4096). The charge is recorded before the SET and rolled back if the SET fails, so a stored bundle can no longer be left uncharged. putBundle's admit hook is replaced by prepareBundle, the one definition of the stored value. Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Paolo Dettori <dettori@us.ibm.com>
…ular
The /promote toast's --dry-run hint said a literal DIR and every count
was plural ('1 skills', '1 warnings'). describePromotion now builds the
line with the directory as the user typed it and singular/plural nouns.
Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com>
Signed-off-by: Paolo Dettori <dettori@us.ibm.com>
…partway A recordBundle that failed after adding the global zset member but before the owner's left a 30-day phantom charge on the deployment-wide budget, because only a failed SET was rolled back. Record inside the same rollback guard as the SET; unrecordBundle tolerates missing members. Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Paolo Dettori <dettori@us.ibm.com>
ingpaolodettori-dev
left a comment
There was a problem hiding this comment.
Re-review at e44dfa0 (15 new commits since 5531473, all signed off).
Both blocking findings from the earlier reviews are fixed, and I checked the fixes against the code:
- #1
untarDoS: size and mode must be unsigned octal, an entry running past the end throws, and the offset always advances by at least one block. A hostile-header test goes through the handler. - #2 shared-Redis exhaustion: there is a per-subject (32 MiB) and a deployment-wide (64 MiB) budget on stored bytes. Admit, charge and SET for a new digest are serialized, with a 4 KiB floor per bundle and rollback when a record or SET fails partway. That bounds bundle storage well below the 256Mi Redis.
Every suggestion and nit from review 5447237048 is addressed as described in its thread: file symlinks now get the isWithin check, the report is shown before upload and --dry-run exists, the stored bytes are canonical with no lockfile, /runs link release is guarded, detach runs under the lease, the exchange rejects a non-digest configRef, createSession checks the bundle exists, the base64 check is strict, and refusals are audited. I also confirmed that nothing reads the in-bundle lockfile.json at run time. The bind-repoint case is tracked in #459.
There are two new non-blocking suggestions on the budget, plus one nit:
touchBundlecan revive a phantom charge for an expired bundle.- With the default ratio of 32 MiB per subject to 64 MiB total, two accounts can lock everyone out of new uploads, and there is no eviction path.
.claude/ fixture inspection gate: cleared. The approving reviewer has personally inspected packages/mocactl/test/fixtures/promote-skills/.claude/skills/moca-smoke-marker/SKILL.md and confirmed it is a benign test fixture (skill frontmatter plus a fixed reply, no hooks, commands, permissions or executables). With both blockers fixed and the gate cleared, approving. The two budget suggestions and the nit above are non-blocking.
Author: pdettori (MEMBER — maintainer)
Areas reviewed: TypeScript (control-plane, config-bundle, harness, knative-server, mocactl), Docs/OpenAPI (incremental 5531473..e44dfa0); prior-review status across the full PR
Agent/IDE config (.claude/.vscode): FLAGGED, inspected and confirmed benign by the reviewer — packages/mocactl/test/fixtures/promote-skills/.claude/skills/moca-smoke-marker/SKILL.md
Commits: 42 commits, all signed-off: yes
CI status: no workflow runs on e44dfa0 yet (only DCO, passing); 5531473 was green
| digest: string, | ||
| nowMs: number, | ||
| ): Promise<void> { | ||
| await redis.expire(bundleKey(digest), DEFAULT_BUNDLE_TTL_SECONDS); |
There was a problem hiding this comment.
suggestion (budget accounting): touchBundle refreshes the budget entry even when the bundle key is gone. Once a bundle's 30-day TTL lapses, its meta field and zset members stay until some later admitBundle cleans up expired entries. On a quiet deployment, the next exchange of a session that still names that digest runs expire() on a missing key, which is a no-op. refreshBundle then finds the old meta and re-scores both zset members 30 days out. That leaves a phantom charge on the owner and on the deployment total for bytes that are no longer stored, and every later turn attempt on that dead session (each fails with 410 after the exchange) keeps it alive. It also bites the recovery path mocactl recommends: re-promoting the same directory goes through admitBundle, which counts the phantom entry for that digest on top of the new charge, so a subject near its limit gets a spurious 429. Suggested fix: only call refreshBundle when expire reports the key existed (node-redis returns 1/0), or check exists first. The fake's expire always returns 1, so a test needs the fake to return 0 for a missing key.
There was a problem hiding this comment.
Fixed in 7986c92. touchBundle refreshes the budget entry only when EXPIRE returns 1, so a dead session's exchanges can no longer revive the charge of an expired bundle. The re-promotion path is also fixed by itself: admitBundle now takes the digest and drops any stale entry for it before counting. The caller has just found no key, so any entry it still has is stale by definition. That entry is no longer counted on top of the new charge, and it is removed from the previous owner's zset. The fake's expire returns 0 for a missing key. Two new tests failed before the fix with exactly the spurious 429: a touch after expiry followed by a new upload, and a re-store of a digest whose key is gone.
| sandboxNamespace: env.SH_SANDBOX_NAMESPACE || 'default', | ||
| publicHarnessUrl: urlEnv(env, 'SH_PUBLIC_HARNESS_URL'), | ||
| sandboxTiers: parseSandboxTiers(env), | ||
| bundleSubjectBytes: byteBudgetEnv(env, 'SH_BUNDLE_SUBJECT_BYTES', 32 * 1024 * 1024), |
There was a problem hiding this comment.
suggestion (cross-tenant availability): With these defaults the per-subject budget is half the deployment budget. Any two subjects can therefore fill SH_BUNDLE_TOTAL_BYTES with about six max-size bundles, and from then on nobody can store a NEW digest. Re-uploading a stored digest is free and refreshes it, so the two subjects can keep the budget full indefinitely. As far as I can see, GithubOAuthProvider has no allowlist, so a subject costs one GitHub account. Nothing lets an operator evict an entry except hand-editing sh:cp:bundles:*, and nothing lets a user delete their own bundles. Redis itself is now protected, which was the blocking concern, so this isn't blocking. Consider a much smaller per-subject share by default (e.g. total/8), plus an admin-role delete (the admin role already exists) or at least a documented operator runbook for freeing the budget.
There was a problem hiding this comment.
Addressed in 7986c92. SH_BUNDLE_SUBJECT_BYTES now defaults to 16 MiB, a quarter of the 64 MiB total, so it takes four subjects to fill the budget. I stopped short of total/8. A bundle is stored gzip+base64, so an incompressible max-size tar takes about 10.7 MiB, and 8 MiB per subject would refuse it. There is also a way out now: DELETE /v1/config-bundles/{digest} (mocactl bundles delete DIGEST) removes the key and its budget entry under the bundle lock. It is allowed to the subject the digest is charged to, or to an admin. A bundle with no budget entry, i.e. one written by ADR-0030 /promote, is admin-only. Other callers get 403 forbidden, and a digest with neither key nor entry gets 404. The delete is audited as config_bundle_deleted, and mocactl's quota message points at the command. ADR-0038 records the remaining cost: the budget is still first-come, so freeing it from accounts that hold their share depends on an admin.
|
|
||
| /** | ||
| * Store the bundle under its digest, gzipped and base64'd (base64 keeps the injectable | ||
| * Store the bundle's verified content (canonical tar, no `lockfile.json`) under its digest, gzipped and base64'd (base64 keeps the injectable |
There was a problem hiding this comment.
nit: This JSDoc is now orphaned. It still describes putBundle, but it sits directly above prepareBundle's own doc comment, so editors show it on neither function. Move it down to putBundle (line 66), or merge the two.
There was a problem hiding this comment.
Fixed in 91cece1. The comment now sits on putBundle, rewrapped.
It sat orphaned above prepareBundle's own comment, so editors showed it on neither function (rossoctl#458 review). Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Paolo Dettori <dettori@us.ibm.com>
…a quarter share
touchBundle refreshed a digest's budget entry even after its key had
expired, so every exchange of a dead session revived a charge for bytes
Redis no longer held, and re-promoting the same directory was counted
twice and could be refused with 429. It now refreshes only when EXPIRE
reports the key existed, and admitting a digest first drops any stale
entry it left.
With the per-subject budget at half the total, two accounts could hold
the deployment full indefinitely with nothing to evict them. The
default share is now 16 MiB (a quarter of 64 MiB, still enough for one
incompressible max-size bundle), and DELETE /v1/config-bundles/{digest}
lets the subject a digest is charged to, or an admin, delete it and free
its budget. mocactl gains `bundles delete DIGEST`, and its quota message
points at it. (rossoctl#458 review)
Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com>
Signed-off-by: Paolo Dettori <dettori@us.ibm.com>
ingpaolodettori-dev
left a comment
There was a problem hiding this comment.
Re-review of e44dfa0..7986c92. I checked the fixes for both suggestions and the nit from review 5450527853 against the code:
- Phantom charges:
touchBundlenow refreshes only whenEXPIREreturns 1.admitBundledrops a stale entry for the digest before it counts, and that also moves the entry off the previous owner's zset. Both new tests would fail with the spurious 429 without the fix. - Budget share and eviction: the 16 MiB default is a quarter of the total. I agree with stopping short of total/8, because an incompressible max-size bundle is about 10.7 MiB stored.
DELETE /v1/config-bundles/{digest}is restricted to the owner or an admin, runs under the bundle lock and is audited, andmocactl bundles deletecalls it. The OpenAPI file, the contract test, the README, the ADR and the spec are all updated. - Orphaned JSDoc: fixed.
I left two non-blocking nits on the delete path inline.
Author: pdettori (MEMBER — maintainer)
Areas reviewed: TypeScript (control-plane, config-bundle, mocactl), Docs/OpenAPI. This round covers the incremental diff only.
Agent/IDE config (.claude/.vscode): packages/mocactl/test/fixtures/promote-skills/.claude/skills/moca-smoke-marker/SKILL.md is unchanged since e44dfa0. The approver inspected it and confirmed again for this head.
Commits: 44 commits, all signed-off: yes
CI status: only DCO has run on 7986c92, and it passed. There are no workflow runs on this head.
| throw new CpError('config_bundle_not_found', 'no config bundle with that digest'); | ||
| } | ||
| // A bundle with no budget entry was stored by /promote straight into Redis: nobody's to delete. | ||
| if (!(p.roles ?? []).includes('admin') && owner !== subjectHash(p.sub)) { |
There was a problem hiding this comment.
nit (audit): Refused deletes leave no audit trail. A refused upload writes config_bundle_refused with a reason, but a 403 here, where a non-owner tries to delete someone else's digest, isn't recorded, and only a successful delete is audited. Consider auditing the forbidden refusal (and possibly config_bundle_not_found) the same best-effort way. When an admin deletes someone else's bundle, the config_bundle_deleted row records only the admin, so adding the owner hash would make that row self-explanatory.
| still holds one incompressible max-size bundle (~10.7 MiB stored). | ||
| - Positive: `DELETE /v1/config-bundles/{digest}` (`mocactl bundles delete`) frees budget. The subject | ||
| the digest is charged to, or an `admin`, may delete it; a bundle stored by ADR-0030's `/promote` | ||
| has no budget entry and is admin-only. Sessions on a deleted bundle get 410 on their next turn. |
There was a problem hiding this comment.
nit (docs): Please say outright that a delete affects other subjects' sessions too. Bundles are content-addressed, and re-uploading a stored digest is free and doesn't change the owner. So the first uploader, who could be anyone holding the same tar, can revoke a digest that other subjects' sessions are running on. Those subjects get a 410 and have to re-promote, and after that they own it. The impact is small: one 410 per take-over, and the spec already says the first-uploader field gates only deletion. But "Sessions on a deleted bundle get 410" reads as though it means only the deleter's own sessions.
Conflicts, each resolved by keeping both sides: - packages/control-plane/package.json: this branch's @moca/config-bundle dependency, with main's redis ^6.3.0. - deploy/k8s/setup.sh: the kind-only image-id stamps stay directly after the sandbox replicas op (they continue that patch); main's SH_SANDBOX_EGRESS_EXCEPT NetworkPolicy patch follows as its own target. - packages/supervisor/test/k8s: the overlay writer takes both GO_*_IMAGE_ID and GO_EGRESS_EXCEPT, and both test blocks are kept. Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Paolo Dettori <dettori@us.ibm.com>
Summary
A
mocactluser can promote a Claude-Code-shaped skills directory (.claude/skills,.claude/commands) through the control plane and start a session whose turns run with those skills. This needs nokubectland no Redis access. Before this, the only way to promote was ADR-0030's/promote, which writes to the cluster's Redis over akubectl port-forwardand feeds only the batch/runspath.docs/specs/2026-10-06-mocactl-config-bundle-promotion-design.md. Read its amendment note: planning against the code corrected four claims in the first version.Live acceptance passed on
kind(deploy/k8s, this branch's images) withMOCACTL_LIVE_SMOKE=1, 3 of 3. The test promotes a fixture skill, runsmocactl run --config <digest>, and the model names the promoted skill.How it fits together
The harness takes
configRefonly from the control-plane exchange. The client never sends it on a turn, so a session's bundle is fixed when the session is created.What's in it
@moca/config-bundleentryis optional inbuildBundle. Interactive sessions have no fixed first prompt.harness/src/promote-cli.tsstill requires--entry.putBundle/getBundle) moved here fromharness/src/config-store.ts, so the control plane can use it without depending on the harness.MAX_BUNDLE_BYTES = 8 MiB.Control plane
RouteSpec.maxBodyBytes). The default stays 64 KiB.POST /v1/config-bundles:digest_mismatch;invalid_requestwith a message that names the cap;SH_BUNDLE_SUBJECT_BYTES, 32 MiB) and deployment-wide (SH_BUNDLE_TOTAL_BYTES, 64 MiB) byte budget on stored bundles, with a 4 KiB floor per bundle; a new digest over either returns429 bundle_quota_exceeded. Re-uploading a stored digest is free;POST /v1/sessionsacceptsconfigRef(a malformed one returnsconfigRef_invalid, the same code knative-server already uses). It is stored onSessionRecordand returned by get and list.configRefwhen the session has one.Harness
run-leaf.tsintopromoted-config.ts(attachPromotedConfig).run-leaf-promoted.test.tsis unmodified and still passes.executeTurnappliesconfigRef:/runsbehaves exactly as before.knative-server
TurnAuth.configRefcomes from the exchange and is passed toexecuteTurnon the sync and SSE paths.410 config_bundle_not_found, never a turn silently run without its skills.mocactl
@moca/config-bundle. This is a named exception to ADR-0036;layering.test.tsenforces an allow-list of exactly that one package.mocactl promote DIR [--json]: exits 2 on preflight errors, 3 on a structural credential match, 1 otherwise;mocactl run "…" --config DIGEST: new sessions only; an empty value is refused./promote DIR:/promote --cleardrops it;remote-worker image
taris installed in both runtime Dockerfiles. The overlay unpacks the bundle withtar, and its absence made every promoted turn fail withconfig overlay failed (exit 127). Only the live run on kind found this.gzipandutil-linux-core(flock) are now listed explicitly.deploy/k8s/setup.shmoca.dev/image-id. A rebuild keeps the:localtag, so before this the apply saw no change and the pods kept the old image; the live run needed a manualkubectl rollout restart. An unchanged image rolls nothing, and OCP targets carry no ID.Merge with
mainmain(P6.3 sandbox tiers, feat(deploy): container sandboxes and P4 on one stack (P6.3 PR 3, #425) #454) is merged in. The conflicts were all additive; bothconfigRefandsandboxTiernow flow throughSessionRecord, the exchange,TurnAuthand the OpenAPI doc.executeTurn: placement is recorded, then the bundle is attached, then the P6.3 reset frame is sent. The reset frame flushes the SSE headers, so attaching first keeps an expired bundle a plain 410.Rollout order
Upgrade the harness before the control plane. An older harness ignores
configRef, so its turns would run without skills and with no error.Known follow-ups (recorded in ADR-0038)
deploy/k8srenders). A second replica needs a Redis-side reservation./runsleaf and a/v1/turnturn of one session naming different bundles can repoint their shared overlay link. Only the session's own principal can trigger it./tmp/sh-configdigest cache.GET /v1/config-bundles/{digest}(metadata), if bundle reuse across sessions turns out to matter.Test plan
pnpm -r testgreen on the merged tree;go test ./...inremote-workergreen;pnpm -r typecheckclean; pre-commit cleanrun-leaf-promoted.test.tsunmodified (the/runspath is unchanged)MOCACTL_LIVE_SMOKE=1on kind (deploy/k8s/setup.sh --target kind --build): 3 of 3, including promote andrun --config306cc03, after the merge withmain): 3 of 3 on kind🤖 Assisted-By: Claude Code