Repository navigation
fix(coding-agent): bound public daemon client JSONL ingress #13
Description
Activity
Comet's next host-only session contract is now tracked at rynfar/comet#5 and treats this issue as a hard production dependency.
One compatibility detail needs to be part of the #13 design: passing an options object is not enough for Comet to detect support because older JavaScript constructors silently ignore extra arguments. The public SDK should expose a dedicated explicit client-side feature token (for example
bounded_daemon_ingress_v1) alongside the documented finite default/override. This is an SDK transport-safety signal, not a daemon server capability. Comet will require it before session creation rather than infer safety from version, branch, constructor arity, method presence, schema, or hello capabilities.No implementation branch is claimed yet; upstream overlap and merge-order review is in progress under the issue's existing coordination rule.
Upstream-overlap review is complete. No checkout or active worktree was modified.
Decision: hybridize
Current remote evidence:
pylon@8551520f42d03852736f019f85030ac86dfd4b30- fork mirror
main@5b6c0e94e11a97fcfdd7a9fc9dc4f7acbda9c853 - current Prime upstream
main@a903d4b6768f484bd6d459b7b0aa7dee38e461e2
The relevant
daemon-client.ts,rpc/jsonl.ts, and their tests are byte-identical between current Pylon and current upstream.DaemonClient.connect()still callsattachJsonlLineReaderwithout options. The generic reader's optionalmaxLineLengthcounts decoded JavaScript string units and can discard/resume; it does not enforce raw bytes before decode/parse, fail the transport, reject hello/request waiters with a typed error, or prevent replay after an ingress violation.No current upstream issue, PR, or release was found that supplies this public contract. Upstream PR PrimeIntellect-ai#480 overlaps
daemon-client.ts/tests for request replay and attach recovery, but its head still uses the unbounded reader and does not supersede #13. Older snapshot/chunk and saved-session OOM work provides useful evidence, not a client ingress bound.Contract requiring joint review
The likely additive shape is:
- a public declarative SDK feature registry containing a client-local
bounded_daemon_ingress_v1token; DaemonClientOptions.maxInboundFrameBytes;- a documented finite default or explicitly staged finite opt-in migration;
- a typed frame-too-large error containing only the limit/context, never bytes/prefix;
- raw-byte enforcement excluding LF (with CRLF accounting documented) before UTF-8/JSON;
- terminal failure of that socket, rejection of all hello/request waiters even when request recovery is enabled, no replay from the failed transport, one close notification, and clean explicit reconnect state.
The feature registry is needed because stock JavaScript silently ignores an extra constructor argument. It is public SDK metadata, not
DaemonClientCapability, daemon hello, protocol version, or schema. The exact token, API names, default, memory peak, and migration remain subject to Pylon and Comet review; Comet #5 now phrases this as the reviewed proof/options contract rather than assuming it is settled.A provisional 128 MiB default is only a review input, not an accepted value. Single messages can exceed snapshot chunk targets, pasted images can be large, and legacy attach can be monolithic. Validation must include large single-message cases plus generated 100/500 MiB transcript flows. Comet will set a lower explicit bound based on its isolated host budget after requiring the public proof.
Proposed implementation inventory
- new
packages/coding-agent/src/sdk-features.ts src/modes/rpc/jsonl.tssrc/modes/daemon/daemon-client.tssrc/modes/index.tsand publicsrc/index.tsdocs/daemon.mdanddocs/sdk.md- focused
test/rpc-jsonl.test.tsandtest/daemon-client.test.ts - changelog fragment
.pylon/features.yamlentry (owner: shared,decision: hybridize)- dated
.pylon/upstream-review.mdevidence entry without advancing the top-level reviewed-upstream commit
Tests must cover no-LF bytes, one/many-buffer overflow, exact LF/CRLF and multibyte boundaries, terminal/no-later-line behavior, default/override validation, hello and pending-request rejection, request-recovery suppression, single close notification, no raw error content, within-bound snapshot chunks, and reconnect with no retained partial buffer.
Coordination / no claim yet
No branch or worktree is claimed. The safe order is to let current PR #12 and active #11 work settle, then complete the reviewed upstream integration tracked by #8, and only then branch #13 from the latest exact
origin/pylon. This avoids racing active work and.pylonledger conflicts. After that, declare the exact base/worktree/files here and cross-review the shared SDK token/default/error/reconnect contract before merge.Comet's blocked consumer contract: rynfar/comet#5
Claiming implementation after reviewed upstream gate #8 merged.
- exact base:
e7871eb699d0f65047a21d179216ebfec7755d0c(origin/pylon) - branch:
fix/bounded-daemon-ingress-e787 - isolated worktree:
/Users/rynfar/.prime/worktrees/prime-bounded-daemon-ingress-e787 - owner: current Prime/Comet integration agent
Declared file set:
packages/coding-agent/src/sdk-features.ts(new)packages/coding-agent/src/modes/rpc/jsonl.tspackages/coding-agent/src/modes/daemon/daemon-client.tspackages/coding-agent/src/modes/index.tspackages/coding-agent/src/index.tspackages/coding-agent/docs/daemon.mdpackages/coding-agent/docs/sdk.mdpackages/coding-agent/test/rpc-jsonl.test.tspackages/coding-agent/test/daemon-client.test.ts- one scoped
.changes/*.mdfragment .pylon/features.yaml.pylon/upstream-review.md
The worktree is clean and no source edit has been made yet. Two read-only reviews are finishing the public SDK/raw-byte framing design and the Comet consumer proof/budget contract. Their findings will be recorded before implementation. Any required file-set expansion will be declared here first.
The contract stays additive and client-local: an exported declarative proof token, finite raw-byte bound before UTF-8/JSON, privacy-safe typed terminal error, deterministic hello/request rejection even with request recovery, no replay/auto-recovery after a framing violation, one close notification, and a clean explicit reconnect. No daemon protocol/schema/capability change is planned.
- exact base:
Joint Prime/Comet contract resolution before implementation
Two read-only reviews completed against the source now merged at exact base
e7871eb699d0f65047a21d179216ebfec7755d0c. Neither modified a checkout. Both found no design blocker if the following P0 gates are implemented and tested.Public SDK proof and options
export const PRIME_AGENT_SDK_FEATURES = Object.freeze([ "bounded_daemon_ingress_v1", ] as const); export type PrimeAgentSdkFeature = (typeof PRIME_AGENT_SDK_FEATURES)[number]; export const DEFAULT_DAEMON_CLIENT_MAX_INBOUND_FRAME_BYTES = 128 * 1024 * 1024; export interface DaemonClientOptions { readonly maxInboundFrameBytes?: number; } export class DaemonInboundFrameTooLargeError extends Error { readonly code = "daemon_inbound_frame_too_large"; readonly maxInboundFrameBytes: number; }
DaemonClient(socketPath, options?)synchronously validates a positive safe integer. The registry, type, default, options, and error are public-root exports. The token is client-artifact metadata, never a daemon capability, hello offer, protocol/schema/version gate, method/arity inference, or hostile-code attestation. Older ESM packages are namespace-imported; Comet requiresArray.isArray(PRIME_AGENT_SDK_FEATURES)plus exact token membership before construction.Prime defaults to 128 MiB. Comet #5 will explicitly use 64 MiB and a 512 MiB isolated Node heap. Stock 0.8.1 remains valid for the tiny bootstrap bridge but must fail the session-host token gate before create/attach.
Framing contract
- A separate byte-bounded JSONL reader leaves existing decoded
maxLineLength/discard-resume behavior unchanged for its current callers. - The owned
net.Socketstays in byte mode. Count raw bytes before UTF-8 decode and JSON parse. - The limit is per LF-framed record. LF is excluded; an immediately preceding CR counts, then is stripped.
- Exact limit succeeds; byte
limit + 1fails immediately, with or without LF. - Overflow is terminal for that socket epoch. Never truncate, resume after LF, or process later lines in the same chunk.
- Pending storage uses bounded owned pages rather than arbitrary subarray retention or one segment per socket write. Decode a valid completed frame without a second full raw concatenation.
- Failure/detach/end releases pages, decoder state, and partial length. Every connect gets a fresh reader.
Terminal client state
One socket-identity-gated path must:
- detach/reset the reader and clear socket, hello, and closing state;
- reject an in-progress connect, every hello waiter, and every pending request with the same typed error, always with request preservation disabled;
- clear timeouts, replay, and
awaitingReconnectstate; - notify every close listener exactly once, isolating a throwing listener;
- destroy the socket and ignore all later events from that epoch;
- suppress automatic same-peer recovery/replay after the integrity failure;
- permit only a later explicit fresh reconnect with the same immutable bound and zero retained bytes/requests.
The typed error exposes only static code and configured limit: no observed size, prefix/content, decoded text, JSON, cause, native/session ID, socket/log path, or stack projection across Comet. Comet maps it to a fixed small host control code.
Required receipts
Tests will cover raw one/many-buffer no-LF overflow, complete oversized lines, exact LF/CRLF/multibyte boundaries, EOF, terminal no-later-line behavior, option validation, immediate-connect races, hello/all-request rejection, request-recovery suppression, one close, stale-hello clearing, explicit clean reconnect/no replay, bounded snapshot chunks, the existing 36 MiB indivisible message, generated 100/500 MiB cumulative transcript attach, and public-root exports. Resource validation must account for valid-frame decode/parse memory above the raw bound.
No daemon protocol/schema/capability bump is required. Wire compatibility remains unchanged.
- A separate byte-bounded JSONL reader leaves existing decoded
File-set expansion before edit: add
packages/coding-agent/test/daemon-bounded-ingress-bench.ts.Independent resource review correctly found that the existing 100/500 MiB multi-client benchmark includes legacy monolithic/raw receivers and therefore is not proof of bounded
DaemonClientingress. The new standalone real-socket receipt will send deterministic snapshot-like JSONL chunks through the actual public client at a 64 MiB configured limit, assert every raw frame stays below the limit, reconstruct total bytes and SHA-256 on the receiving side, and support both generated 100 MiB and 500 MiB cumulative flows. The legacy comparison benchmark remains unchanged and is not counted as this safety receipt.Implementation candidate committed and hash-verified on the remote branch.
- exact base:
e7871eb699d0f65047a21d179216ebfec7755d0c - exact head:
9013a63a76fa339b5e2898947e9cff040505d72c - exact tree:
814cfa93bfdbab0d43acaa92c208205fd7dd5749 - branch:
fix/bounded-daemon-ingress-e787 - changed files: 13 (900 insertions, 31 deletions)
Validated at this source/artifact:
- focused framing/client: 53/53;
- wider affected run: 640 passes, with the sole contaminated update-restart timeout passing immediately after all inherited
PRIME_AGENT_INTERNAL_*state was scrubbed; - real supervisor process suite: 13 passed / 8 fixture-gated skips;
- stock/current 0.8.1 adoption directions plus compiled 36 MiB path: 3/3;
- bounded real-client cumulative transport: 100 MiB / 267 chunks / 269 frames and 500 MiB / 1,334 chunks / 1,336 frames, with complete SHA-256 reconstruction;
- exact 64 MiB boundary under a 512 MiB V8 heap, including high-expansion invalid UTF-8 replacement input;
- exact default 128 MiB boundary under a 1 GiB V8 heap;
npm run check, root build, installer/browser, YAML, diff check, public-root runtime/d.ts, andnpm pack --dry-runcontents;- generated
models.generated.tsrestored after each live build.
All provisional P1 findings were repaired before commit: null/invalid option validation, candidate ledger state, exact-attempt auto-reconnect fencing, immediate explicit reconnect survival and fresh policy recovery, stale hello clearing, root namespace/types, changelog attribution, accurate resource units/chunk counts, and the bounded receipt timer/pipe path. Three fresh exact-head reviews are now running: transport security, API/resource/governance, and Comet consumer compatibility. No PR will open until these approve with no P0/P1.
- exact base:
All three fresh exact-head reviews approved
9013a63a76fa339b5e2898947e9cff040505d72c/ tree814cfa93bfdbab0d43acaa92c208205fd7dd5749against basee7871eb699d0f65047a21d179216ebfec7755d0cwith no P0/P1:- transport security/concurrency;
- public API/resource/governance and package artifact;
- Comet consumer compatibility and fail-closed host budget.
Draft PR #16 is open at those exact SHAs. Its initial draft CI skips are expected and are not validation. I am marking it ready so trusted exact-head CI runs. Merge remains blocked on the new hosted run, zero unresolved conversations, and recorded exact head/base maintainer approval.
Problem
The public
DaemonClientJSONL reader currently accepts an unbounded line/buffer. A daemon or mismatched listener that sends bytes without a newline can grow every SDK consumer process until out-of-memory before the consumer can validate or normalize the message.This affects both native consumers:
Comet can isolate its bootstrap-only client in a small bounded Node process, but a long-lived asynchronous session host must not rely on process OOM as framing control.
Required outcome
This is client-local safety behavior. Do not infer support from package version, method presence, branch name, or daemon capability. If compatibility requires an opt-in API before a safe default can ship, document that migration explicitly.
Tests
Cover:
Coordination
No branch or worktree is claimed by this issue yet. Follow
PYLON.mdbefore implementation and cross-review the public client contract with both Pylon and Comet consumers.