Skip to content

feat(coding-agent): unify synthetic user-channel messages under one bracket grammar (RES-1321) - #2115

Merged
snimu merged 18 commits into
mainfrom
snimu/synthetic-message-grammar
Sep 10, 2026
Merged

snimu merged 18 commits into
mainfrom
snimu/synthetic-message-grammar

Conversation

@snimu

@snimu snimu commented Sep 8, 2026 •

Copy link
Copy Markdown
Contributor

Fixes RES-1321

What

Every machine-injected user-channel message now opens with one bracket grammar: first line [<kind>(: <qualifier>)( <address>)] (kebab-case kind), a blank line, then the payload. Machine data (message ids, endpoint ids, pids, schedules) lives in message.details, never in prose. Detection keys on customType, never text regexes. Refinement notices ([auto-/user-/self-refinement], from #2098) already followed this grammar and are untouched; they are the reference implementation.

Per-kind before/after

Kind Before (first lines) After
agent_message 5-line header (Agent-to-agent message received. / Source: / From: / To: / Message id:) [agent-message from <relationship>:<name>] + blank + body
rlm_child_terminal_notice RLM child <name> (<uuid>) completed without sending a reply. Last assistant text: ... [child-exited: no-reply child:<name>] + optional Last assistant text: ...
RLM child <name> (<uuid>) was cancelled: <reason> [child-exited: cancelled child:<name>] + optional reason line
rlm_child_failure RLM child <name> (<uuid>) failed: <error> [child-failed child:<name>] + error payload
async_bash_completion Shell message received. / Source: bash / Command completed (pid ..., exit code ...) / Command: ... / standing Inspect the saved BashHandle... hint [bash-done pid:<pid> exit:<code>] + Command: <cmd>; hint DELETED (duplicated the system prompt)
heartbeat_prompt raw job.prompt, no marker (masqueraded as the user) [heartbeat: <schedule> run#<n>] + blank + prompt
goal_context <goal_context>...</goal_context> XML wrapper [goal: continuation|budget-limit|objective-updated] + prompt
autonomous_status Autonomous mode: on. Continuations: ... (unmarked) [autonomous-status: on|off] + usage payload
autonomous continuation raw continuation prompt as a plain user message (unmarked) [autonomous-continuation] + prompt; gate retries: [autonomous-continuation: gate-failed] + failure report (one owner: buildAutonomousGateFailureContinuation, also covers the headless injector)
prime-agent.update_complete Prime Agent updated to v... (unmarked) [update-complete] + notice
ipython_state <ipython_state>...</ipython_state> [python-state] + blank + body
ipython_state_restored <ipython_state_restored>...</ipython_state_restored> [python-state-restored] + blank + body (customType string kept for compat)
compactionSummary prefix explanatory sentence + <summary> [compaction-summary] first line; sentence and <summary> delimiters kept
branchSummary prefix explanatory sentence + <summary> [branch-summary] first line, same treatment
harness digest (#2098) explanatory sentence + <harness_state> [harness-digest] first line; one owner (HARNESS_DIGEST_PREFIX) covers both the standalone message and the compaction-summary rider

No relay path exists for agent messages today, so no relaying grandchild:<y> form was invented.

All remaining built-in customTypes are either filtered out of LLM context by convertToLlm (session_slash_command, session_slash_command_result, compaction_outcome, refinement_outcome) or entry-only (thread_goal_state, rlm_max_depth_state, ipython_sent_agent_message, prime-agent.refinement). A table-driven pin walks every constructor-backed synthetic kind and asserts a grammar-conforming first line. Status/continuation/update notices regenerate rather than being re-parsed from transcripts, so no accept-old shims were added for them.

Detection migration + compat

  • clearQueuedAgentMessages now matches queued actions by the delivered custom message's customType (isAgentSessionMessage), with isAgentSessionMessagePrompt retained only as the legacy-text fallback.
  • parseAgentSessionMessagePromptId / isAgentSessionMessagePrompt keep parsing the old header format forever (accept-both, emit-new); persisted old-format transcripts still resolve their message ids and are pinned in tests.
  • New-format prompts intentionally carry no id in text; id resolution uses customMessage.details.id.
  • TUI renderers for every kind read customType/details, not prose (verified; no renderer changes needed beyond test pins).
  • Goal prose now says "Python REPL" instead of "ipython" (tool-name references keep ipython; that rename is a separate later PR).

Tests

One pin per variant format string, merged into existing suites: bus/fire-and-forget (agent-message grammar + legacy parse + sanitization), 4482 (heartbeat marked, provider passthrough), async-bash-completion + 2068 (bash-done grammar, hint gone), recursion + 617 (child-exited/child-failed variants), goal suite (goal kinds), 4530 (python-state-restored), 4531 (customType-keyed clearing of new-format queued messages), session-command-messages (per-kind convertToLlm passthrough + compaction/branch bracket lines), compaction suite (harness-digest ordering). Net: +244/-194 lines.

Validated in a Prime sandbox: coding-agent suites for messages/agent-messages/kernel bridges/heartbeat/goal/compaction/interactive rendering/regressions (only the documented environmental failures: kernel-* bridges, recursion asyncio-kernel test, 4428, 4603 — all pass locally with a real kernel) plus root npm run check.


Note

Medium Risk
Broad changes to LLM-visible prompt text and queue-clearing logic, but behavior is guarded by customType, legacy parsers, and extensive test pins.

Overview
Machine-injected messages that reach the model as user-channel text now share a single bracket header pattern ([<kind>…], blank line, body) instead of multiline headers, XML wrappers, or raw unmarked prompts.

Agent-to-agent prompts shrink from the old Agent-to-agent message received. block to [agent-message from <relationship>:<name>], with ids and routing metadata kept in details. Heartbeats, async bash completions ([bash-done …], BashHandle hint removed), RLM child notices, goal context, Python kernel state, autonomous status/continuations, post-update notices, and compaction/branch/harness digest prefixes all move to the same grammar. New sanitizeMessageHeaderValue strips characters that could break or spoof headers.

Detection shifts toward customType (e.g. clearQueuedAgentMessages matches the queued custom message, not only legacy text); parseAgentSessionMessagePromptId remains for old persisted transcripts only.

Reviewed by Cursor Bugbot for commit 2480f8f. Bugbot is set up for automated code reviews on this repo. Configure here.

Note

Unify synthetic user-channel messages under bracket grammar in coding-agent

  • Replaces multiline and XML-like wrappers with compact bracket headers (e.g., [bash-done], [python-state]) for machine-injected messages across coding-agent.
  • Adds sanitizeMessageHeaderValue in messages.ts to strip whitespace, commas, colons, and closing brackets from header metadata.
  • Updates message builders for bash completions, RLM child events, heartbeats, goals, compaction summaries, and autonomous continuations to use the new grammar.
  • Behavioral Change: AgentSession.clearQueuedAgentMessages in agent-session.ts now identifies current agent messages by custom message type instead of legacy text predicates, though legacy text parsing is retained for older persisted transcripts.

Macroscope summarized 2480f8f.

snimu added 12 commits September 7, 2026 22:23
…and deliver harness state in-context

Fixes RES-1318.

Applying a refinement no longer rebuilds and swaps agent.state.systemPrompt,
so the provider prefix cache survives every refinement. Instead:

- _applyRefine appends one durable refinement_notice custom message per
  applied refinement at the apply seam, labeled [auto-refinement] /
  [user-refinement] / [self-refinement] by source, with the trigger line and
  applied edits in the digest's own notation (rollbacks print as rollbacks).
  Zero-applied-edit refinements emit nothing and the notice never starts a
  turn. convertToLlm passes the new type through as a user message while the
  refinement_outcome audit message stays filtered and untouched.
- The harness digest injection is removed from the system prompt entirely
  (buildSystemPrompt loses the harnessState option). The digest now appears
  at cold context boundaries only, reusing formatHarnessStateForPrompt
  byte-identically: fresh sessions get a digest custom message as the first
  context message; compaction attaches a digest snapshot to the compaction
  entry mechanically (never through the summarizer) so the post-compaction
  head message renders memories-first before the summary on both the initial
  and update-merge paths; resumes append a fresh digest at the tail only when
  the newest digest in live context no longer matches disk state (identity
  dedupe), which is also the migration path for pre-change sessions.

The new compaction entry field and custom message types ride the existing
schema-tolerant JSON entry stream (optional field, generic custom messages):
backward-compatible, no daemon protocol or schema revision change.
…harness digest

Every session now opens with a harness_digest custom message, so tests that
pinned exact message sequences, first-message identity, provider-context text
lists, or entry counts see one extra leading custom message. Adds a
conversationMessages helper to the suite harness (filters the digest) and
updates the affected assertions; provider-context probes now select the last
user message instead of the first.
… of the summarizer

Review fixes on #2098:

- _latestContextHarnessDigest picked the last digest by array position, but
  retained pre-compaction messages are presented after the compaction head
  while being chronologically older, so an old retained digest could defeat
  resume dedupe. Recency is now the greatest message timestamp among all
  in-context digest carriers.
- The session-start harness digest entry was fed to the first compaction's
  summarizer as an ordinary custom message. prepareCompaction now excludes
  harness_digest entries from summarizer input; the digest is only attached
  mechanically at the new compaction head.
…finement

Review fix on #2098: _consumePendingRequestedRefine forwarded agent-callable
refine.run requests through refine() without a source, so the notice carried
the [user-refinement] label in non-serialized sessions. refine() now accepts
an explicit source override and the pending-request consumer passes "self".
…ontext everywhere

Review round on #2098 (three findings):

- Untouched sessions stay empty: the fresh-session digest is now injected
  lazily at the first committed turn instead of at construction, so
  abandoned-draft cleanup and raw message-count emptiness checks
  (daemon-session-list lifecycle/activity, isEmptyDraftContent, branch
  seedability, hasExistingSession) keep seeing an empty session. Non-empty
  contexts (resume) still inject at construction with identity dedupe.
- Tree navigation is a cold boundary: navigateTree rebuilds the context, so it
  now runs the same refresh-iff-stale digest check as resume (fork/clone/switch
  construct a new session and were already covered by the constructor path).
- Headless terminal selection skips harness_digest like other non-output
  customs, so resume-and-print returns the saved final assistant output.

Sweep of other message-kind-sensitive consumers: branch summarization now
excludes harness_digest from summarizer input (same rule as compaction);
estimateContextTokens and side-question seeding intentionally keep the digest
(it is live context); _repliedToParentSinceTask and getUserMessagesForForking
are kind-filtered already.
…ts on the digest work

Comments cut to one-line invariant guards; duplicate digest-notation and
content asserts removed where a kept unit pin already covers the same
rendering (labels and notation stay pinned in refinement-outcome-message.test,
source plumbing stays pinned per path). No behavioral change.
…ers it

Review fix on #2098 (Bugbot thread 2919fcf3): the first-turn digest was
persisted directly at commit, before agent.prompt ran, so cancelling or
clearing that first turn stripped the turn's captured messages but left the
digest in state and in the session file - resurfacing the emptiness-check
class through the cancel path. The digest now rides the turn's next-turn
delivery records: it persists at message_end like the rest of the turn,
first-turn cancellation strips and restores it with the turn, and the lazy
injection re-delivers on the next committed turn.
… and re-arm on cancel

Two follow-up review findings on the lazy-injection shape:

- Custom-triggered turns use the skip next-turn-context policy, so a fresh
  session whose first turn was sendCustomMessage(triggerTurn) ran without the
  digest. The pending digest now attaches on every committed first turn; skip
  policy still withholds ordinary pending next-turn messages.
- A cancelled first turn whose digest record was already durable dropped the
  digest from live context without restoring it (restorable records skip
  durable ones). Lazy injection now owns digest delivery uniformly: cancelled
  turns never restore the digest message and instead re-arm the pending flag,
  so the next committed turn re-delivers a fresh one.
… context

Review fix on #2098 (Bugbot skip-turn-digest-lost-on-failure): a failed commit
parked the undelivered digest in _pendingNextTurnMessages, which skip-policy
custom-trigger turns never drain, and a parked copy plus a re-armed injection
could double-deliver. Parking now filters the digest out and re-arms
_harnessDigestPending instead, keeping lazy injection the single owner of
digest delivery.
…racket grammar

Every machine-injected user-channel message now opens with one grammar:
`[<kind>(: <qualifier>)( <address>)]`, a blank line, then the payload.

- agent_message: `[agent-message from <relationship>:<name>]`; message id and
  endpoint ids live in details only. Detection keys on customType; the old
  header parser stays for persisted legacy transcripts (accept-both, emit-new).
- heartbeat_prompt: `[heartbeat: <schedule> run#<n>]` instead of the raw
  prompt masquerading as the user.
- async_bash_completion: `[bash-done pid:<pid> exit:<code>]`; the standing
  BashHandle inspection hint is gone (it duplicated the system prompt).
- rlm child notices: `[child-exited: no-reply|cancelled child:<name>]` and
  `[child-failed child:<name>]`.
- goal_context: `[goal: continuation|budget-limit|objective-updated]` replaces
  the <goal_context> XML wrapper.
- ipython_state_restored: `[python-state-restored]` label (customType keeps
  its string); goal prose says "Python REPL" instead of "ipython".
- compactionSummary/branchSummary/harness digest prefixes gain
  `[compaction-summary]`/`[branch-summary]`/`[harness-digest]` first lines.

clearQueuedAgentMessages now matches queued actions by the delivered custom
message's customType, with the legacy text parser as fallback.

Fixes RES-1321
Comment thread packages/coding-agent/src/core/messages.ts Outdated

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 3 potential issues.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Want reviews to match your repository better? Bugbot Learning can learn team-specific rules from PR activity. A team admin can enable Learning in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 5945f93. Configure here.

Comment thread packages/coding-agent/src/core/agent-messages.ts
Comment thread packages/coding-agent/src/core/messages.ts Outdated
Comment thread packages/coding-agent/src/core/agent-messages.ts
One owner (sanitizeMessageHeaderValue) strips brackets, newlines, commas,
and ":" from header-interpolated names: an RLM child name can no longer
close the [child-failed]/[child-exited] header early, and an unlabeled
sender named "parent:root" can no longer render as a relationship label.

Addresses the Bugbot/Macroscope review findings on PR 2115.
Comment thread packages/coding-agent/src/core/agent-session.ts
…ce-bearing schedules

parseAgentCronSchedule's every/in regexes accept embedded newlines, so the
schedule shown in the [heartbeat: ...] header goes through the shared
header sanitizer; the exact expression stays in details.schedule.
…acket grammar

Review sweep of every built-in customType against convertToLlm found three
model-visible kinds still emitting unmarked content:

- autonomous_status -> `[autonomous-status: on|off]` + usage payload
- autonomous continuations -> `[autonomous-continuation]` + prompt, and
  `[autonomous-continuation: gate-failed]` owned by
  buildAutonomousGateFailureContinuation (covers the headless injector too)
- prime-agent.update_complete -> `[update-complete]` + notice
- ipython_state (kernel persisted through compaction) -> `[python-state]`

All other custom types are either grammar-compliant or filtered out of LLM
context by convertToLlm / entry-only. A table-driven pin now walks every
constructor-backed synthetic kind and asserts a grammar-conforming first line.
Base automatically changed from feat/refinement-cache-visibility to main September 9, 2026 02:31
…ge-grammar

# Conflicts:
#	packages/coding-agent/src/core/messages.ts
#	packages/coding-agent/test/suite/agent-session-action-races.test.ts
#	packages/coding-agent/test/suite/agent-session-compaction.test.ts
#	packages/coding-agent/test/suite/agent-session-prompt.test.ts
#	packages/coding-agent/test/suite/regressions/4482-heartbeat-injected-prompt.test.ts
… bracket grammar

The full CI matrix (first run since the PR retargeted main) caught two
expectations outside test/suite still pinning pre-grammar text: the retained-
subagent send now asserts the exact `[agent-message from ...]` prompt, and the
model_select injected-prompt pin carries the `[heartbeat: ...]` header.
@snimu
snimu merged commit 5a84302 into main Sep 10, 2026
23 checks passed
@snimu
snimu deleted the snimu/synthetic-message-grammar branch September 10, 2026 17:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants