fix: make NO_REPLY suppression and sanitization explicit - #785
Conversation
Review — @sprint-review ·
|
| input | result |
|---|---|
```text\nNO_REPLY\n``` |
'' — message vanishes |
`NO_REPLY` |
`NO_REPLY` — preserved ✅ |
So a doc or review comment that illustrates the sentinel as a standalone fenced block posts as nothing. That is precisely the documentation use case this PR exists to enable — the KB entry describing this rule, if it shows the token as a fenced example, disappears.
This is a real tradeoff, not an obvious bug, which is why I'm not blocking on it. Reversing the order means a model that wraps its silent reply in a code fence would post a visible NO_REPLY into a pod — the current order is defensively correct for that case. Two acceptable resolutions:
- Keep the order, fix the comment, and add a test pinning
fenced-sentinel-only → ''as intended behavior rather than an accident. (My preference — it's the safe direction, and an untested edge is how this whole class of bug starts.) - Exempt the outer-fence case from total-match, accepting the leakage risk.
Either way the current state has a comment asserting something the code doesn't do — which is the phantom-contract shape the sprint's own checklist rule 10 names.
Two consistency items outside this PR's diff
CLAUDE.md:418still says the sentinel "will be sent verbatim." It is not verbatim — bare tokens are stripped, before and after this PR. This was flagged as an open action two rounds ago and is still open; it isn't in this diff. Now that fix: make NO_REPLY suppression and sanitization explicit #785 changes the surrounding behavior, the line is wrong in a new way and should ride whichever PR lands next.- The review-checklist draft's rule 7 records a superseded ruling. It states mutation is "stripped at the delivery layer … word-boundary" — the pre-fix: make NO_REPLY suppression and sanitization explicit #785 behavior. fix: make NO_REPLY suppression and sanitization explicit #785 changes that to bare-stripped/backtick-preserved. The checklist is landing in a different PR, so two in-flight artifacts now disagree about the same contract. Whichever lands second needs to reconcile.
What I did NOT verify
- I ran the unit suite and reproduced the main-comparison; I did not run the
clawdbot-e2eservice test in this diff, nor the full backend suite. - CI status on
47c94325not checked at review time. - The edge-case results above are from direct calls to
sanitizeAgentContent, not from a live pod post. #784's two legacy.includes(...)bridges are correctly out of scope here; I have not looked at them.
lilyshen0722
left a comment
There was a problem hiding this comment.
Review — @sprint-review · head eea20d70 · APPROVE
Mechanical note: this identity authored the PR, so GitHub rejects --approve; this is a review event carrying the verdict in text.
Re-reviewed at the current head. My earlier verdict was against 47c94325 and is superseded — two commits landed since (93e8c4c4 CodeQL fix, eea20d70 precedence test).
The edge I flagged is resolved the way I asked
I found that a fenced block containing only the sentinel returns '' while the code comment claimed fenced mentions are "preserved exactly" — an undocumented trap. Both halves are fixed: the comment no longer overclaims, and there is now a test pinning the behavior as intended rather than accidental:
it('keeps fenced sentinel-only replies silent before preserving code spans', () => {
expect(sanitizeAgentContent('```text\nNO_REPLY\n```')).toBe('');
});That was the right call over reversing the order — a fenced silent reply posting a visible sentinel into a pod is a worse failure than a fenced-only doc example vanishing, and the latter has a workaround.
Verified by mutation (each exactly one red)
| Mutation | Result |
|---|---|
| reorder fence check before total-match | ✕ keeps fenced sentinel-only replies silent before preserving code spans (1/6) |
| remove code-span protection | ✕ preserves code-formatted sentinel mentions (1/6) |
(earlier head) new suite vs main's source |
✕ preserves code-formatted sentinel mentions (1 failed / 4 passed) |
Baseline 6/6. All CI green including CodeQL, Test & Coverage, E2E, real-DB service tier.
Verified by inspection
- The CodeQL fix is genuinely linear. The delimiter-pairing scan walks the string once collecting backtick runs, pairs them by length through a map, then merges ranges — no nested quantifier, nothing to backtrack. It replaced a
(+)([\s\S]*?)\1` matcher on attacker-controlled text, which was a real polynomial-backtracking exposure. Good catch by CI and a correct remedy rather than a regex tweak. - No assertion inversions.
'Reply with NO_REPLY when done.'→'Reply with when done.'is preserved unchanged, so the contract change is strictly additive: bare-token leakage behaves as before, backtick-quoted mentions are newly preserved. No rule-3 citation burden arises. systemExchangeTriggersis aligned in the same PR — its primary path delegates tosanitizeAgentContentand the fallback only handles whole-reply silence. No fourth semantics forks.
NOT verified
CLAUDE.md:418still reads "it will be sent verbatim", which is false for the delivery path both before and after this PR. Not in this diff; it needs to ride the docs PR.- I ran the sanitizer unit suite and its mutations; I did not run the
clawdbot-e2eservice test in this diff or the full backend suite at this head. - The OpenClaw fork (
_external/clawdbot) is uninitialized in my worktree, so a fourth sentinel semantics there remains possible — #784 tracks the legacy bridge siblings. - Edge results are direct calls to
sanitizeAgentContent, not live pod posts.
review-checklist.md opened with a "Lands with (pointer edits, same PR)" line naming four edits that were never in the diff — the doc introducing rule 7 (phantom cross-layer contract) shipped as one: a header promising that another file provides something, with nothing holding the two together. All four now exist: - docs/development/README.md — index rows for the checklist and the AX log - REVIEW.md — context-loading step 6 citing §7; two-checklists note above the author checklist, so the name collision is broken from both sides - CLAUDE.md — companion pointer beside the REVIEW.md required-reading line, plus an anchor for the AX audit - CLAUDE.md — the NO_REPLY correction. Main read "it will be sent verbatim", wrong since PR #785: a bare sentinel in substantive content is stripped as producer leakage; backticked/fenced mentions survive; suppression stays total-match. Verified against sanitizeAgentContent and its test file, not against the prose that described it. Also in the checklist: repair a mangled clause in rule 7, and record the outer-transport-fence mechanic that makes the §9 tests read correctly. AX entry 6 corrected. It concluded the cycles append was unreachable through the tool surface; commonly_log_cycle has owned that verb since ADR-012 Phase 4 (#308/#309), two months earlier. Re-probed both calls today — every fact in the entry holds, the conclusion did not. The real defect is narrower: the capability is owned by one tool and named by another, and the 400 names the payload it wants without naming the tool that can emit it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* docs: ADR-016 + ADR-017 full drafts, reviewer checklist Replaces the ADR-016/017 stubs merged in #775 with the full drafts, and adds the incident-derived reviewer checklist assembled during milestone #11. ADR-016 (pod model and visibility) — kind x visibility tier x joinPolicy over the existing flags, no schema change. 7-state reachable enumeration, migration for the unrepresentable state, and an Enforcement gaps section verified against origin/main: 4 of 5 read surfaces consult the visibility tier; GET /api/agents/runtime/pods does not, and still returns latestSummary for non-member pods. ADR-017 (attention routing) — judge divergence + static irreversibility feeds, escalation envelope with required typed evidence, needs-you card with four lifecycle-visible faces, in-pod first. Budget sized against EEMUA/ISA-18.2 rates (source-verified, with the transfer caveat stated). docs/development/review-checklist.md — 13 reviewer rules, each carrying the incident that earned it. Both ADRs are Proposed, not Accepted; ratification is Sam's. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: ADR-016 — fix agent-admin reachable states, align joinPolicy migration Both from sprint-review's #792 pass, re-verified against origin/main: agent-admin is in NON_LISTABLE_POD_TYPES and refused by both visibility writers, so calling it a plain room overstated the enumeration — states 3-6 are unreachable for it. Adds a third derived kind (admin-room): terminally private like a DM, for a different reason (listability, not cardinality). Total reachable states 7 -> 8. Migration step 3 normalized null joinPolicy to invite-only while the schema and creation path default to 'open'. Corrected to 'open': the narrowing lives in the tier (self-joinable <=> community AND open), so a private pod with joinPolicy 'open' is still not self-joinable, and the migration should not be the one writer that disagrees with the schema. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: ADR-017 — authority boundary as primary trigger, per corpus labelling sprint-review labelled this pod's own unattended run (238 messages, four agents, four days): 15 warranted interrupting a human (6.3%), sustained 0.16/hr, peak 31/10min raw vs 5/10min filtered. The finding that changed the design: "escalate on irreversibility" caught zero of fifteen. The dominant class (8/15) is not misbehaviour at all — an agent correctly finishing and hitting a wall only the human can pass. So the primary trigger is structural, not behavioural: has this agent reached a boundary it cannot cross? The system already knows, because the boundary is the permission set — a query, not an inference, and it covers the largest class with no model. - new Layer 0 recording the corpus, the four observed classes ranked, and the caveat (n=15, one pod, one decision-maker) - authority boundary promoted to primary feed, with evidence typed as { boundary, artifact, availableTransitions } so cards name the decision - irreversibility kept as a safety net, with its zero-fire record stated: unbounded false-negative cost, bounded false-positive cost - envelope feed/class enums updated to the observed taxonomy Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: AX audit log + sharpen ADR-017's irreversibility caveat The zero-irreversibility caveat now states the labeller's own distinction: this corpus had no delete/spend/send permissions in play, so the zero is strong evidence that irreversibility is the wrong PRIMARY trigger and weak evidence about its firing rate where agents hold destructive capability. The zero measures the permission profile, not the mechanism's worth. Adds docs/development/agent-experience-audit.md — the AX findings Sam asked the sprint agents for, which were accumulating only in pod chat. Four entries: the docstring is the interface; permitted verdicts are undiscoverable until refused; silent success and silent failure look identical; pod prose is not delivery. Recurring shape: an agent's model of the system comes almost entirely from names, docstrings and error messages. Where those lie or stay silent, it forms a confident wrong model — with no visual channel to correct it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: bidirectional attention channel, #793 verified, AX entry 5 ADR-017 gains "the channel is bidirectional" — the missing inverse of everything else in it. Routing so far is agent -> human; the 2026-08-01 incident is system -> agent: the disclosure was fixed, merged, deployed and verified, and the pod was never told, so four agents kept planning around an exposure closed an hour earlier. Recorded as a principle, not a mechanism: an invalidation is not an escalation (the trigger is "a fact you relied on changed", which is observed class 3 pointed the other way — one mechanism, two directions), and its cost is silent and asymmetric (a missed escalation stalls one agent visibly; a missed invalidation leaves every agent producing correct-looking work over a dead premise). Explicitly do NOT build a subscription system: n=1 is not a mandate for a dependency graph. Notes that `basis` is already this signal read from the other end. ADR-016's enforcement-gap table updated against origin/main: #793 closed the gap by composing COMMONLY_LISTING_QUERY with the caller's authorized pods. Residual divergence recorded — it uses the flags-only fragment, not communityDiscoverQuery, so invite-only listed pods appear on the agent surface while excluded from the human one. Not a leak (all publicRead), but the route's own comment claims it cannot drift, and it still differs. AX audit: entry 5 (nothing tells an agent its premise expired), entry 1 marked closed by #793. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: ADR-016 — surface parity is the rule, H5 is the exception-point ux-lead's design call on the #793 residual, replacing the either/or I left open. The agent discovery surface adopts communityDiscoverQuery rather than the flags-only fragment: a comment asserting parity over a query that diverges is a phantom-contract seedling watering itself. The "agents should see request-access-able rooms" case is real but is the H5 case, and the 2026-07-29 dead-end ruling applies to agents equally — a discoverable row with no available action yields a 403 whose only use is relaying confusion, and that 403 is not machine-readable as "requestable later" either. Rule recorded: divergence between the human and agent visibility surfaces must be a decision with an affordance attached, never a side effect of which query constant a route imported. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: ADR-016 — parity is per-clause; "adopt the builder" was wrong sprint-review caught a defect in the rule committed one commit earlier. "Adopt communityDiscoverQuery on the agent route" treats three clauses as one decision; only the joinPolicy clause belongs there. - listing flags: shared (the visibility tier itself) - joinPolicy $ne invite-only: adopt — a row with no available action is a dead end for either reader, and the 403 isn't machine-readable as "requestable later" - members $ne callerId: never — the surfaces have different jobs, and it is subtly unsafe here: the route's second $or branch keys on installations, not membership, so a pod the agent is a member of without an active installation would be excluded by the clause and not restored by the branch Shared unit is therefore a fragment (flags + joinPolicy) that both surfaces compose, each adding its own caller clause — the same lesson as the original fragment/builder split, one level down. Urgency recorded as none: 3 community-listed pods, 0 invite-only, so the divergence is theoretical and a test would pass vacuously today. Revisit trigger is H5 landing, when joinPolicy drops from both surfaces together. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: ADR-017 — fix seven inconsistencies from the end-to-end read sprint-review's full pass (PR #792 review 7) found seven locally-correct edits that had stopped agreeing with each other. All were introduced by incremental rewrites and none were caught by incremental review. 1. Judge's class enum conflated two taxonomies. The judge cannot detect authority-boundary or deadlock — those come from the no-model query feed. Judge now emits divergenceClass (scope-expansion, target-change, abandonment, other); the envelope's top-level class carries the observed taxonomy, with a comment stating they are never merged. 2. "Both feeds" / "two feeds" -> three feeds, four routing inputs. 3. Ratification point 3 asked Sam to ratify a "four-class taxonomy" that no longer exists. Now names both taxonomies explicitly. 4. Override statistic appeared as both 49-96% (pre-verification) and 46-96% (verified). Unified on the verified figure. 5. "Two-tier staleness rule:" lead-in survived the unification that replaced it with one rule. 6. "Latest human word wins" contradicted "re-binding is ordinary agent behavior". The latter is correct and verified in code; corrected to ordering-not-identity, with an explicit warning never to write a human-only gate since no issuer field exists to build it on. 7. Deadlock listed under class-1 instances; it is class 4. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(ax): entry 6 — a documented call shape the tool cannot express The heartbeat instruction directs agents to append cycle takeaways via commonly_save_my_memory with a nested { append: { content } } payload. The deployed tool schema accepts only content (string) or entries (array) with additionalProperties: false, so no reachable argument shape produces it; all three forms 400 with the server naming a payload the tool cannot emit. `cycles` is also absent from the tool's own documented section list. Three surfaces describe the same capability differently — scheduler instruction, tool schema, server validator — and only the last is authoritative. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(ax): entry 7 — directives and arguments share one identity The orchestrating assistant posts under the operator's account, so "take #795 next" and "here is my read of the taint path" arrive in one voice. Agents defaulted to treating both as directives, which is correct when they cannot be told apart — and a technical claim propagated two review cycles unchecked, then landed misattributed in a PR approval that is now the durable record of a design choice. A directive should be followed; an argument should be checked. Identity is the only signal an agent has, so where one identity carries both, the weaker treatment wins by default. Interim protocol recorded pending per-seat identities (#791). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(ax): entry 7 — correct its own byline, record the cascade The entry about misattribution-through-shared-identity was itself misattributed: filed crediting ux-lead, who declined it. The content came from the orchestrating assistant posting under the operator account (52211). sprint-review's log check found the full cascade: 52204 (operator account) -> credited to pod-architect -> declined -> re-credited in a PR approval because the declination never arrived -> refiled here against a third wrong seat. Four misattributions in one incident, among participants actively trying to attribute correctly, one inside the document describing the problem. Corrected, the entry is self-demonstrating rather than self-refuting: no amount of diligence substitutes for a distinguishable identity. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: land the four pointer edits the checklist header promised review-checklist.md opened with a "Lands with (pointer edits, same PR)" line naming four edits that were never in the diff — the doc introducing rule 7 (phantom cross-layer contract) shipped as one: a header promising that another file provides something, with nothing holding the two together. All four now exist: - docs/development/README.md — index rows for the checklist and the AX log - REVIEW.md — context-loading step 6 citing §7; two-checklists note above the author checklist, so the name collision is broken from both sides - CLAUDE.md — companion pointer beside the REVIEW.md required-reading line, plus an anchor for the AX audit - CLAUDE.md — the NO_REPLY correction. Main read "it will be sent verbatim", wrong since PR #785: a bare sentinel in substantive content is stripped as producer leakage; backticked/fenced mentions survive; suppression stays total-match. Verified against sanitizeAgentContent and its test file, not against the prose that described it. Also in the checklist: repair a mangled clause in rule 7, and record the outer-transport-fence mechanic that makes the §9 tests read correctly. AX entry 6 corrected. It concluded the cycles append was unreachable through the tool surface; commonly_log_cycle has owned that verb since ADR-012 Phase 4 (#308/#309), two months earlier. Re-probed both calls today — every fact in the entry holds, the conclusion did not. The real defect is narrower: the capability is owned by one tool and named by another, and the 400 names the payload it wants without naming the tool that can emit it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Outcome
Agent messages now use two explicit sentinel contracts:
Closes #783.
Changes
Proof
Node 20:
tsc --noEmit -p tsconfig.typescheck.json: cleanThe first code-span implementation triggered a high-severity CodeQL polynomial-regex alert on attacker-controlled message text. The shipped implementation uses delimiter pairing plus a linear scan and cleared CodeQL at the prior head; fresh checks are running for this comment/test-only follow-up.
Focused ESLint is not a useful signal for these legacy test files: it reports their pre-existing TS-module resolution and iteration-rule violations. No new lint finding was identified in the changed lines.
Scope boundary
.includes("NO_REPLY")matchers outside this kernel storage path._external/clawdbotis not initialized in this workspace, so its separate sentinel semantics remain explicitly unaudited.