Repository navigation
remove the manual record_refinement from the kernel harness surface - #2155
Conversation
The kernel exposed twelve per-kind harness methods that differed only in the kind they hardcoded. rlm.harness now has create(title, content, kind=...), update(id, title, content, kind=...), and delete(id, kind=...). kind defaults to memory because that is the kind models actually write today; reference and arguments stay skill-only and are rejected for other kinds. The removed names raise an error naming their replacement, so stale kernels and copied transcripts get a pointer instead of a bare AttributeError.
path read like a filesystem location for what is just a grouping label. The model-facing field, parameters, prompts, and digests now say topic. Both writers of harness_state.json (the Python kernel store and the TypeScript refinement engine) accept files that still carry path and write topic on the next save, so existing state keeps loading.
Rollback rebuilt its edits from the entry snapshot in refinement history. History written before the grouping was named topic stores it as path, so rolling back a deletion recreated the entry under the fallback grouping. Both readers now go through one accessor that accepts either spelling.
7000d9d to
484fa2b
Compare
![macroscopeapp[bot]](https://avatars.githubusercontent.com/in/900172?s=60&v=4){kind=small}
The global harness store is shared by whatever build runs next. A writer that predates the topic field drops unknown keys, so it would resave every entry ungrouped. Both writers now emit the grouping twice, as topic and as path, and readers keep preferring topic. The mirror retires once no pre-topic build can reach a shared store.
The TypeScript refinement engine writes the same file, so it emits the grouping under both names too. The transcript diff for a refinement outcome now reads the grouping through the accessor the loader and rollback already use, so an entry recorded under the older name renders its real grouping instead of nothing.
delete(id) with a mismatched kind returned False, so a bare delete of a prompt note or a skill left the entry in place and in every later prompt. update and delete now share one message that names the kind the id really has, while a plain miss still reports absence. kind is validated before the skill-only fields so an unknown kind is reported as one, upsert drops the source argument the trio never exposed, and the bootstrap failure text no longer promises the deleted per-kind methods.
3b173ca to
cd8a8b4
Compare
|
Round 1, all three items:
Validation on cd8a8b4 (rebased onto the updated #2154 head): |
Bare create/update/delete on rlm.harness read as "create what?". The per-kind wrappers were all ways to manage memories, and prompt notes, skills, and subagent specs are subtypes of memory, so the single set keeps the *_memory names and selects the rest through kind: create_memory(title, content, kind=...), update_memory(id, title, content, kind=...), delete_memory(id, kind=...). The common old call, create_memory(title, content), is unchanged; the nine other wrappers name their replacement, and an old path= argument says it is now topic.
cd8a8b4 to
5fb205c
Compare
|
Docstring fixed in 5fb205c: The branch is also rebased onto the updated #2154 head, which renamed the unified methods to |
…eader A planner can still emit the grouping as path, copying an older harness dump out of its own context, and the proposal parser was the one reader that looked only at topic; the entry then landed ungrouped. It now uses the accessor the loader, the rollback path, and the transcript diff already share.
The refinement engine writes a RefinementEvent for every refinement it applies. The manual call duplicated those events where the two overlapped and created unverifiable ones where they did not: a model could claim evidence and an outcome that nothing checked. Reading events is unchanged through get_harness_state().refinements and the harness overview, and the engine's own event writing is untouched.
plan_refinement returned a fixed three-line checklist and had no caller in the kernel, the host, the prompts, the docs, or the tests; the refinement engine plans through its own model call. It leaves with the same guidance error as the other removed names. The new refinements test now writes the engine's events after the kernel state is already cached, so it exercises the reload before save instead of a fresh load.
A proposal whose edits are all rejected still records an event, so the docstring promising applied edits was wrong.
dac6922 to
7f69a75
Compare
…rness-crud # Conflicts: # packages/coding-agent/src/core/kernel/bootstrap.ts
… into sebastian/res-1337-record-refinement # Conflicts: # packages/coding-agent/src/core/kernel/bootstrap.ts
Prime Agent performance — completedPR Overall: 0 regressed · 0 improved · 41 no clear change.
Python runtime
Session transport
UI interactions
Sandbox cost: ~$0.1032 — no inference calls. Methodology and samplesMain resolved at 2026-09-22T17:32:15.716017+00:00. Harness
|
…rness-crud # Conflicts: # packages/coding-agent/src/core/kernel/bootstrap.ts # packages/coding-agent/src/core/refinement/refinement.ts # packages/coding-agent/src/modes/interactive/components/refinement-outcome-message.ts # packages/coding-agent/test/refinement-outcome-message.test.ts # packages/coding-agent/test/refinement.test.ts # packages/coding-agent/test/suite/agent-session-prompt.test.ts # packages/coding-agent/test/suite/agent-session-queue.test.ts # packages/coding-agent/test/system-prompt.test.ts # prime-agent-runtime/src/rlm/harness.py
…only field checks, and consolidate tests
…7-record-refinement # Conflicts: # packages/coding-agent/src/core/kernel/bootstrap.ts # prime-agent-runtime/src/rlm/harness.py
…resh stale comments and the changelog
…7-record-refinement # Conflicts: # prime-agent-runtime/src/rlm/harness.py
…7-record-refinement
…ic builds round-trip the grouping
…7-record-refinement
sethkarten
left a comment
sethkarten
left a comment
There was a problem hiding this comment.
Design review passed: claim-vs-diff verified, revert-probe coverage checked, edge cases and open threads assessed. LGTM.
— Prime Agent (sethkarten's agent)
The merge-base changed after approval.
…nsolidation blocks and keeps the removal of record_refinement/plan_refinement)
sethkarten
left a comment
sethkarten
left a comment
There was a problem hiding this comment.
Design review passed; post-#2154-squash heal applied (branch drops the duplicated consolidation blocks, keeps the record_refinement/plan_refinement removal; 55/55 harness tests green). LGTM.
— Prime Agent (sethkarten's agent)
6f413ac skip superseded push builds in the release workflow (PrimeIntellect-ai#2567) 298cf40 fix: resolve the conflict markers left in the kernel prompt strings (PrimeIntellect-ai#2581) 8ee46be fix(runtime): restore snapshot functions with live namespace globals (PrimeIntellect-ai#2471) 025d134 remove the manual record_refinement from the kernel harness surface (PrimeIntellect-ai#2155) d41a061 fold the per-kind harness CRUD wrappers into create_memory/update_memory/delete_memory (PrimeIntellect-ai#2154) 8969d24 ci: drop the test-LOC budget rule from the test policy (PrimeIntellect-ai#2564) 346a463 fix(ai): keep the Opus 5.5 test biome-canonical and net-zero for the 1:1 policy (PrimeIntellect-ai#2560) b11fd5e fix(ai): give the Opus 5.5 payload test a compilable, registered surface (PrimeIntellect-ai#2555) 26ec38f fix(coding-agent): provide the required RlmChildRun label in the retained-deletion tombstone (PrimeIntellect-ai#2553) 1c1ad1e [RSI, bug] Free child names at delete receipt and return cancelled collect envelopes (PrimeIntellect-ai#2388) 1e2cc22 fix: treat claude opus 5.5 as always-on adaptive thinking (PrimeIntellect-ai#2533) 36b912f fetch model and mcp catalogs from prime-agent-catalog (PrimeIntellect-ai#2507) 9fb6cde Give ignore parsing, extension discovery and path helpers one owner (PrimeIntellect-ai#2496) c258a07 [RSI, performance] perf(coding-agent): skip identical roster broadcasts and per-delta child-update serialization (PrimeIntellect-ai#2481) 4f6df9d [RSI, performance] compaction: summarize via auxiliaryModel without evicting the session prompt cache (PrimeIntellect-ai#2411) 236ca7b [RSI, performance] Fetch session history once per switch (PrimeIntellect-ai#2432) a1792be feat: service-tier control (/tier, default-tier setting, completions forwarding, shared tier pricing) (PrimeIntellect-ai#2144) 05398e4 [RSI, feature] Headless sessions operator table for the daemon CLI (PrimeIntellect-ai#2422) f022336 [RSI, feature] Daemon incident forensics: prime-agent incident CLI and agents-view notices (PrimeIntellect-ai#2406) d73349d [RSI, bug] fix(coding-agent): absorb owned worker bridge EPIPE instead of crashing the frontend (PrimeIntellect-ai#2382) b1d82f0 [RSI, performance] stop the heartbeat/scheduler fan-out to every worker, client, and artifact path (PrimeIntellect-ai#2487) a144f9a fix(coding-agent): show a preparing turn's prompt in the TUI queued-messages area (PrimeIntellect-ai#2063) cc55bb1 [RSI, performance] perf(coding-agent): jitter provider retry backoff to de-synchronize retry waves (PrimeIntellect-ai#2474) 238a361 test(coding-agent): pin the RSI review findings within the test-line budget (PrimeIntellect-ai#2501) 893f9d5 Fix six small contract-hygiene issues found by a main audit (PrimeIntellect-ai#2493) 42be58f Move daemon response and session-summary wire guards into daemon-protocol.ts (PrimeIntellect-ai#2509) 02e5bab [RSI, performance] perf(kernel): single-pass state snapshots, bounded restore, post-restore snapshot skip (PrimeIntellect-ai#2478) 1ae4c62 Move shared Prime HTTP helpers into core/prime-http.ts (PrimeIntellect-ai#2510) 0e3c900 fix(kernel): cap error text, refuse oversized host_request payloads, and pin the kernel fixes (PrimeIntellect-ai#2500) cade9ed fix(coding-agent): stop auto-retrying provider safety-filter failures (PrimeIntellect-ai#2472) 514b4b8 [RSI, performance] perf(coding-agent): fingerprint a bounded turn-body subset for semantic-edge retry identity (PrimeIntellect-ai#2476) 561401b [RSI, performance] perf(coding-agent): cap cell source attached to kernel host requests (PrimeIntellect-ai#2475) 031f12b [RSI, performance] Throttle event-flood hot paths in child updates, roster flushes, top bar, and summarizer (PrimeIntellect-ai#2480) ab11746 [RSI, performance] perf(coding-agent): cache context-tree child parses keyed by file stats (PrimeIntellect-ai#2477) 5062b1b fix(kernel): compare recorded Python skills by key and pin the marker swap (PrimeIntellect-ai#2499) 1b453dc fix(coding-agent): daemon protocol and quota-park follow-ups (PrimeIntellect-ai#2502) a0660bb Release the daemon worker client socket after a failed connect (PrimeIntellect-ai#2508) 64a4b84 fix: one owner for the Prime team header, retry failed shell-command credentials (PrimeIntellect-ai#2497) fb8d741 fix(agent): end truncated proxy streams and send serviceTier through the proxy (PrimeIntellect-ai#2491) Gates: all passed Conflicts: auto=1 agent=6
|
oops this stack was merged by accident. reverting |
…ppers and manual refinement methods (#2670) * Revert "remove the manual record_refinement from the kernel harness surface (#2155)" This reverts commit 025d134. Conflicts in bootstrap.ts and prompts/rlm.ts (the #2581 conflict-marker cleanup touched the same lines) are resolved to the pre-#2155 text: REQUIRED_HARNESS_METHODS keeps record_refinement and the REPL control prompt names rlm.harness.record_refinement again. * Revert "fold the per-kind harness CRUD wrappers into create_memory/update_memory/delete_memory (#2154)" This reverts commit d41a061. It applied without conflicts: the only later change to the touched files was the #2581 conflict-marker cleanup, which is preserved. Restores the twelve per-kind harness CRUD wrappers and the path grouping field on both the Python and TypeScript sides, and drops the removed-name guidance that the consolidation introduced. * docs: changelog fragment for the #2154/#2155 revert * fix(refinement): migrate topic-spelled harness groupings to path on load The #2154 revert removed the topic dual-read, so harness state written while the grouping was spelled `topic` reached the loader without a `path`: entries rendered `undefined`, and an update or a rollback of a refinement from that window fell back to the `general` grouping, losing the original one for good. Both readers of harness_state.json now migrate `topic` to `path` at load, and the rollback path restores a recorded `before` snapshot's grouping from either spelling. The writers still emit the `path` spelling only, and the Python loader drops `topic` through _ENTRY_FIELDS exactly as before. Pins: a topic-spelled persisted entry loads with its grouping preserved, renders it, and saves back under `path` (TS and Python), and a legacy snapshot whose `edit.before` carries only `topic` rolls back to the original grouping.
e260085 prepare prime agent v0.9.6 release (PrimeIntellect-ai#2513) acc5bc0 [PR -- Features] Revert PrimeIntellect-ai#2154/PrimeIntellect-ai#2155: restore the per-kind harness wrappers and manual refinement methods (PrimeIntellect-ai#2670) 703cc45 bump the Claude Code client version claimed by Anthropic subscription requests (opus-5.5 gate) + ban-risk warning (PrimeIntellect-ai#2645) Gates: all passed Conflicts: auto=0 agent=0
Fixes RES-1337
What
Removes
rlm.harness.record_refinementandplan_refinementfrom the kernel surface. The refinement engine already records an event for every refinement pass it applies, so the manual call only duplicated those events or added unverified ones. Reading is unchanged:rlm.get_harness_state().refinementsand the harness overview still show recorded events.Why
evidenceandoutcomeon a manual event were model claims about work the harness never verified, and where the engine had already recorded the pass the manual call wrote a duplicate. One writer keeps the refinement list trustworthy.plan_refinementonly returned a fixed checklist that told the model to record the outcome — the call being removed — and had no caller.Behavior changes
record_refinementorplan_refinementraises anAttributeErrorthat says what to do instead:record_refinement was removed; refinement events are recorded automatically when refinements run;plan_refinement was removed; use await refine.run() to schedule a refinement(harness.py).refinement.tsis untouched by this PR).rlm.get_harness_state().refinementsandoverview()still render recorded events, and engine-written events survive a kernel-side save (covered by a new test).record_refinement; it checks the three CRUD methods (bootstrap.ts).record_refinementand no longer tells the model to "record the outcome" after a refinement; it now says events are recorded automatically (rlm.ts)._validate_refinement_event) is deleted along with its only caller.Stacked on #2154; review that one first.
Testing
Prime sandbox (node:24):
biome check --error-on-warnings .clean over 877 files,tsgo --noEmitclean,test_harness.py55 tests on CPython 3.11, full unittest suite 348 tests; vitest onrefinement.test.ts,kernel-bootstrap.test.ts,kernel-bootstrap-runtime-ready.test.ts,suite/agent-session-prompt.test.ts,suite/agent-session-serialized-refine.test.ts,suite/agent-session-compaction.test.ts,suite/agent-session-queue.test.ts— 7 files, 254 tests;acp-kernel-features.test.tskernel-heavy — 4 tests.On the final merge commit, biome (877 files) and the 55
test_harness.pytests were re-run green;scripts/check-test-policy.mjspasses against the #2154 tip (a93849d).Note
Remove
record_refinementandplan_refinementfromHarnessStateRemoves the manual refinement-recording and planning methods from the kernel harness surface so refinement events are only produced by the refinement engine.
HarnessState.__getattr__now raisesAttributeErrorwith method-specific replacement guidance when either removed name is accessed.REPL_CONTROL_PROMPTconstant, which prevent both modules from compiling as shown.Macroscope summarized 1309f55.
Note
Medium Risk
Changes the model-facing harness API and how refinement events are written, though reads and engine recording stay the same; legacy kernels or prompts that still call the removed methods will fail at runtime until updated.
Overview
Removes
rlm.harness.record_refinementandplan_refinementfrom the model-facing Python harness so refinement history has a single writer: the host refinement engine. Readingrlm.get_harness_state().refinementsandoverview()is unchanged; engine-written events are preserved when the kernel reloads and saves the same state file.Calling the removed names now raises
AttributeErrorwith guidance to rely on automatic recording orawait refine.run(). The kernel runtime ready-check and RLM system prompt no longer expose or instruct manual outcome recording; refine guidance drops the “record the outcome” step. Manual event validation and related harness tests are removed; coverage adds engine refinements surviving a kernel save and explicit errors for the old API.Reviewed by Cursor Bugbot for commit 1309f55. Bugbot is set up for automated code reviews on this repo. Configure here.