Skip to content

fix(*): stop naming the memory files when a backend owns memory - #894

Merged
gloryfromca merged 2 commits into
mainfrom
fix/memory_prompt_under_a_backend
Oct 10, 2026
Merged

gloryfromca merged 2 commits into
mainfrom
fix/memory_prompt_under_a_backend

Conversation

@gloryfromca

@gloryfromca gloryfromca commented Oct 10, 2026 •

Copy link
Copy Markdown
Member

Summary

With a MemoryBackend plugin wired, memory lives in the plugin's store and reaches the
turn as the # Memory section. The identity block still named two files under agent home,
so "remember this" became an edit_file against user_memory/profile/user.md. On a
non-interactive turn that write is refused at the ask tier, and the agent reported the
refusal as the memory system having failed -- while the backend had already stored the
turn and recalled it in the next session.

identity_text takes a has_memory_backend flag, threaded from the one place each caller
already holds the backend: build_context_engine, and AgentLoop for the
token-estimation prompt in ContextBuilder, which must render what the turn renders.

With a backend the two file lines become one:

- Agent home: <home> -- your own skills, not a place for user artifacts.
  - Memory is not yours to write: what is relevant is recalled into the `# Memory`
    section each turn, and the host persists the conversation after it ends. You
    cannot observe that write, so never report it to the user as done or as failed.
  - Custom skills: <home>/skills/{skill-name}/SKILL.md

Three clauses, three jobs. Do not write it. It is being saved, so stop looking for a place
to put it. You cannot see that write, so do not narrate it.

The last clause is the other half of the same defect, and the third item issue 440 asks to
separate. backend.store is dispatched after the turn ends, so a claim either way is one
the agent never checked -- and whatever it claims is extracted into long-term memory next
turn as if it were an observation of the system. The issue's own repro produced exactly
that: an episode titled Memory Save Failure for Green Brompton Bike 'Pickle' Noted on 2026-09-16, whose content was right and whose narrative was wrong. Filtering that
server-side is not available from here, so the prompt is where it is addressed.

Without a backend the rendered prompt is unchanged, byte for byte. Whether the episodic
log earns its line at all is issue 122's question, and answering it on this branch would
couple the two.

The approach is AmirF194's, from the closed PR #216. That branch cannot be reopened: it
predates the history linearization, so it shares no merge base with main
(git merge-tree refuses unrelated histories), its six files no longer apply, and it did
not allow maintainer edits. It is reimplemented here against current main, with the
author credited as co-author on the commit.

Type

  • Fix
  • Feature
  • Docs
  • CI / tooling
  • Refactor
  • Other

Verification

Run on this branch's head, rebased onto main at c3d2821, Python 3.12, --extra dev.

  • pytest over the 17 test files that construct ContextBuilder, IdentitySegmentBuilder,
    build_context_engine or call identity_text (the git grep -l list): 756 passed, 0 failed.
    SERPER_API_KEY is unset for the run; with it set,
    test_agents_code_launcher.py::test_the_products_tool_face_is_the_forks_config_intent_minus_the_ledger
    fails on an extra web_search in the tool face, on this branch and on the base alike.
  • The three new tests were each watched failing without the change, by neutering the
    behaviour in place rather than the signature, then restoring the file and confirming it
    byte-identical by checksum:
    • forcing the backend branch off reddens test_identity_with_a_backend_names_no_memory_file;
    • dropping the flag from ContextBuilder._get_identity reddens
      test_identity_with_a_backend_matches_the_estimation_prompt;
    • test_identity_without_a_backend_still_names_both_memory_files pins the unchanged half.
  • The no-backend claim is measured, not argued: the whole identity block rendered from the
    base and from this head is 3341 bytes on both sides and diff is empty.
  • make lint-python, make lint-types, make lint-imports, make check-large-files -- all exit 0.
  • RANGE=github/main..HEAD scripts/check_source_language.py -- exit 0.
  • commitlint --config commitlint.config.cjs on the commit body and on the prospective
    squash header -- both exit 0.

Driven against a live EverOS after this description was first written. All three
symptoms issue 440 describes reproduce on the merge base and none reproduce here: on
main turn 1 calls read_file then edit_file on user_memory/profile/user.md and
grows it to 897 bytes, at the ask tier that write is refused and the agent reports the
save as failed, and EverOS extracts that false report into four durable memory entries.
On this branch turn 1 makes no tool call, agent home is untouched, and the same grep over
the store returns nothing. Both sides recall the fact in a fresh session. Full method,
verbatim output and teardown in a comment on this PR.

One run per cell, not a distribution. The host still dumps user.md into # Memory with
a backend wired, which is unchanged here and out of scope below.

  • Relevant tests pass locally
  • Relevant lint / type checks pass locally
  • User-facing docs or screenshots are updated when needed

No docs change: the only user-facing text is the system prompt itself, and no document
quotes this block.

Risk

User-visible change is confined to installs that have a memory backend wired. There, three
prompt lines become three different ones; nothing else in the prompt, and no behaviour
outside prompt assembly, moves. Installs without a backend render a byte-identical prompt,
which is the measurement above.

has_memory_backend defaults to False on all four signatures it is added to, so every
call site not updated here renders what it rendered before. The three updated call sites
are the only ones that hold a backend.

Rollback is a revert of the single commit. Nothing persists and no format changes.

Checked and deliberately not changed:

  • MemorySegmentBuilder still calls get_memory_context() unconditionally, so user.md
    content reaches # Memory with a backend wired. Turning that off is the question of
    whether two memory mechanisms should coexist, which is larger than this issue and which
    four comparable systems answer by keeping both.

  • The episodic log line, under the default no-backend configuration. It belongs to issue 122.

  • Security impact considered

  • Backward compatibility considered

  • Rollback path is clear for risky changes

Related Issues

Fixes #440 -- items 1 and 3 of the three the issue asks to separate, both reproduced on
the merge base and verified gone here. Item 2 landed in #544.

#122 -- untouched, deliberately.

With a MemoryBackend plugin wired, memory lives in the plugin's store and
reaches the turn as the `# Memory` section. The identity block still named
two files under agent home, so "remember this" became an edit_file against
user_memory/profile/user.md. On a non-interactive turn that write is refused
at the ask tier, and the agent reported the refusal as the memory system
having failed, while the backend had already stored the turn and recalled it
in the next session.

identity_text takes a has_memory_backend flag, threaded from the one place
each caller already holds the backend: build_context_engine, and AgentLoop
for the token-estimation prompt in ContextBuilder, which must render what
the turn renders.

With a backend the two file lines become one that says memory is not the
agent's to write, that the host persists the conversation after the turn
ends, and that the agent cannot observe that write so must never report it
as done or as failed. The last clause is the other half of the same defect:
the store is dispatched after the turn, so a claim either way is one the
agent never checked, and whatever it claims is extracted into long-term
memory next turn as if it were an observation of the system.

Without a backend the rendered prompt is unchanged, byte for byte. Whether
the episodic log earns its line at all is a separate question with its own
issue, and answering it here would couple the two.

The approach is AmirF194's, from the closed PR #216. That branch cannot be
reopened: it predates the history linearization, so it shares no merge base
with main, its six files no longer apply, and it did not allow maintainer
edits.

Co-authored-by: Amir Fathi <amirfathi.me@gmail.com>
Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>

@gloryfromca gloryfromca left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: remove the new issue-number references from Python source docstrings.

The backend-presence signal is otherwise threaded consistently through the production loop, token-estimation renderer, unified context factory, and curator wrapper. The no-backend default preserves existing callers, and the tests were strengthened rather than weakened.

Covered: the full diff; AGENTS.md and CONTEXT-MAP.md/Runtime terminology; changed-component callers and backend recall/store lifecycle; relevant history; backward compatibility; test changes; and architecture boundaries. After syncing the declared dev extra with uv sync --extra dev, all 403 tests in every file that directly constructs or calls the changed components passed. The initial run did not reach tests because the fresh environment lacked raven_everos; no test was skipped or weakened.

No live EverOS behavioral run was performed. The remaining item is the repository-rule violation marked inline.

Comment thread tests/test_segments.py Outdated
@gloryfromca

Copy link
Copy Markdown
Member Author

Run against a live EverOS

The description said this had not been driven against a real backend. It has now, and the
result is better than the branch claimed: all three symptoms issue 440 describes reproduce
on main and none of them reproduce here.

Four isolated RAVEN_HOMEs, each with its own fresh EverOS root and port (18796 / 18798 /
18800 / 18801), memory.backend = "everos", owned: true, extraction LLM on deepseek and
embedding on Qwen3-Embedding-4B. main is the PR's merge base c3d2821, loaded over the
same interpreter by PYTHONPATH and confirmed at runtime by resolving
render.identity_text to the base tree and asserting has_memory_backend is absent from
its signature. Both sides ran the issue's own two turns verbatim.

main (c3d2821) this branch
tool calls, turn 1 read_file then edit_file on user_memory/profile/user.md none
cost of that turn 5 model calls, 44.9k in / 871 out, 18.8s 1 model call, 13.9k in / 208 out, 7.5s
user.md after turn 1 897 bytes, contains Pickle 837 bytes, untouched template
same turn at --permission-mode ask edit_file refused at the ask tier; agent reports the save failed no attempt; "Got it -- your bike is a green Brompton called Pickle. I'll remember that."
backend store has the fact has the fact
the agent's claim, extracted into memory 4 entries 0
turn 2, fresh session recalls correctly recalls correctly

Item 1, verbatim

main, one-shot CLI at the ask tier, is the issue's screenshot:

Raven
记下了:你的车是绿色的 Brompton,叫 Pickle。不过这一条我没能写进长期记忆文件 ——
这个会话不是交互式的,写入我的个人资料需要批准。目前它只在本对话里有效;
等你有交互会话时我可以再存一次。

1 action(s) were refused in this run:
  - edit_file: ... path='/private/tmp/cla.../profile/user.md'
    This call requires user approval (ask tier), but this turn is not interactive

("Noted: your bike is a green Brompton called Pickle. But I could not write this into the
long-term memory file -- this session is not interactive and writing my profile needs
approval. For now it only holds in this conversation; I can save it again when you have an
interactive session." The locale is the host's language setting, not part of the repro.)

The backend had stored it the whole time. Same command on this branch, same home layout:

Raven
Got it -- your bike is a green Brompton called Pickle. I'll remember that.

No refused action, nothing written under agent home, and the next session answers "Your
bike is green, a Brompton, and it's called Pickle."

Item 3, which is why the last clause is in the prompt

This is the part worth a reviewer's attention, because it is the half that is easy to
dismiss as belt-and-braces. EverOS extracted the agent's false report from the main run
and stored it as four separate memory entries:

2026-10-10 05:43 UTC, the assistant confirmed it had noted the information (green
  Brompton, called Pickle), but stated that this item could not be written to the
  long-term memory file because the session was not interactive and writing its
  profile requires approval; the information holds only in this conversation ...
2026-10-10 05:43 UTC, the assistant stated this item could not be written to the
  long-term memory file
2026-10-10 05:43 UTC, the assistant stated the reason ... was that writing its profile
  requires approval
2026-10-10 05:43 UTC, the assistant stated the reason ... was that the session was not
  interactive

Four durable, recallable records of something that never happened. The same grep over the
branch's store returns nothing.

Scope of the evidence

One run per cell, not a distribution; the wording a model picks varies and these are
single samples. What does not vary is the tool-call trace and the bytes on disk, and those
are what the table turns on. The host still dumps user.md into # Memory with a backend
wired -- unchanged here, and noted in the description as deliberately out of scope.

Teardown: the four EverOS servers were killed by pid, the temporary main worktree
removed, and ~/.raven/config.json and ~/.everos/raven/everos.toml confirmed unmodified
by mtime.

Comment thread raven/agent/loop/main.py
@LivXue

LivXue commented Oct 10, 2026

Copy link
Copy Markdown
Member

No blockers in the reviewed scope.

Scope: I reviewed only the raven/agent/** portion of this PR -- 2 of 6 changed files (raven/agent/context/builder.py, raven/agent/loop/main.py). The remaining 4 files were read as context but are not covered by this verdict. That includes the open blocker at tests/test_segments.py:111 (issue numbers in the new test docstrings), which still applies at this head; this summary neither re-files nor clears it.

I examined the in-scope diff and where its backend comes from: engine.backend, the same object self.backend holds, build_context_engine(backend=...) receives, and the curator lane's MemoryContext.bind reads back as loop.backend, so the turn and the estimate are decided from one object. Also every ContextBuilder construction site (the loop's, and raven/agent/subagent/backends/raven_loop.py:166, which renders only the runtime-context block), both estimation consumers of the builder (harness.memory and MemoryConsolidator(build_messages=...)), whether anything rebinds loop.backend after construction, the post-turn store path the new prompt text describes, and the cargo/loop seat contract.

Confirmed findings:

  • Nonblocking, inline at raven/agent/loop/main.py:333: no test drives the decision of has_memory_backend, here or at its turn-side twin raven/context_engine/factory.py:146; the new parity test builds both renderers by hand with the flag already set. Deleting either line leaves the full suite green, and deleting the factory one puts both memory files back into the prompt with a backend wired.

Verification:

  • git archive dc8f95aa into a scratch tree, then python -m pytest -q from it with the tree and its three plugins-dist members first on PYTHONPATH (Python 3.12.13, the repo's addopts, SERPER_API_KEY unset): 23 failed, 27643 passed, 122 skipped, 2 errors at collection, none in identity, segments, context or the loop. CI on this head is green.
  • The same run on two copies, each with one has_memory_backend=backend is not None line deleted (main.py:333, factory.py:146): failing IDs identical to the unmutated tree in both. A probe that builds AgentLoop with a stub backend through EngineWiring(backend=...) shows each deletion is observable.
  • identity_text rendered from the merge base and from this head with the same arguments and no backend: cmp reports them identical, matching the description's byte-for-byte claim.
  • The import sets of both in-scope files are identical at base and head, so the cargo/loop seat contract cannot have moved.

Checked and not reported:

  • The flag is captured at construction while StorePipeline reads self.backend live. Only tests rebind loop.backend (four assignment sites in three files, none of which mentions the identity), and RavenRuntime documents that a memory backend only arrives with the next generation, so the snapshot sits on that boundary.
  • "The host persists the conversation after it ends": both dispatch sites in turn_path.py call the store unconditionally after _save_turn; StorePipeline.enqueue skips only with no backend or an empty slice; a turn it sheds under backpressure goes through _note_memory_failure, the path that raises memory.health after a run of failures; and writes still unsettled at exit are reported by report_memory_write_outcome. That is the division of labour the last clause assumes.
  • Reach: MemoryConfig.backend defaults to "everos" and all five agents/*/config.json set it, so wherever everos-memory is installed the backend branch is the default prompt, including the four products that render this identity. The Risk section's "confined to installs that have a memory backend wired" is accurate; it is most of them.

Review found both wiring lines deletable with the suite green. The existing
test hands `IdentitySegmentBuilder` and `ContextBuilder` the flag itself, so
it pins that the two renderers agree once told, not that anything tells them.
Removing `factory.py`'s argument reverted the user-visible half of the fix and
nothing went red.

Two tests, each asserting on what a real object renders rather than on an
argument passed to it. One builds an `AgentLoop` with and without a backend
and reads `loop.context._get_identity()`; one builds the engine through
`build_context_engine` and reads the `IdentitySegmentBuilder` it chose. Each
was watched failing with its own call site deleted, and the files restored and
checksummed afterwards.

Also drops the issue numbers from the new test docstrings, which section 1.1
forbids in source. The behavioural rationale stays; each docstring now says
what it means without the reference.

Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
@gloryfromca

Copy link
Copy Markdown
Member Author

You are right, and the mutation you ran is the part that mattered: deleting
factory.py's argument reverted the user-visible half of this fix with the suite
green. My own reverse-verification only mutated ContextBuilder._get_identity,
which is the one line of the three that the existing test could see. I took the
green on the other two as coverage it never gave.

Fixed in f8d59bf, with both tests asserting on what a real object renders rather
than on an argument handed to it, as you suggested:

  • TestConstructorWiring::test_the_loop_tells_its_estimation_renderer_a_backend_is_wired
    builds an AgentLoop with and without a stub backend and reads
    loop.context._get_identity().
  • TestTheEngineTellsItsIdentityWhoOwnsMemory in tests/test_context_engine_factory.py
    goes through build_context_engine and reads the IdentitySegmentBuilder the engine
    chose, so the turn's own path is what is asserted.

Each was watched failing with its own call site deleted, one mutation at a time, and
the source restored and checksummed byte-identical afterwards:

deleted before now
factory.py has_memory_backend=backend is not None suite green test_with_a_backend_the_turn_names_no_memory_file red
main.py has_memory_backend=backend is not None suite green test_the_loop_tells_its_estimation_renderer_a_backend_is_wired red

797 passed across the 18 files that touch these constructors, up from 756.

The blocking finding on tests/test_segments.py is in the same commit: both new
docstrings dropped their issue numbers and say what they mean without the reference.

@gloryfromca gloryfromca left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No blockers; this can merge as far as I am concerned.

The incremental commit fixes the prior source-documentation violation and adds regression coverage at both decision points: a real AgentLoop now verifies the estimation renderer, and a factory-built engine verifies the request-path identity builder. These tests would fail if either production flag handoff were removed; they do not weaken existing assertions.

Reviewed the incremental diff and rechecked the full change against repository rules, callers, history, backward compatibility, and the memory/context architecture covered in the prior round. uv run pytest across all direct changed-component callers completed with 444 passed. The previously opened blocker thread is resolved.

@gloryfromca
gloryfromca merged commit b10acb6 into main Oct 10, 2026
25 checks passed
@gloryfromca
gloryfromca deleted the fix/memory_prompt_under_a_backend branch October 10, 2026 09:41
LivXue pushed a commit that referenced this pull request Oct 12, 2026
## Summary

With a `MemoryBackend` plugin wired, memory lives in the plugin's store
and reaches the
turn as the `# Memory` section. The identity block still named two files
under agent home,
so "remember this" became an `edit_file` against
`user_memory/profile/user.md`. On a
non-interactive turn that write is refused at the ask tier, and the
agent reported the
refusal as the memory system having failed -- while the backend had
already stored the
turn and recalled it in the next session.

`identity_text` takes a `has_memory_backend` flag, threaded from the one
place each caller
already holds the backend: `build_context_engine`, and `AgentLoop` for
the
token-estimation prompt in `ContextBuilder`, which must render what the
turn renders.

With a backend the two file lines become one:

```
- Agent home: <home> -- your own skills, not a place for user artifacts.
  - Memory is not yours to write: what is relevant is recalled into the `# Memory`
    section each turn, and the host persists the conversation after it ends. You
    cannot observe that write, so never report it to the user as done or as failed.
  - Custom skills: <home>/skills/{skill-name}/SKILL.md
```

Three clauses, three jobs. Do not write it. It is being saved, so stop
looking for a place
to put it. You cannot see that write, so do not narrate it.

The last clause is the other half of the same defect, and the third item
issue 440 asks to
separate. `backend.store` is dispatched after the turn ends, so a claim
either way is one
the agent never checked -- and whatever it claims is extracted into
long-term memory next
turn as if it were an observation of the system. The issue's own repro
produced exactly
that: an episode titled `Memory Save Failure for Green Brompton Bike
'Pickle' Noted on
2026-09-16`, whose content was right and whose narrative was wrong.
Filtering that
server-side is not available from here, so the prompt is where it is
addressed.

Without a backend the rendered prompt is unchanged, byte for byte.
Whether the episodic
log earns its line at all is issue 122's question, and answering it on
this branch would
couple the two.

The approach is AmirF194's, from the closed PR #216. That branch cannot
be reopened: it
predates the history linearization, so it shares no merge base with
`main`
(`git merge-tree` refuses unrelated histories), its six files no longer
apply, and it did
not allow maintainer edits. It is reimplemented here against current
`main`, with the
author credited as co-author on the commit.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Run on this branch's head, rebased onto `main` at c3d2821, Python
3.12, `--extra dev`.

- `pytest` over the 17 test files that construct `ContextBuilder`,
`IdentitySegmentBuilder`,
`build_context_engine` or call `identity_text` (the `git grep -l` list):
756 passed, 0 failed.
  `SERPER_API_KEY` is unset for the run; with it set,

`test_agents_code_launcher.py::test_the_products_tool_face_is_the_forks_config_intent_minus_the_ledger`
fails on an extra `web_search` in the tool face, on this branch and on
the base alike.
- The three new tests were each watched failing without the change, by
neutering the
behaviour in place rather than the signature, then restoring the file
and confirming it
  byte-identical by checksum:
- forcing the backend branch off reddens
`test_identity_with_a_backend_names_no_memory_file`;
  - dropping the flag from `ContextBuilder._get_identity` reddens
    `test_identity_with_a_backend_matches_the_estimation_prompt`;
- `test_identity_without_a_backend_still_names_both_memory_files` pins
the unchanged half.
- The no-backend claim is measured, not argued: the whole identity block
rendered from the
base and from this head is 3341 bytes on both sides and `diff` is empty.
- `make lint-python`, `make lint-types`, `make lint-imports`, `make
check-large-files` -- all exit 0.
- `RANGE=github/main..HEAD scripts/check_source_language.py` -- exit 0.
- `commitlint --config commitlint.config.cjs` on the commit body and on
the prospective
  squash header -- both exit 0.

Driven against a live EverOS after this description was first written.
All three
symptoms issue 440 describes reproduce on the merge base and none
reproduce here: on
`main` turn 1 calls `read_file` then `edit_file` on
`user_memory/profile/user.md` and
grows it to 897 bytes, at the ask tier that write is refused and the
agent reports the
save as failed, and EverOS extracts that false report into four durable
memory entries.
On this branch turn 1 makes no tool call, agent home is untouched, and
the same grep over
the store returns nothing. Both sides recall the fact in a fresh
session. Full method,
verbatim output and teardown in a comment on this PR.

One run per cell, not a distribution. The host still dumps `user.md`
into `# Memory` with
a backend wired, which is unchanged here and out of scope below.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

No docs change: the only user-facing text is the system prompt itself,
and no document
quotes this block.

## Risk

User-visible change is confined to installs that have a memory backend
wired. There, three
prompt lines become three different ones; nothing else in the prompt,
and no behaviour
outside prompt assembly, moves. Installs without a backend render a
byte-identical prompt,
which is the measurement above.

`has_memory_backend` defaults to `False` on all four signatures it is
added to, so every
call site not updated here renders what it rendered before. The three
updated call sites
are the only ones that hold a backend.

Rollback is a revert of the single commit. Nothing persists and no
format changes.

Checked and deliberately not changed:

- `MemorySegmentBuilder` still calls `get_memory_context()`
unconditionally, so `user.md`
content reaches `# Memory` with a backend wired. Turning that off is the
question of
whether two memory mechanisms should coexist, which is larger than this
issue and which
  four comparable systems answer by keeping both.
- The episodic log line, under the default no-backend configuration. It
belongs to issue 122.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

Fixes #440 -- items 1 and 3 of the three the issue asks to separate,
both reproduced on
the merge base and verified gone here. Item 2 landed in #544.

#122 -- untouched, deliberately.

---------

Co-authored-by: gloryfromca <23442919+gloryfromca@users.noreply.github.com>
Co-authored-by: Amir Fathi <amirfathi.me@gmail.com>
Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Agent reports a memory save failed when the backend stored it

3 participants