Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 7 additions & 5 deletions emrg/server/daemon.py
Original file line number Diff line number Diff line change
Expand Up @@ -1462,11 +1462,13 @@ def _cap_memory_index(self, path) -> str:
"""Cap a MEMORY.md embedded into the system prompt (defense in depth).

Rant 2026-08-23T11:00:31: #941's write-time guards only fire on
memory_store API writes, but agents append MEMORY.md rows directly
(evolution_prompt §6) and bypass them — the index once reached
787KB/2931 lines = 77% of a 452,972-char prompt (~250K all-miss
tokens per request). Cap what gets embedded; the full index and
cycle-archive-*.md stay readable on disk via the read tool.
memory_store API writes, but MEMORY.md rows also get appended
directly, bypassing them — the per-cycle mandate that used to
drive that was removed (rant 2026-09-14T20:14:56), the bypass was
not. The index once reached 787KB/2931 lines = 77% of a
452,972-char prompt (~250K all-miss tokens per request). Cap what
gets embedded; the full index and cycle-archive-*.md stay readable
on disk via the read tool.
"""
# One knob, two readings. This used to spell the cap itself with a
# comment promising it matched `memory.INDEX_SIZE_WARN` — a promise nothing
Expand Down
6 changes: 3 additions & 3 deletions emrg/server/evolution_prompt.md
Original file line number Diff line number Diff line change
Expand Up @@ -383,12 +383,12 @@ Create a **cycle memory entry** (rant 2026-08-12T18:03:26 — no more standalone
- `event_at` / `created_at` / `updated_at`: ISO timestamps
- `type: task`, `scope: project`, `status: active` (cycle in progress) or `completed` (final)
- Body: findings, changes, verification results, expected effects (same content as before, just a memory file)
- Update the `MEMORY.md` index in the same directory (add one row, id linked to the filename) — this is the **single index** for cycle records
- ⚡ **Index hygiene protocol** (rants 2026-08-23T08:04:26 + 11:00:31 — the daemon embeds MEMORY.md into the system prompt raw, and evolution's direct file writes bypass memory_store's guards; an unbounded index once reached 787KB/2931 lines = 77% of a 452,972-char prompt, ~250K all-miss tokens per request). Apply to **every MEMORY.md you maintain** (evolution-level, source-project-level, session-level). Rules:
- The `MEMORY.md` index in that directory is **not** a per-cycle obligation — it is kept on demand, by the session, when there is something worth indexing (rant 2026-09-14T20:14:56). The durable record is the `cycle-<ts>.md` detail file above
- ⚡ **Index hygiene protocol — whenever you write to one of these indexes** (rants 2026-08-23T08:04:26 + 11:00:31 — the daemon embeds MEMORY.md into the system prompt raw, and evolution's direct file writes bypass memory_store's guards; an unbounded index once reached 787KB/2931 lines = 77% of a 452,972-char prompt, ~250K all-miss tokens per request). These rules bind **the write**, not the cycle — apply to **every MEMORY.md you maintain** (evolution-level, source-project-level, session-level). Rules:
- **Title-only rows**: each index row is a **one-line summary** (id linked to the filename). **Never embed a cycle's summary/NTE text into the index row** — that text lives in the `cycle-<ts>.md` detail file only.
- **Hard cap: keep at most the 50 most recent cycle rows** in each MEMORY.md. Before adding a row that would exceed 50: append the oldest cycle rows to `cycle-archive-YYYYMMDD.md` **in the same directory** (create-if-missing, append-only, never rewrite or dedupe), then remove those rows from MEMORY.md. **Detail files (`cycle-*.md`) are never deleted** — only index rows move.
- **Archive files are excluded from the system prompt** (the daemon embeds only `MEMORY.md`): never reference `cycle-archive-*.md` in MEMORY.md rows, never re-add archived rows to the index, never paste archive content into MEMORY.md or the prompt. Archived rows stay readable via the `read` tool.
- **Row-cap check**: if a MEMORY.md already exceeds 50 cycle rows (e.g. after a missed cleanup), archive down to the latest 50 this cycle before adding the new row.
- **Row-cap check**: if a MEMORY.md already exceeds 50 cycle rows (e.g. a missed cleanup), archive down to the latest 50 before adding a row.
- Keep the file format identical to other memory entries (frontmatter + Markdown body)

> Transition note: legacy `evolution-cycle-*.md` files remain in place (readable, never deleted); only new records use the memory entry path.
Expand Down
10 changes: 6 additions & 4 deletions scripts/archive-memory-index.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,12 @@

The class this exists for
-------------------------
Every evolution cycle curates two memory indexes (the evolution project's and
the session's), and the protocol is mechanical: keep at most 50 cycle rows,
append the rows that fall off to `cycle-archive-<YYYYMMDD>.md`, never delete a
detail file, never reference the archive from the index. It has been done by
An evolution cycle used to curate two memory indexes (the evolution project's
and the session's); that per-cycle mandate was removed (rant 2026-09-14T20:14:56),
so an index is now maintained on demand. When it is maintained, the protocol is
mechanical: keep at most 50 cycle rows, append the rows that fall off to
`cycle-archive-<YYYYMMDD>.md`, never delete a detail file, never reference the
archive from the index. It has been done by
hand-written scripts, and the hand-written scripts have failed twice:

* an empty shell variable turned ``sed -i '' "${n}d"`` into the sed script ``d``,
Expand Down
Loading