Skip to content

[Feature]: Attribute recalled-memory utility after verified outcomes #3214

Open

Description

@huangruiteng

Problem

LoopX can record whether an agent says recalled memory was applied, ignored, or refuted. The current reward-memory dogfood path can then map applied directly to a hit. That is useful lineage, but it does not establish that the memory improved the verified work outcome.

These signals need to remain distinct:

  • retrieval relevance;
  • application decision;
  • task or effect outcome;
  • marginal memory utility;
  • memory lifecycle and authority.

Otherwise a frequently retrieved but harmful memory can be reinforced, and a global evaluator can accidentally become an unbounded supervisor.

Proposal

Add a provider-neutral, post-outcome memory utility attribution contract under the existing reward-memory capability:

  1. Bind the exact recall/application receipt to a later verified outcome.
  2. Let an optional, default-off memory_utility_evaluator emit a typed, append-only observation.
  3. Reduce observations deterministically into a scoped, bounded utility projection.
  4. Keep semantic relevance as the retrieval anchor; use utility only as a bounded modifier.
  5. Let provider adapters optionally apply/read back a rank-prior update and return a receipt.
  6. Fail open: evaluator or provider failure must never block or change the main work result.

The evaluator is proposal-only. It must not select todos, gate work, spend another lane's quota, rewrite memory, or grant recalled content authority.

Attribution rules

  • applied + success is not sufficient evidence for helpful.
  • When several memories were used, record set-level utility unless evidence distinguishes individual contribution.
  • Explicit user correction, deterministic effect evidence, or controlled replay is stronger than model inference.
  • Weak inference may be retained as an observation without changing a strong ranking prior.
  • Negative utility attenuates or quarantines retrieval; it does not delete or rewrite memory.
  • Retried evaluation must be idempotent.
  • Scope must include agent/project/corpus/surface boundaries, plus the retrieval or policy snapshot used at execution time.

OpenViking fit

OpenViking already provides useful seams:

  • context/search results carry stable entry URIs, scores, and a response digest;
  • Agent Evolution links explicitly read Experiences to trajectories and aggregates terminal outcomes;
  • the training domain exposes rollout evaluation and policy_snapshot_id;
  • the current training RFC models experience memory as a policy set.

However, current Agent Evolution lineage shows consumption plus a shared final outcome, not per-memory causal credit. The current hotness score is access frequency/recency, not task utility, and there is no first-class external utility/Q-value update API.

The smallest safe integration is therefore a LoopX sidecar projection that consumes OpenViking identity/lineage read-only. A later OpenViking adapter may apply a bounded rank prior only after an explicit provider contract exists. Do not encode utility in memory prose or overload active_count/hotness.

Smallest useful slice

  • Define memory_utility_observation_v0 over existing recall/application receipts and verified outcome refs.
  • Repair Stage 5 semantics so applied is not automatically a utility hit.
  • Add an idempotent deterministic reducer and read-only utility projection.
  • Add fixtures for harmful-applied memory, ambiguous multi-memory use, stale scope, user correction, duplicate delivery, and evaluator failure.
  • Keep OpenViking integration read-only in v0; document future rank-prior/update receipt requirements.

Acceptance criteria

  • Recall relevance, application, outcome, and utility are separate typed fields.
  • No per-memory credit is assigned from a trajectory-level reward when attribution is ambiguous.
  • Utility updates are scoped, bounded, evidence-tiered, and idempotent.
  • Semantic relevance remains an anchor; utility cannot make unrelated memory rank globally.
  • The main lane is unchanged when evaluation is disabled, unavailable, or malformed.
  • Public packets contain opaque refs and compact evidence only, never raw memory or transcripts.
  • Recalled memory remains observational and cannot grant authority.
  • OpenViking policy/retrieval snapshot lineage is preserved where available.
  • Provider writeback, if added later, has explicit effect and readback receipts.

Non-goals

  • An omniscient global leader agent.
  • Automatic memory rewriting or deletion.
  • Cross-project or cross-user utility transfer by default.
  • A production reinforcement-learning claim without controlled evaluation.
  • Reviewing every trajectory on every heartbeat.

Related work

This is adjacent to #2353, but not a duplicate: that issue concerns loop reflection and retention; this issue concerns post-outcome credit for already recalled memory.

LoopX references:

OpenViking and research references:

Activity

  1. added
    enhancementNew feature or request
    triageNeeds maintainer triage or routing.
    control-planeQuota, todo, scheduler, registry, state, goal, or runtime control-plane change.
    capability-extensionCapability, extension, provider, adapter, or skill contract change.
    on Aug 15, 2026
  2. huangruiteng commented on Aug 15, 2026

    @huangruiteng
    CollaboratorAuthor

    Draft RFC proposed in #3215. It keeps the evaluator proposal-only and fail-open, stores utility in a scoped LoopX sidecar first, and treats OpenViking as the source of recall identity/lineage until an explicit rank-prior effect and readback contract exists.

  3. yuefengw commented on Aug 17, 2026

    @yuefengw
    Contributor

    我想认领 GH-C93 的 Stage 1 实现。

    我的理解是,当前真实缺口在于 Stage 5 会把 application receipt 中的 applied 直接映射为 utility hit,但“Agent 使用了记忆”和“该记忆改善了已验证结果”是两件事。计划严格按已合并 RFC #3215 第 9 节的 Stage 1 边界推进:

    • 在现有 reward_memory capability 下新增并验证 provider-neutral、append-only 的 memory_utility_observation_v0;
    • 通过现有 application receipt 的稳定标识、memory digests 与 verified outcome_ref 建立绑定,不修改 reward_memory_application_receipt_v0 的职责;
    • 没有足够独立归因证据时,包括 applied + success,utility 只记录为 unknown;多条 memory 的归因无法区分时保持 set-level,不把轨迹级结果复制成逐条 credit;
    • evaluator 保持 default-off、proposal-only、fail-open,缺席或输出畸形时不改变主任务结果,也不获得 memory 写入、todo、quota 或执行权限;
    • 用 focused positive/negative tests 覆盖 harmful-applied、ambiguous multi-memory、scope/snapshot mismatch、duplicate/invalid observation、fail-open,以及 no-authority/no-write 边界。

    为避免过度设计,首个 PR 不做 Stage 2 reducer/projection、检索排序、OpenViking 集成或写回、provider rank-prior、quota/scheduler、memory lifecycle,也不新增独立 capability 或全局 supervisor。

    兼容性方面,我会先用 characterization tests 固定现有 dogfood consumers;不会在同一个 reward_memory_dogfood_receipt_v0 schema 下静默改变 hit/miss/refute 的既有含义。若 Stage 5 语义修复需要改变输出契约,会采用明确的版本化或兼容路径,并在 PR 中披露默认行为影响。

    Issue 原始 “smallest useful slice” 包含 reducer,而后续合并的 RFC #3215 §9 将 reducer 放在 Stage 2。若上述 Stage 1 边界符合维护者预期,我就按此范围从最新 main 开始实现一个完整但窄的 PR;如果希望首个 PR 同时包含 reducer/projection,请告知,我会先调整范围再动代码。

  4. huangruiteng commented on Aug 18, 2026

    @huangruiteng
    CollaboratorAuthor

    可以,按你列出的 Stage 1 边界推进,这与已合并 RFC #3215 §9 一致。

    首个 PR 请只包含:

    • memory_utility_observation_v0 的 provider-neutral、append-only 合同与校验;
    • 现有 Stage 3 application receipt 到 verified outcome_ref 的稳定绑定;
    • 修正 Stage 5:applied 不能单独证明正向 utility,证据不足时保持 unknown;
    • main lane 在 evaluator disabled / unavailable / malformed 时行为不变;
    • focused 正反例覆盖你列出的 harmful-applied、ambiguous set-level、scope/snapshot mismatch、duplicate/invalid observation、fail-open 和 no-authority/no-write 边界。

    首个 PR 不要带 Stage 2 reducer/projection、检索或排序变化、OpenViking 写回、provider rank-prior、quota/scheduler、memory lifecycle,也不要新增独立 capability 或 supervisor。若现有 dogfood receipt 的语义必须变化,请走明确版本化或兼容读取路径,并在 PR 中说明 old/new default,不能在同一 schema 下静默改义。

    这条回复确认由 @yuefengw 认领 #3214 的 Stage 1;请从最新 main 开独立 PR 并关联本 Issue。后续阶段等 Stage 1 的 exact-head 评审和验证完成后再单独决定。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    capability-extensionCapability, extension, provider, adapter, or skill contract change.control-planeQuota, todo, scheduler, registry, state, goal, or runtime control-plane change.direction/architecture-evolutionArchitecture evolution and research-incubator work.enhancementNew feature or requesttriageNeeds maintainer triage or routing.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions