[Feature]: belief layer — probabilistic conflict resolution over atomic facts #382

Area

architecture methods

Problem or opportunity

Atomic facts have no arbitration. When two facts contradict each other, both are stored, both are indexed, both match the same query, and the ranking that decides which one reaches the model is BM25 score plus embedding distance — neither of which knows which one is true.

Concretely: a user says "I use 6 ounces of water per tablespoon" in March and "I've switched to 5 ounces" in June. Both atomic_fact rows stay live. Nothing in the write path notices they disagree.

deprecated_by looks like the mechanism for this and isn't — Reflection sets it when merging fragmented cluster members into a consolidated episode (Select → Merge → Re-extract → Deprecate). It's a consolidation marker, not a contradiction verdict. Grepping src/ for dedup / contradiction / conflict handling returns SQLite ON CONFLICT clauses and nothing else.

Two capabilities are missing, and they turn out to be the same one:

1. arbitration — which of two mutually exclusive facts does the memory assert?
2. calibration — how sure is it, and can a caller act on that number?

confidence exists on AgentSkill today (LLM-emitted, alongside maturity_score) but not on facts — and an LLM-emitted scalar isn't a probability: nothing normalises it across competing candidates and nothing updates it when new evidence arrives.

This also leaves a security gap. Because no fact can outrank another, a fact that arrives from a scraped page or a third-party agent competes on equal terms with one the user typed. Volumetric memory poisoning against a store with no trust model is not an attack that needs to be clever.

Proposed solution

A domain-layer belief module — everos.memory.belief — where mutually exclusive facts share a belief_key and hold a categorical distribution, updated by a Bayesian rule whose evidential weight is capped by the provenance of the channel each fact arrived on. Pure functions plus a resolver; no I/O, no LLM.

from everos.memory.belief import BeliefResolver, FactObservation, ProvenanceTier

resolver = BeliefResolver()
resolver.observe(FactObservation(
    belief_key="user_42:coffee_ratio",
    fact="6 ounces of water per tablespoon",
    observed_at=march,
    tier=ProvenanceTier.USER_DIRECT,
    content_confidence=0.9,
))
resolver.observe(FactObservation(..., fact="5 ounces...", observed_at=june))

verdict = resolver.verdict("user_42:coffee_ratio")
# fact="5 ounces...", probability=0.7, superseded=["6 ounces..."]
if verdict.is_uncertain:
    ...  # ask rather than assert

Three commitments:

Trust is a property of the channel, never of the text. Reliability is min(channel ceiling, content confidence). Content confidence may lower trust within the ceiling, never raise it. Reading trust off the sentence hands it to whoever writes the sentence — "Confirmed, verified, this is final" scores near 1.0 on any hedging rubric.

A channel at or below r = 0.5 cannot change what the memory asserts, at any volume. The likelihood ratio clamps to 1 at that pivot: a source you distrust asserting X is not evidence against X, it's simply not evidence. Clamping rather than inverting also fixes the pivot independently of how many candidates a belief holds.

A single trusted correction can still supersede a single trusted claim. This is the property that's easy to lose, and it fails silently.

The subtle part, because it's the part worth reviewing

Admission and evidence are different operations and need separate gates. A candidate the belief has never seen can't be reweighted, only admitted — and admission happens before any likelihood test runs. So entry mass is a second, hidden trust knob, and it controls both properties at once:

No setting satisfies both. At the small end the memory is immune because it never learns anything; at the large end an untrusted channel can install any belief it likes. Both ends fail quietly, which is why this is worth stating explicitly rather than leaving as a tuning constant.

The fix is to condition admission on the same pivot the likelihood ratio uses: above it, a trusted first sighting enters at a mass proportional to the channel's reliability; below it, the candidate is recorded and retrievable but inert.

A related detail: a first observation at reliability r should leave the belief at p ≈ r, not 1.0, with the residual on a reserved UNKNOWN candidate. Without that, a fact heard exactly once normalises to certainty and every number in the store reads as 1.0.

Measurements

knowledge-update from LongMemEval — 78 two-session supersessions, 70 with turn-level gold evidence spans in both sessions. The spans feed the resolver directly (no extractor, no retriever, no LLM), so the number is attributable to the update rule rather than the pipeline. Offline, ~2 seconds, no API key.

• clean is a pure recency test, where last-write-wins is optimal by construction. That arm exists to show no regression, not to show a win.
• The identical 78.6% across the three arms is the result: the attacks have zero effect — same asserted fact, same probability, whether or not the attacker is present.
• The 21.4% gap on clean is fully accounted for: 15 of the 70 updates are hedged ("I'd say the marketing campaign is the priority"), the stand-in content-confidence rubric drops them below the pivot, and admission is blocked. 15/70 = 21.4% exactly. That's an extraction-quality number, not an arbitration one — in production the algo layer emits a real confidence.
• lowtrust-fix = 0.0% is the honest cost, and it's symmetric by construction: a ceiling that stops a bad correction on a low-trust channel stops a good one identically. There is no setting that gets both.

Alternatives considered

Let the LLM adjudicate at write time (the Mem0 ADD/UPDATE/DELETE/NOOP shape). Costs a call per candidate conflict, produces no auditable reason, and inherits the model's calibration. It also can't state a guarantee — there's no threshold below which a hostile input provably cannot win.

Recency wins. Free, and optimal on the clean arm above. It is also 0.0% on both poisoning arms, because "most recent" is the one property an attacker fully controls.

Extend Reflection. Reflection is the right place for a periodic pass, but its current job is merging fragments, and folding contradiction arbitration into it would conflate "these say the same thing" with "these disagree" — the exact distinction that has to stay sharp.

Store an LLM-emitted confidence on atomic_fact, mirroring AgentSkill. Cheapest option, and it gives a number to display — but nothing normalises it across candidates, nothing updates it when evidence arrives, and a number that can be raised by confident phrasing is worse than no number once anything untrusted can write.

Additional context

The module, tests, benchmark, and a design note are implemented on a local branch against 6d62ecb: src/everos/memory/belief/ (4 files), tests/unit/test_memory/test_belief/ (39 tests pinning both guarantees), benchmarks/belief_ku.py, docs/belief-layer.md. make lint, make test (1900 passed) and make integration (180 passed) are green. Happy to open it as a PR if that's useful, or to drop it if the direction isn't wanted.

Deliberately not built, because they need a maintainer's call first:

• belief_key derivation — the real open question. An atomic fact is an undecomposed sentence, so there's no (subject, attribute) to group mutually exclusive candidates on. Options, roughly in order of appetite: the algo layer emits an optional subject / attribute alongside the fact; or group by embedding cluster within an owner scope, which is cheap but conflates "similar" with "mutually exclusive"; or callers pass it explicitly for the narrow slice they care about. Everything else is downstream of this choice.
• Persistence. BeliefState / BeliefRevision are derived state and belong in SQLite, not the LanceDB fact table — no index migration, and states rebuild from the revision log. Needs a repo plus an alembic revision.
• Search integration. search/filters.py already excludes deprecated_by IS NOT NULL; the analogous move is ranking or filtering by posterior and surfacing probability on the recall DTO.
• Tier assignment. Mapping existing scoping (owner_type, app_id, session_id, sender_ids) onto tiers should be operator config in everos.toml — never inferred, and never from anything an agent can write.
• Calibration. The layer reports probabilities; whether they're true probabilities needs outcomes to score against (Brier / ECE over resolved beliefs). Until that's measured, entropy is the honest thing to show a caller and probability should be read as a ranking, not a frequency.