Skip to content

Keep the fully-read marker monotonic - #565

Merged
jevolk merged 3 commits into
matrix-construct:devfrom
ruka-hamanasu:fully-read-monotonic
Oct 17, 2026
Merged

jevolk merged 3 commits into
matrix-construct:devfrom
ruka-hamanasu:fully-read-monotonic

Conversation

@ruka-hamanasu

@ruka-hamanasu ruka-hamanasu commented Aug 28, 2026 •

Copy link
Copy Markdown
Contributor

What does this PR do?

Unlike m.read and m.read.private receipts, which already have monotonic gates, the m.fully_read marker accepted any position, including backwards. In a multi-device setup, any client acting on a stale view (e.g. reposting its last-known position on startup, or a "mark as read" action on an old notification) silently regressed the marker that another device had already advanced. Users saw the unread line stuck in place or jumping backwards between syncs, and no amount of re-reading could be trusted to fix it, since whichever device was stalest would pull it back again. There is also a Matrix Rust SDK quirk that can cause older private read receipts to take precedence over fully-read if they're on events the client doesn't have locally, like when using an existing account on a new client.

Fix: before storing, resolve both the stored and the incoming marker event to their timeline positions and skip the write unless it advances, the same semantics as the existing receipt gates (position_advances): an unresolvable position on either side accepts the write. Backwards writes are accepted-but-ignored (200 OK) so the endpoints stay idempotent. Applies to both POST /read_markers and POST /receipt (m.fully_read). As a side effect, storage errors in /read_markers now propagate instead of being silently swallowed.

The spec doesn't say anything about whether fully-read can or should be allowed to go backward, but there isn't an obvious use case for it, and I'm not sure it can really be implemented coherently since I think it should be order-independent, which it can't be if it's not monotonic.

Testing: new HTTP-level integration test (fully_read_monotonic) — forward write stores; backwards writes via /read_markers (all three fields, as clients send them) and via /receipt leave the marker untouched. Verified to fail without the guard. Also verified live against a production server with a deterministic repro script (included in a separate commit, drop if unwanted).

I currently have this running on matrix.agiadn.org along with #564, and along with some FluffyChat fixes that were preventing it from updating the read marker in some cases, FluffyChat now marks rooms as read when I read them in all cases.

Found and fixed another case (possibly a regression?) for sliding sync clients: the monotonicity check was causing duplicate receipts to be rejected entirely with no new EDU. But the only case where we should be seeing these is if a client thinks it's reading unread messages. So we now send a new EDU in these cases so that those clients (Shildichat Next is the one I've been testing with) mark the room as read.

Checklist

  • Code is formatted with nightly cargo fmt and satisfies clippy and
    rustc lints; any allowed lint is justified by an obvious reason or a
    comment.
  • Complement compliance changes (new passes or new failures), if any,
    are noted in the description above.
  • [n/a] Config option changes were made in src/core/config/mod.rs doc
    comments and the regenerated tuwunel-example.toml is committed.
  • [n/a] User-facing changes are reflected in docs/.
  • I agree that my changes may be licensed under the Apache-2.0 licence
    and my conduct is in line with the Contributor's Covenant and
    Tuwunel's Code of Conduct.

@ruka-hamanasu
ruka-hamanasu changed the base branch from main to dev August 28, 2026 21:12
@jevolk
jevolk force-pushed the dev branch 5 times, most recently from e9458f6 to 4106440 Compare September 7, 2026 14:05
@jevolk
jevolk force-pushed the dev branch 11 times, most recently from 33a2a1d to 0c2598d Compare September 15, 2026 05:07
@jevolk
jevolk force-pushed the dev branch 5 times, most recently from 4f636f3 to 34168ae Compare September 19, 2026 15:54
@jevolk
jevolk force-pushed the dev branch 6 times, most recently from 9d17b47 to 54fccbb Compare October 5, 2026 02:17
@jevolk
jevolk force-pushed the dev branch 12 times, most recently from a804eba to 55a9c2c Compare October 10, 2026 15:30
A backwards m.fully_read write was accepted verbatim, so any stale
device or client could regress the marker another one already
advanced, leaving the unread line stuck or jumping backwards between
syncs.

Resolve both the stored and the incoming marker event to their
timeline positions and skip the write unless it advances, matching
the existing monotonic gates for m.read and m.read.private. An
unresolvable position on either side accepts the write, as for public
receipts. Backwards writes are silently accepted-but-ignored to keep
the endpoints idempotent. Applies to both /read_markers and /receipt;
storage errors in /read_markers now propagate instead of being
swallowed.

Adds an HTTP-level integration test (fully_read_monotonic) covering
forward writes, backwards writes via /read_markers, and backwards
writes via /receipt.
repro-fully-read-regression.sh verifies the monotonic fully-read
guard against a running server (forward write stores, backwards
writes via /read_markers and /receipt are ignored). probe-read-state.sh
dumps the server-side signals a client's unread state derives from
(notification counts, own receipts, m.fully_read, marked-unread) for
a given account and room.

Both default to a test server and test accounts; they are diagnostic
tooling, not general-purpose tests.
A receipt naming the currently stored event was rejected without
allocating a stream position, producing no EDU and no sync traffic at
all. A client which missed the original receipt (e.g. read on another
device while the room was outside its sliding-sync window) could never
repair its derived unread state in a quiet room: re-reading the same
position stayed silent, and no new events arrived to force a
recomputation. Element X / SchildiChat Next showed such rooms as
unread (grey client-side count) forever while other clients correctly
showed them read.

Identical receipts now rewrite the row under a fresh stream position
so the receipt EDU is re-emitted to sync (v3 ephemeral and the MSC4186
receipts extension), but still report no advance: notification counts
are not reset and the last-notification-read stamp is untouched, so
legitimate unread accumulated after the receipted event is preserved.
Strictly older receipts remain rejected (monotonicity, matrix-construct#516). The
private read marker path re-announces identically through its sync
gate; unannounced identical markers (own-send path) stay a no-op.
@jevolk
jevolk merged commit 779b4a5 into matrix-construct:dev Oct 17, 2026
57 checks passed
@jevolk

jevolk commented Oct 17, 2026

Copy link
Copy Markdown
Member

Thanks for sticking with this one, and sorry it sat for so long without us explaining what was holding it up.

We merged the PR as is, so the credit stays with you, and followed up on top of it. The fully-read gate is the part we kept: a stale device can no longer pull the marker back, which matches Synapse and the default MSC4446 proposes.

Re-announcing identical receipts: we reverted this one. Synapse drops a receipt that matches the stored one, MSC4446's default does the same, and our dedupe is what stopped the bridge receipt loop in #516. Re-announcing would turn every duplicate, including ones arriving over federation, into a write and a sync wakeup for every local member of the room. The stuck rooms you saw look client-side to us: matrix-org/matrix-rust-sdk#7048 fixed unread counts not refreshing after cached chunks load, and it landed after your commit. If a room still sticks on a current SchildiChat build, we'd like to hear about it.

The live-server scripts: removed, since they target one deployment and nothing in CI runs them. The integration test now runs on the shared client harness and also covers a forward write over an existing marker.

Moving the marker back on purpose: MSC4446 lets a client ask for that with com.beeper.allow_backward, and SchildiChat's SDK already implements the sending side, so the gate alone would have dropped those requests. Both endpoints now honour the flag for m.fully_read, /receipt rejects it for other receipt types, and /versions advertises com.beeper.msc4446.

One caveat on scope: the gate holds for writes that arrive one after another. Two writes for the same room landing at the same instant aren't ordered against each other, the same as the rest of account data today.

@ruka-hamanasu

Copy link
Copy Markdown
Contributor Author

Thanks for the detailed explanation! I'll go ahead and update my homeserver to current dev and see how things look. I'll let you know if I still see issues with read receipts between Cinny/Pinniped and Schildichat Next or any other client I happen to try.

@jevolk jevolk added bug Something isn't right. client compat Problem with client-server interaction. Issue is preventing a client from working. labels Oct 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't right. client compat Problem with client-server interaction. Issue is preventing a client from working.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants