Repository navigation
Keep the fully-read marker monotonic - #565
Conversation
e9458f6 to
4106440
Compare
33a2a1d to
0c2598d
Compare
4f636f3 to
34168ae
Compare
9d17b47 to
54fccbb
Compare
a804eba to
55a9c2c
Compare
A backwards m.fully_read write was accepted verbatim, so any stale device or client could regress the marker another one already advanced, leaving the unread line stuck or jumping backwards between syncs. Resolve both the stored and the incoming marker event to their timeline positions and skip the write unless it advances, matching the existing monotonic gates for m.read and m.read.private. An unresolvable position on either side accepts the write, as for public receipts. Backwards writes are silently accepted-but-ignored to keep the endpoints idempotent. Applies to both /read_markers and /receipt; storage errors in /read_markers now propagate instead of being swallowed. Adds an HTTP-level integration test (fully_read_monotonic) covering forward writes, backwards writes via /read_markers, and backwards writes via /receipt.
repro-fully-read-regression.sh verifies the monotonic fully-read guard against a running server (forward write stores, backwards writes via /read_markers and /receipt are ignored). probe-read-state.sh dumps the server-side signals a client's unread state derives from (notification counts, own receipts, m.fully_read, marked-unread) for a given account and room. Both default to a test server and test accounts; they are diagnostic tooling, not general-purpose tests.
A receipt naming the currently stored event was rejected without allocating a stream position, producing no EDU and no sync traffic at all. A client which missed the original receipt (e.g. read on another device while the room was outside its sliding-sync window) could never repair its derived unread state in a quiet room: re-reading the same position stayed silent, and no new events arrived to force a recomputation. Element X / SchildiChat Next showed such rooms as unread (grey client-side count) forever while other clients correctly showed them read. Identical receipts now rewrite the row under a fresh stream position so the receipt EDU is re-emitted to sync (v3 ephemeral and the MSC4186 receipts extension), but still report no advance: notification counts are not reset and the last-notification-read stamp is untouched, so legitimate unread accumulated after the receipted event is preserved. Strictly older receipts remain rejected (monotonicity, matrix-construct#516). The private read marker path re-announces identically through its sync gate; unannounced identical markers (own-send path) stay a no-op.
f042b66 to
d5d122f
Compare
|
Thanks for sticking with this one, and sorry it sat for so long without us explaining what was holding it up. We merged the PR as is, so the credit stays with you, and followed up on top of it. The fully-read gate is the part we kept: a stale device can no longer pull the marker back, which matches Synapse and the default MSC4446 proposes. Re-announcing identical receipts: we reverted this one. Synapse drops a receipt that matches the stored one, MSC4446's default does the same, and our dedupe is what stopped the bridge receipt loop in #516. Re-announcing would turn every duplicate, including ones arriving over federation, into a write and a sync wakeup for every local member of the room. The stuck rooms you saw look client-side to us: matrix-org/matrix-rust-sdk#7048 fixed unread counts not refreshing after cached chunks load, and it landed after your commit. If a room still sticks on a current SchildiChat build, we'd like to hear about it. The live-server scripts: removed, since they target one deployment and nothing in CI runs them. The integration test now runs on the shared client harness and also covers a forward write over an existing marker. Moving the marker back on purpose: MSC4446 lets a client ask for that with One caveat on scope: the gate holds for writes that arrive one after another. Two writes for the same room landing at the same instant aren't ordered against each other, the same as the rest of account data today. |
|
Thanks for the detailed explanation! I'll go ahead and update my homeserver to current |
What does this PR do?
Unlike m.read and m.read.private receipts, which already have monotonic gates, the m.fully_read marker accepted any position, including backwards. In a multi-device setup, any client acting on a stale view (e.g. reposting its last-known position on startup, or a "mark as read" action on an old notification) silently regressed the marker that another device had already advanced. Users saw the unread line stuck in place or jumping backwards between syncs, and no amount of re-reading could be trusted to fix it, since whichever device was stalest would pull it back again. There is also a Matrix Rust SDK quirk that can cause older private read receipts to take precedence over fully-read if they're on events the client doesn't have locally, like when using an existing account on a new client.
Fix: before storing, resolve both the stored and the incoming marker event to their timeline positions and skip the write unless it advances, the same semantics as the existing receipt gates (position_advances): an unresolvable position on either side accepts the write. Backwards writes are accepted-but-ignored (200 OK) so the endpoints stay idempotent. Applies to both POST /read_markers and POST /receipt (m.fully_read). As a side effect, storage errors in /read_markers now propagate instead of being silently swallowed.
The spec doesn't say anything about whether fully-read can or should be allowed to go backward, but there isn't an obvious use case for it, and I'm not sure it can really be implemented coherently since I think it should be order-independent, which it can't be if it's not monotonic.
Testing: new HTTP-level integration test (fully_read_monotonic) — forward write stores; backwards writes via /read_markers (all three fields, as clients send them) and via /receipt leave the marker untouched. Verified to fail without the guard. Also verified live against a production server with a deterministic repro script (included in a separate commit, drop if unwanted).
I currently have this running on matrix.agiadn.org along with #564, and along with some FluffyChat fixes that were preventing it from updating the read marker in some cases, FluffyChat now marks rooms as read when I read them in all cases.
Found and fixed another case (possibly a regression?) for sliding sync clients: the monotonicity check was causing duplicate receipts to be rejected entirely with no new EDU. But the only case where we should be seeing these is if a client thinks it's reading unread messages. So we now send a new EDU in these cases so that those clients (Shildichat Next is the one I've been testing with) mark the room as read.
Checklist
cargo fmtand satisfies clippy andrustc lints; any allowed lint is justified by an obvious reason or a
comment.
are noted in the description above.
src/core/config/mod.rsdoccomments and the regenerated
tuwunel-example.tomlis committed.docs/.and my conduct is in line with the Contributor's Covenant and
Tuwunel's Code of Conduct.