[4.41/C2a] Core mentions & message model: tri-state is_mention, mention regex, Author.email/is_system, Message.reply_to

[patrick-chinchill]

Summary

Port four core changes: tri-state is_mention (adapter False is definitive; only None falls back to text detection); a tighter mention regex (emails, URL userinfo, @bot-dev stop matching @bot); Author.email/Author.is_system; Message.reply_to surviving serialization, rehydration, history and SentMessage. Discord's switch to True | None must land in this PR, or its literal @botname fallback stops routing.

Upstream changes

• 2531a422 fix username regexp (#621) — chat@4.34.0 — detectMention uses (?![\w-]) instead of \b. Telegram half → [4.41/TG1] Telegram inbound: stickers/animations/locations/polls, media identity, mention regex, allowlist, early typing #225.
• 0701679e fix: Enhance mention regex and sleep method with abort support (#706) — chat@4.35.0 — Telegram-only, no core change: regex cache → [4.41/TG1] Telegram inbound: stickers/animations/locations/polls, media identity, mention regex, allowlist, early typing #225; abortable sleep N/A (stop_polling cancels).
• 46681f50 fix(teams): hydrate incoming author email (#711) — chat@4.35.0 — core half: Author.email? (omitted when undefined). Hydration → [4.41/T3] Teams inbound: author email hydration, protected inline attachments, secure downloads #218.
• 80def3ab feat: add author.isSystem to distinguish platform-generated messages (#707) — chat@4.35.0 — core half: Author.isSystem? (absent = false, omitted when undefined). Slack → [4.41/SL4] Slack inbound normalization: self-mention decode, content is_mention, mrkdwn fixes, bot author ids, socket retries #209.
• b547f458 fix(chat): stop treating email addresses as bot mentions (#761) — chat@4.36.0 — username/user-id patterns become `(?<!\\w)@${name}(?![\\w-])`, flag i.
• 0f24cc30 feat(chat): preserve replied-to message context (#802) — chat@4.38.0 — Message.replyTo; toJSON/fromJSON/setMessageAdapter/rehydrateMessage recurse; history nulls raw along the chain; createSentMessageFromMessage carries it. Telegram population → [4.41/TG4] Telegram replies: replied-to context, reply-to-bot as mention, native replies, portable file data #228.
• fcdc1c9e fix(chat): honor adapter-reported non-mentions (#946) — chat@4.41.0 — isMention ?? detectMention(...) for dispatched and skipped messages, hasMention via === true; DM override stays; Discord reports isMentioned || undefined. Linear → [4.41/L1] Linear: stable agent-session thread ids, undetermined mentions for ordinary comments #232; Notion/X not ported.

Current Python behavior

• src/chat_sdk/chat.py:2316: message.is_mention = message.is_mention or self._detect_mention(...) — an adapter False is overridden (lives in _set_mention_flags after [4.41/C1a] Core concurrency: lock heartbeat, max_lock_lifetime_ms, per-thread drain isolation, debounce drain #190). types.py:519 is_mention: bool | None = None is already tri-state; to_json emits isMention only when not None (types.py:649-650).
• chat.py:2455/:2460: rf"@{re.escape(name)}\b", compiled with re.IGNORECASE (:2446) — jane@acme.com mentions a bot named acme, @bot-dev mentions @bot.
• Adapters: Discord adapters/discord/adapter.py:705-711 computes a bool, :768 passes it → definitive False when unmentioned. Teams adapter sets only True (teams/adapter.py:773-780), as upstream; the webhook primitive teams/webhook/parse.py:72 fills TeamsMessagePayload.is_mention: bool (upstream isMention: boolean), not fed into Message. Telegram telegram/adapter.py:2241 passes a bool, as upstream.
• types.py:240-248 Author has only full_name, is_bot, is_me, user_id, user_name; grep -rn 'is_system\|isSystem' src/chat_sdk is empty.
• No reply_to/replyTo in types.py, chat.py, thread.py, thread_history.py. Serialization sites: types.py:598 to_json, :667 from_json, :757 from_json_compat; chat.py:2475 _rehydrate_message, :2638 _message_from_json.
• History caches null only top-level raw: chat.py:2688-2704 (_ThreadHistoryCache, used by Chat, :380) and thread_history.py:71-73. thread.py:1232 create_sent_message_from_message and types.py:862 SentMessage lack reply_to.

Scope

• chat.py _set_mention_flags: if m.is_mention is None: m.is_mention = self._detect_mention(adapter, m) for message and each skipped; has_mention via is True. Keep the DM override (chat.py:2362-2363), commented as a routing rule.
• adapters/discord/adapter.py:768: is_mention=True if is_mentioned else None; update Discord tests that assert False.
• Teams: no code change (Verify first); regression test — channel message without a mention entity but with @<bot user_name> in text routes to on_mention.
• _detect_mention: username/user-id patterns become (?<![A-Za-z0-9_])@{re.escape(name)}(?![A-Za-z0-9_-]) with re.IGNORECASE; keep the Discord <@!?id> pattern.
• Author: email: str | None = None, is_system: bool | None = None (after required fields; optional keys on SerializedMessageAuthor). to_json emits email/isSystem only when not None; from_json, from_json_compat, chat._message_from_json and the _rehydrate_message dict fallback read them (camelCase, then snake_case).
• types.py: reply_to: Message | None = None as the last Message field (after user_key), also on MessageData/SentMessage/SerializedMessage. to_json emits replyTo only when set; from_json/from_json_compat recurse and accept an already-revived Message.
• set_message_adapter and chat._rehydrate_message (incl. rehydrate_attachment; dict fallback reads replyTo/reply_to) recurse into reply_to; both history caches null raw along the replyTo chain; create_sent_message_from_message passes reply_to.

Out of scope

• Slack content is_mention + USLACK is_system → [4.41/SL4] Slack inbound normalization: self-mention decode, content is_mention, mrkdwn fixes, bot author ids, socket retries #209; Teams email hydration → [4.41/T3] Teams inbound: author email hydration, protected inline attachments, secure downloads #218; Telegram reply_to parsing → [4.41/TG4] Telegram replies: replied-to context, reply-to-bot as mention, native replies, portable file data #228, hyphen regex + cache → [4.41/TG1] Telegram inbound: stickers/animations/locations/polls, media identity, mention regex, allowlist, early typing #225; Linear ordinary comments → [4.41/L1] Linear: stable agent-session thread ids, undetermined mentions for ordinary comments #232; Discord email mangling (490fa00e) → [4.41/D1] Discord correctness & security: thread-parent validation, starter-message routing, mentions/URLs, forwarded snapshots, downloads #229, respond_to_* → [4.41/D2] Discord features: ephemeral slash responses, select values, channel allowlist, global mentions opt-in, rename threads #230; GitHub newline-mention tests (5c926f19) → [4.41/C2b] Shared text utilities: bare-mention scanner, code-fence helpers, to_plain_text structural whitespace #193.

Porting notes

• Hazard security: Fix all critical and high findings from security audit #1: never collapse with or; None = "not determined". Serializers must not coerce None to False for is_mention/is_system (a False is_system is still emitted, as upstream).
• \w: JS \w (no u) is ASCII-only, Python's is Unicode — spell ASCII classes out. Not re.ASCII (it makes IGNORECASE ASCII-only; JS i folds non-ASCII). Test é@bot (a mention upstream).
• Field order: new fields go last so positional construction of Author/Message keeps working.
• Recursion: no depth cap, as upstream. The JSON object_hook revives bottom-up, so from_json must pass a Message through unchanged (types.py:681). camelCase only at the wire (replyTo, isSystem, email).

Tests

packages/chat/src/chat.test.ts (fidelity-mapped → tests/test_chat_faithful.py):

• "should keep a definitive non-mention reported by the adapter"; "should keep a definitive mention reported by the adapter"; "should keep a definitive non-mention on skipped queued messages"
• "should not trigger onNewMention when a bot with a hyphen-suffixed name is mentioned"; "should not trigger onNewMention when a hyphen-suffixed user ID is mentioned"
• it.each "should not treat %s as a mention (%s)" (4 rows) and "should still detect %s as a mention (%s)" (5 rows) — names per [4.41/P0] Fidelity tooling for the 4.41 wave: single pin constant, SHA pin, it.each expansion, map new core test files #185's it.each convention.
• Extend "should call rehydrateAttachment on deserialized attachments missing fetchData" (test_chat_faithful.py:2811) with upstream's replyTo assertions.

Mapped: serialization.test.ts "should round-trip replied-to message context"; thread-history.test.ts rename test_should_strip_raw_field_on_storage (tests/test_thread_history.py:160) → "should strip raw fields on storage" + replyTo.raw is None (same for chat._ThreadHistoryCache); thread.test.ts extend "should wrap a Message as a SentMessage with same fields" (test_thread_faithful.py:2776) with sent.reply_to.

packages/chat/src/message.test.ts is not mapped at 4.31 (#185 may map it): "should preserve author email through serialization"; "should preserve author.isSystem through a full JSON roundtrip"; "should leave author.isSystem absent for non-system authors".

Discord (unmapped adapter-discord/src/index.test.ts): mirror "keeps allowlisted forwarded messages in their Discord thread" (is_mention is None when unmentioned).

Python-specific: Teams text-fallback routing; é@bot; from_json_compat with snake_case reply_to/is_system.

Acceptance criteria

• Full validation command from CLAUDE.md passes.
• docs/UPSTREAM_SYNC.md records: ASCII-class regex spelling; Teams webhook-primitive is_mention: bool kept for parity; 0701679e abortable-sleep N/A.
• CHANGELOG under "Unreleased (4.41 wave)": custom adapters must return None, not False, for "not detected"; emails and @name-suffix no longer trigger on_mention; new Author.email/is_system, Message.reply_to, serialized replyTo/email/isSystem keys.
• Discord and Teams text-fallback mention paths proven by tests in this PR.

Dependencies

Blocked by #191 (transitively #190 for _set_mention_flags). Blocks #199, #209, #216, #218, #228, #232, #203.

Verify first

• Teams coupling: the planning brief says teams/webhook/parse.py must switch to True | None here. On main nothing outside adapters/teams/webhook/ turns TeamsMessagePayload into a Message (grep -rn 'parse_teams_webhook_body\|TeamsMessagePayload' src/), and the adapter path already leaves None. Confirm, then keep parse.py unchanged (parity with upstream isTeamsMention(): boolean).
• Re-run grep -rn 'is_mention' src/chat_sdk/adapters/ to confirm no other adapter builds a Message with a definitive False where upstream leaves it undefined.

Metadata

• Effort: M (~200 LOC source, ~350 LOC tests)
• Consumer impact: low — no routing change for Slack/Teams streaming users (Teams text fallback kept). Visible: emails/@bot-dev no longer mention the bot; serialized messages gain optional keys; custom adapters returning False change behavior.
• Suggested branch: sync/4.41-c2a

Part of #184.