[4.41/GMAIL] Gmail adapter (demand-gated): primitives + Chat adapter

[patrick-chinchill]

Summary

Demand-gated scoping issue: do NOT start implementation until a customer asks for email; until then it stays deferred and #203 records Gmail as a known non-parity row. Upstream 4.41 added @chat-adapter/gmail: standalone primitives (/api OAuth token providers, messages, drafts, labels, history, watches; /format MIME parse/compose and serializable reply context; /webhook Pub/Sub parsing plus JWT verification) that never load the Chat runtime, and an optional Chat adapter (label-based intake via Pub/Sub watch, history-cursor sync, threaded replies). A Python port is feasible without Google's API client (stdlib email, pyjwt) and needs core reply() from #200. Gmail covers Google Workspace/consumer Gmail only; Microsoft 365 customers would need a separate Outlook/Graph adapter.

Upstream changes

• bfee00af feat(gmail): add email adapter and standalone primitives (#939) — chat@4.41.0 — the new package packages/adapter-gmail: about 2,060 non-test LOC across 11 modules (index.ts 699, api.ts 327, sync.ts 322, format.ts 200, webhook.ts 156, schema.ts 125, http.ts 118, ids.ts 57, …) and 95 it cases in 12 test files. npm deps: html-to-text, jose, mimetext, postal-mime, zod.
• f034cd27 docs(gmail): add adapter guide and catalog listing (#958) — chat@4.41.1 — the docs guide, chat/adapters catalog entry and scaffolder wiring. The catalog and scaffolder are a documented Python skip (docs/UPSTREAM_SYNC.md catalog row), so only the guide content is relevant here.

Upstream design, for reference:

• Intake. Messages are labelled into a handoff label (GMAIL_LABEL_ID). Gmail users.watch publishes to a Pub/Sub topic, and a wrapped, authenticated push subscription POSTs to the webhook. The webhook verifies the push JWT (audience plus push service account email, or a custom webhook_verifier), then runs sync(). sync() pages users.history from a cursor in state, applies label checks before and after body fetch, coalesces per conversation, keeps durable receipts, records failed messages without replaying them, and recovers from an expired cursor by scanning labelled messages.
• Thread and message ids. gmail:{b64url(mailbox)}:{gmailThreadId}, and …:dm:{b64url(address)} for recipient routes. lockScope = "thread".
• Outbound.
  • post_message and reply send immediately, building reply headers (In-Reply-To, References, Subject) from a serializable continuation. Replies go to the sender only by default; GMAIL_REPLY_ALL enables reply-all only when its value is exactly "true", and Bcc is never copied.
  • Drafts are a separate primitive.
  • stream buffers the whole response into one email.
  • edit, delete, reactions and typing are unsupported.
  • post_ephemeral with fallback_to_dm sends a permanent private email prefixed (private only).
• Mailbox. mailbox must be a real address, not "me", because it scopes notification validation and state.

Current Python behavior

• ls src/chat_sdk/adapters/ shows no gmail, and grep -rniE 'gmail' src tests docs finds nothing.
• Reusable: the PyJWKClient verification pattern at src/chat_sdk/adapters/google_chat/adapter.py:747-753 (identity-binding fixes land in [4.41/G1] Google Chat: bind webhook JWT verification to configured identity #222; reuse the fixed version); google-auth already in the google-chat extra (pyproject.toml:67); state set_if_not_exists for cursors/receipts (Postgres expiry fix in [4.41/ST1] Postgres state: reclaim expired set_if_not_exists rows, migration-managed schemas #240); Thread.reply and post_ephemeral(options=) from [4.41/C6] Thread.reply, Thread.mark_as_read, post_ephemeral options #200.

Scope

When unblocked, split the work into two PRs. Each maps to one reviewable PR.

• PR A: primitives. Add chat_sdk/adapters/gmail/{api,format,webhook,_http,_ids,_schema}.py. None of them may import chat_sdk.chat; add an import-boundary test like upstream's boundary.test.ts.
  • api: a token provider (refresh-token grant with single-flight refresh), static-token and async-resolver inputs, and messages/threads/drafts/labels/history/watch/stop calls against gmail.googleapis.com/gmail/v1/users/{mailbox}/…. Include a typed GmailApiError.
  • format: MIME parsing into a structured email, a composer, and extract_gmail_continuation(email, mailbox, reply_all=False).
  • webhook: a Pub/Sub envelope parser and a verifier that checks audience, issuer and exact service-account email.
• PR B: Chat adapter.
  • GmailAdapter / create_gmail_adapter with env fallbacks (GMAIL_*, per the upstream README table).
  • watch(), sync(), handle_webhook, thread-id encode/decode, parse_message, fetch_messages, post_message, reply, stream (buffered) and post_ephemeral (private fallback).
  • Unsupported operations raise ChatNotImplementedError.
  • A new gmail extra.
• Add a docs page (a Python-flavoured version of the upstream guide) and a docs/UPSTREAM_SYNC.md row flipping Gmail from "deferred" to "ported".

Out of scope

• An Outlook / Microsoft Graph mail adapter for M365 tenants: a separate future issue if demanded.
• The chat/adapters catalog entry and create-chat-sdk scaffolder: permanently skipped (see [4.41/C11] Bump fidelity pin + UPSTREAM_PARITY to chat@4.41.1, record non-parity rows, cut 0.4.41 #203).
• Other deferred platforms: [4.41/DEF1] Deferred upstream 4.32–4.41 features (demand-gated backlog) #189.

Porting notes

• Dependency mapping. postal-mime/mimetext → stdlib email (BytesParser(policy=policy.default), EmailMessage); jose → pyjwt[crypto] PyJWKClient; zod → explicit validators (e.g. history id ^\d+$, max 64 chars); html-to-text → stdlib html.parser or an optional lazy import. No google-api-python-client: call REST with aiohttp, as upstream uses raw fetch.

• Auth for single-tenant deployments. Accept a token that is a str or an async () -> str resolver, in the same way twilio/api.py resolves credentials. That lets a deployment plug in:

  1. an Internal-type OAuth app's refresh token (Workspace-internal apps avoid restricted-scope public verification), or
  2. domain-wide delegation: a service account impersonating the mailbox via google.oauth2.service_account.Credentials.with_subject(...) (google-auth, lazy import), or a pyjwt-signed JWT-bearer grant.

  Recommended default: ship the refresh-token provider (upstream parity) plus the resolver hook, and document domain-wide delegation as a recipe, not built-in code. Built-in refresh is single-flight (cached asyncio.Task or asyncio.Lock) and a failed refresh clears the cache; mailbox == "me" raises; explicit OAuth config beats an env access token.

• Config truthiness. reply_all = config.reply_all if config.reply_all is not None else os.environ.get("GMAIL_REPLY_ALL") == "true". Every other env fallback uses is not None, never or.

• Webhook verification. Compare the service-account email exactly, require email_verified is True, and fail closed when the configuration is missing, following the [4.41/G1] Google Chat: bind webhook JWT verification to configured identity #222 pattern. Use hmac.compare_digest for any shared-secret verifier.

• Sync correctness. Advance the history cursor only after every page is handled. Never replay a recorded failure. Record receipts with set_if_not_exists. Guard against repeated page tokens. Keep MIME input and output capped at 25 MiB.

• Logging. Never log message bodies or addresses. Log native ids and a reason code only, and keep consistent with [4.41/LOG1] Stop logging raw webhook bodies / PII across adapters #187.

Tests

Upstream test files: api.test.ts (18), boundary.test.ts (4), cards.test.ts (1), delivery.test.ts (12), failure.test.ts (7), flow.test.ts (1), format.test.ts (6), history.test.ts (6), index.test.ts (12), markdown.test.ts (2), sync.test.ts (15) and webhook.test.ts (11). The top-level describes include "Gmail API primitives", "Gmail primitive boundaries", "Gmail history synchronization", "Gmail Pub/Sub primitives" and "Gmail optional Chat adapter".

None of these files are fidelity-mapped (scripts/verify_test_fidelity.py MAPPING covers core packages/chat/src only). When implementing, port the it() names verbatim from the 4.41.1 files; don't paraphrase. Use AsyncMock transports, locally generated RSA keys for the JWT tests, and a fake clock for token expiry.

Acceptance criteria

• (When unblocked) PR A and PR B each pass the full validation command from CLAUDE.md.
• The primitives have no chat_sdk.chat import, and a test enforces it.
• docs/UPSTREAM_SYNC.md has the Gmail row updated, and the Python-only token-resolver/domain-wide-delegation guidance is recorded as a divergence.
• CHANGELOG entry under an "Unreleased (4.41 wave)" heading, or the then-current heading.
• Consumer-visible behavior is documented: a new optional gmail extra, and no change for existing adapters.

Dependencies

Blocked by #200 (Thread.reply, post_ephemeral options). It should also follow #222 (the JWT identity-binding pattern) and #240 (Postgres set_if_not_exists expiry), because receipts and cursors depend on them. Gated on customer demand.

Verify first

• Confirm demand, and whether the customer is on Google Workspace (this adapter) or Microsoft 365 (which needs an Outlook/Graph adapter instead).
• Confirm the customer's auth model: an Internal OAuth app, or domain-wide delegation. This decides whether the resolver hook is enough.

Metadata

• Effort: XL — split into two L PRs (primitives ~800 LOC plus tests; adapter and sync ~1.2k LOC plus tests).
• Consumer impact: none. This is a new opt-in adapter, with nothing changed for Slack/Teams.
• Suggested branch: sync/4.41-gmail, or sync/4.41-gmail-primitives / sync/4.41-gmail-adapter.

Part of #184.