fix(adapters): stop logging raw webhook bodies and message content (#187)

CHANGELOG.md

@@ -13,6 +13,31 @@ Part of the upstream `4.31.0` → `4.41.1` sync wave (#184). `UPSTREAM_PARITY` s
   - **Consumer-visible (custom `api_url` only):** with a non-default `api_url`, media hosted on any other origin, including inbound media on `https://api.twilio.com`, is now refused (upstream behaves the same). With a non-Twilio or `http` `api_url` (a proxy or local mock), no media URL passes both layers: `api.twilio.com` fails the origin check and the proxy origin fails the host allowlist, so attachment downloads always raise. Before this change such configs downloaded `api.twilio.com` media. The default config (`api_url` unset) is unaffected.
 - **Teams: cap `microsoft-teams-{apps,api,cards}` at `<2.1`.** `uv.lock` is not committed and the extras were unbounded, so fresh installs resolved `microsoft-teams-apps` 2.1.0 (released 2026-09-16). 2.1.0 removed `App.activity_sender`, which the adapter uses to create native DM streams (`teams/adapter.py:943`), and changed the activities-client `update` signature that `edit_message`'s service-URL retargeting relies on. Native streaming and edits could fail on a fresh install, and CI turned red. The cap resolves to 2.0.16 until the adapter supports 2.1.
 
+### Security
+
+- **Webhook log hygiene: raw bodies and message content no longer reach DEBUG logs** (#187; ports upstream `fc7df9c4` / vercel/chat#500 and the logging parts of `f485255b` / vercel/chat#877). Several adapters logged raw webhook bodies, or previews of them, at DEBUG. In some cases this happened before signature or JWT verification, so unauthenticated input and message content could be copied into log sinks. Webhook handlers now log only request-shape metadata:
+  - GitHub: `{bodyBytes, contentType, eventType, signaturePresent}`, under "GitHub webhook signature verification failed" or "GitHub webhook request verified", plus `jsonParseStatus: "error"` on invalid JSON.
+  - GChat, Slack (after verification only) and the Teams bridge: `"… webhook received" {bodyLength}`.
+  - Linear: the raw-body log is gone.
+  - `Chat` "Incoming message": drops `author` and adds `is_bot`, matching upstream's key set.
+- **Consumer-visible (DEBUG logs only; no routing, response or status change):**
+  - The `"GitHub/GChat/Slack/Teams/Linear/WhatsApp webhook raw body"` messages are gone. GChat, Slack and Teams now emit `"… webhook received"` with `bodyLength`, which is a UTF-8 byte count.
+  - GitHub's `"GitHub webhook event type"` is replaced by `"GitHub webhook request verified"`.
+  - `bodyPreview` becomes `bodyBytes` on the GitHub, Linear and WhatsApp invalid-JSON errors.
+  - "Incoming message" loses `author`.
+
+  Anything that parses these log lines must be updated.
+
+### Python-specific (divergence from upstream)
+
+- **WhatsApp** drops its raw-body debug log and invalid-JSON `bodyPreview` (#187). Upstream 4.41.1 still logs both.
+- **Message-content debug logs** (#187):
+  - GChat "message event" logs `{space, textLength}`.
+  - GChat "Pub/Sub parsed message" drops `text` and `author`.
+  - The `Chat`, Slack and Discord slash-command debug logs log `textLength` instead of `text`.
+
+  Upstream still logs this content. Both divergences are recorded in `docs/UPSTREAM_SYNC.md`.
+
 ## 0.4.31.3
 
 Python-only fixes on top of `4.31.0` (`UPSTREAM_PARITY` unchanged at `4.31.0`). Same content as the `0.4.31.2` tag, which never reached PyPI: the publish action's pinned twine rejected the `Metadata-Version 2.5` that uv's build backend now emits (fixed in #182), and the tag is immutable, so the release ships as 0.4.31.3.

docs/UPSTREAM_SYNC.md

@@ -290,6 +290,7 @@ These exist only in the Python port and have no TS equivalent:
 - `from __future__ import annotations` everywhere: Enables PEP 604 union syntax (`X | Y`) without runtime cost.
 - Input validation on adapter config dataclasses (e.g., rejecting empty `signing_secret`).
 - `ContextVar`-based request context in Slack adapter (instance variable, not class variable).
+- **Webhook log hygiene (#187, ports upstream `fc7df9c4` + the logging parts of `f485255b`).** Webhook handlers log request-shape metadata, never bodies. GitHub logs `{bodyBytes, contentType, eventType, signaturePresent}` (plus `jsonParseStatus: "error"` on invalid JSON). GChat, Slack and the Teams bridge log `"<Platform> webhook received" {bodyLength}`. Byte lengths go through `shared/log_utils.utf8_byte_length` (UTF-8 bytes like `Buffer.byteLength`, `None`→0, lone surrogates never raise). Slack logs `bodyLength` only **after** `verify_slack_request` succeeds, and GitHub logs its metadata once verification has decided, so a rejected request never gets more than size and header metadata into the logs. `Chat` "Incoming message" uses upstream's key set (`adapter`, `thread_id`, `message_id`, `is_bot`, `is_me`) with no author. Two further steps go beyond upstream and are recorded in [Known Non-Parity](#known-non-parity-with-typescript-sdk): WhatsApp raw-body logging, and message-content logs. Linear's raw-body log was Python-only (upstream Linear never had one), so removing it restores parity; its Python-only invalid-JSON error now logs `{bodyBytes, contentType}`.
 
 ## Common TS-to-Python Translation Patterns
 
@@ -714,6 +715,9 @@ stay explicit instead of being rediscovered in code review.
 | Linear agent-activity emit: raw GraphQL (chat@4.31 / #151 — L4) | The agent-session EMIT path (`post_message` session branch, `start_typing` session branch, `stream` → `_stream_in_agent_session` with its flush/`task_update`/`plan_update` logic, and `_parse_message_from_agent_activity`) is ported as **raw GraphQL mutations** over the existing `_graphql_query` helper, **schema-hardened against Linear's published GraphQL schema**. Mutation names: `agentActivityCreate(input: AgentActivityCreateInput!)` and `agentSessionUpdate(id: String!, input: AgentSessionUpdateInput!)`. `content` is sent inside the `AgentActivityCreateInput.content` **`JSONObject!`** scalar — so `type`/`body`/`action`/`parameter`/`result` are inline JSON fields, with the **lowercase** `AgentActivityType` enum values `"response"` / `"thought"` / `"error"` / `"action"` (confirmed lowercase, NOT PascalCase). `ephemeral: Boolean` is a sibling of `content` and is **included only when set** (absent — not `false` — on response/thought/error). The `agentActivityCreate` return selection requests only schema-valid fields (`success`, `agentActivity { id agentSession { id } sourceComment { id body parentId createdAt updatedAt url user{…} botActor{…} } }`) — `agentSessionId` is **not** a scalar field on Linear's `AgentActivity` type (the schema exposes only the relation `agentSession: AgentSession!`), so the session id is read off the nested `agentSession { id }` relation; requesting the non-existent scalar would server-reject the whole mutation under GraphQL strict selection validation. `plan` items are `{content, status:"completed"}`. `initialize` additionally captures the viewer's `organization.id` into `_default_organization_id` (mirroring upstream's `defaultOrganizationId`) for the emitted raw message's `organizationId`; on the emit path `organizationId` falls back to `""` when `_default_organization_id` is unset (no per-request installation context is plumbed — a pre-existing adapter-wide divergence from upstream, which throws `AuthenticationError` when no organization is resolvable). | Upstream `adapter-linear/src/index.ts` calls `@linear/sdk`'s `createAgentActivity({agentSessionId, content, ephemeral?})` / `updateAgentSession(id, {plan})`; the SDK owns the GraphQL document, the `AgentActivityType` enum, and the `AgentActivityPayload`/`Comment`/`sourceComment` resolution | `@linear/sdk` is TypeScript-only (no official Linear Python SDK — cf. the `linear_client` getter row), so the SDK calls are reproduced as raw GraphQL. Mutation names, the `content: JSONObject!` shape, the lowercase enum casing, and the `plan` item shape are **schema-hardened against the published schema** (`https://linear.app/developers/agent-interaction`, `https://linear.app/developers/graphql`, and the SDK's generated GraphQL documents). **Live-tenant verification pending**: the exact mutation/field names and enum casing are confirmed against the published schema/docs but have **not** been exercised against a live Linear agent-session tenant; if a future live run surfaces a casing/field mismatch (e.g. an enum the schema renders differently at runtime), update the mutation strings here. Faithful-port hazards preserved: `status ?? "Thinking..."` → `is not None` (an empty status stays `""`); `[title, output].filter(Boolean).join("\n")` → `"\n".join(x for x in [title, output] if x)` (drops `None` and `""`); `markdown.slice(...).trim()` uses the JS-`.trim()` whitespace set (`_JS_WHITESPACE`, mirroring `adapters/telegram/rich.py`), not Python's broader `str.strip()`; `if delta or force`; `ephemeral: status != "complete"`; the missing-final-flush bare `throw new Error(...)` → `RuntimeError`. Regression coverage: `tests/test_linear_agent_session_emit.py`. |
 | Linear agent-session fetch: raw GraphQL (chat@4.31 / #151 — L5) | The agent-session FETCH/read path (`fetch_messages` session dispatch → `_fetch_agent_session_messages`, plus the append-only guards on `edit_message`/`delete_message` and the `agentSessionId` key in `fetch_thread` metadata) is ported as **raw GraphQL queries** over the existing `_graphql_query` helper, **schema-hardened against Linear's published GraphQL schema** (`linear/packages/sdk/src/schema.graphql` @ master). Two queries: (1) `agentSession(id: String!): AgentSession!` selecting `id`, `issue { id }`, and the nullable `comment { id body parentId createdAt updatedAt url user{…} botActor{…} }` root relation; (2) `comments(filter: CommentFilter, first: Int, last: Int): CommentConnection!` filtered by `{parent: {id: {eq: rootComment.id}}}` for the children, selecting the same `Comment` sub-fields + `pageInfo { hasNextPage endCursor }`. Upstream passes ONLY `first`/`last` here — it never reads `options.cursor` — and the sibling `_fetch_issue_comments`/`_fetch_comment_thread` paths forward no cursor either, so no inbound `after` is plumbed (only `next_cursor` is RETURNED, off `pageInfo.endCursor`). **CRITICAL schema-hardening: `AgentSession` has NO scalar `issueId` field in the published schema** (it exposes only the `issue: Issue` relation alongside `comment`/`sourceComment`/`id`); upstream's `agentSession.issueId` works because `@linear/sdk`'s model derives it from the serialized object, but in raw GraphQL requesting a non-existent `issueId` field would server-reject the whole query (the L4 blocking-bug class). So the issue id is read off the `issue { id }` relation — equivalent to upstream's `agentSession.issueId ?? thread.issueId`, and the same `issueId ?? issue?.id` fallback upstream itself uses at `index.ts:959`. Pagination is direction-driven (`forward` → `first`, otherwise `last`, default limit 50); `next_cursor = endCursor if hasNextPage else None`. Each of `[rootComment, *children.nodes]` is parsed via the upstream `parseMessageFromComment(comment, issueId, agentSession.id)` semantics — reusing L4's `_raw_message_from_source_comment` (user-vs-`botActor` author resolution) + `_parse_agent_session_message` (the `parseMessage` agent-session branch), so **each message's `thread_id` encodes the comment's OWN id** (`linear:{issueId}:c:{comment.id}:s:{agentSessionId}`, NOT a single fixed thread id) and `is_mention=True`. `edit_message`/`delete_message` raise `AdapterError` with the exact upstream strings ("…append-only and cannot be edited" / "…cannot be deleted") for session threads, before any network call. | Upstream `adapter-linear/src/index.ts` calls `@linear/sdk`'s `linear.agentSession(id)` (lazy-resolving `issueId` + the `comment` relation off the SDK model) and `linear.comments({filter, first/last})`; the SDK owns the GraphQL documents and the `Comment`/author resolution. | `@linear/sdk` is TypeScript-only (no official Linear Python SDK — cf. the `linear_client` getter row), so the SDK calls are reproduced as raw GraphQL. Query names, the `agentSession(id)` shape, the root `comments(filter: CommentFilter, first/last)` connection, the `CommentFilter.parent → NullableCommentFilter.id → IDComparator.eq` chain, and every selected `AgentSession`/`Comment`/`User`/`ActorBot`/`PageInfo` field were each **verified field-by-field against the published `schema.graphql`** (this is how the absence of a scalar `AgentSession.issueId` was caught). **Live-tenant verification pending**: the query/field names are confirmed against the published schema but have **not** been exercised against a live Linear agent-session tenant; if a future live run surfaces a field mismatch, update the query strings here. Nullish hazards preserved: `agentSession.issue.id ?? thread.issue_id` and `endCursor ?? undefined` → `is not None` (NOT `or` — an empty issue id still short-circuits per `??`). Regression coverage: `tests/test_linear_agent_session_fetch.py`. |
 | `chat/adapters` static adapter catalog (chat@4.31.0, new `./adapters` package subpath) | Not ported | A new SDK-free metadata module (`packages/chat/src/adapters/index.ts`, 19 `test()` cases): types `EnvVar`/`EnvGroup`/`AdapterEnvSpec`/`CatalogAdapter`, an `ADAPTERS` registry of ~25 official + vendor-official adapters, and exports `ADAPTER_NAMES`/`AdapterSlug`/`getAdapter`/`isAdapterSlug`/`listPlatformAdapters`/`listStateAdapters`/`listEnvVars`/`getSecretEnvVars`. Imports no provider SDK, so it's safe for build scripts, onboarding/setup screens, and config-discovery UIs that need package + env-var metadata (incl. secret-masking flags) without loading an adapter. | **Not meaningfully portable verbatim, and not yet needed.** The catalog's spine is TypeScript-ecosystem-specific: every entry is addressed by an **npm `packageName`** (`@chat-adapter/slack`, `@kapso/chat-adapter`, …) and the registry includes ~13 **vendor-official adapters this Python SDK does not ship** (AgentPhone, Kapso, Lark/Feishu, Liveblocks, Beeper Matrix, Resend, Sendblue, ioredis, …). A 1:1 port would ship actively-misleading data to a `pip`-installed consumer (`@chat-adapter/slack` instead of `chat-sdk[slack]`; `get_adapter("kapso")` returning metadata for something uninstallable here). The only Python-applicable slice — per-platform env-var requirements + secret flags — is the same across the 9 platform + 3 state adapters we do ship, but exposing it would be a **new Python-native feature** (a divergent rewrite around `pip` extras, not a port), and no current consumer (chinchill-api configures adapters in code, not via env-var discovery) needs it. Nothing in chat-sdk's core depends on the catalog. Deferred demand-driven: a Python-native `adapters` catalog (our adapters only, `pip`-extra install names, `get_secret_env_vars` masking) can be designed against real requirements if/when a consumer needs config discovery. The unmapped test file is additionally covered by the issue #78 fidelity-scope note. |
+| WhatsApp webhook raw-body logging (#187) | `handle_webhook` logs no body: the pre-verification `"WhatsApp webhook raw body"` debug log is removed, and `"WhatsApp webhook invalid JSON"` logs `{bodyBytes, contentType}` | Upstream `adapter-whatsapp/src/index.ts` (still at chat@4.41.1) logs `body.substring(0, 500)` **before** signature verification and `bodyPreview: body.substring(0, 200)` on invalid JSON | Log hygiene. The body arrives before authentication and carries message content and phone numbers, and DEBUG is commonly on in dev/staging. It is the same weakness class upstream fixed for GitHub (`fc7df9c4`) and GChat/Slack/Teams (`f485255b`). Regression tests: `tests/test_webhook_log_hygiene.py::TestWhatsAppLogHygiene`. Delete this row once upstream drops the logs. |
+| Message-content debug logs (#187) | GChat `"message event"` logs `{space, textLength}` (no `sender` display name, no `text` prefix). GChat `"Pub/Sub parsed message"` drops `text` and `author`. The slash-command debug logs (`Chat` `"Incoming slash command"`, Slack `"Processing Slack slash command"`, Discord `"Processing Discord slash command"`) log `textLength` instead of `text`. `textLength` counts characters (code points). | Upstream (chat@4.41.1) still logs the GChat sender display name + `text.slice(0, 50)`, the full Pub/Sub message `text` + author `fullName`, and slash-command `text` | Upstream's `f485255b` removed message text from `chat.ts` "Incoming message", and these are the same class of log. User-authored message text is kept out of DEBUG sinks. Action/reaction logs that still carry `user`/`user_name` stay at parity. Regression tests: `tests/test_webhook_log_hygiene.py` (`TestGoogleChatLogHygiene`, `TestSlackLogHygiene::test_slash_command_log_has_text_length_not_text`, `TestDiscordLogHygiene`, `TestChatLogHygiene::test_slash_command_log_has_text_length_not_text`). |
+
 ### Platform-specific gaps
 
 | Area | Python | TS | Rationale |

src/chat_sdk/adapters/discord/adapter.py

@@ -470,7 +470,9 @@ def _handle_application_command_interaction(
             "Processing Discord slash command",
             {
                 "command": command,
-                "text": text,
+                # Divergence from upstream — see docs/UPSTREAM_SYNC.md: the
+                # command text's length, not its content.
+                "textLength": len(text),
                 "userId": user.get("id"),
                 "channelId": channel_id,
             },

src/chat_sdk/adapters/github/adapter.py

@@ -39,6 +39,7 @@
 from chat_sdk.logger import ConsoleLogger, Logger
 from chat_sdk.shared.adapter_utils import extract_card
 from chat_sdk.shared.errors import ValidationError
+from chat_sdk.shared.log_utils import utf8_byte_length
 from chat_sdk.types import (
     AdapterPostableMessage,
     Author,
@@ -348,15 +349,25 @@ async def disconnect(self) -> None:
     async def handle_webhook(self, request: Any, options: WebhookOptions | None = None) -> Any:
         """Handle incoming webhook from GitHub."""
         body = await self._get_request_body(request)
-        self._logger.debug("GitHub webhook raw body", {"body": body[:500]})
+        signature = self._get_header(request, "x-hub-signature-256")
+        event_type = self._get_header(request, "x-github-event")
+        # Log request-shape metadata only — never the body or any slice of it
+        # (upstream fc7df9c4). ``bodyBytes`` is the UTF-8 byte length, matching
+        # ``Buffer.byteLength``; ``signaturePresent`` mirrors ``!== null``, so
+        # an empty header still counts as present.
+        webhook_metadata: dict[str, Any] = {
+            "bodyBytes": utf8_byte_length(body),
+            "contentType": self._get_header(request, "content-type"),
+            "eventType": event_type,
+            "signaturePresent": signature is not None,
+        }
 
         # Verify request signature
-        signature = self._get_header(request, "x-hub-signature-256")
         if not self._verify_signature(body, signature):
+            self._logger.debug("GitHub webhook signature verification failed", webhook_metadata)
             return self._make_response("Invalid signature", 401)
 
-        event_type = self._get_header(request, "x-github-event")
-        self._logger.debug("GitHub webhook event type", {"eventType": event_type})
+        self._logger.debug("GitHub webhook request verified", webhook_metadata)
 
         if event_type == "ping":
             self._logger.info("GitHub webhook ping received")
@@ -367,10 +378,7 @@ async def handle_webhook(self, request: Any, options: WebhookOptions | None = No
         except (json.JSONDecodeError, ValueError):
             self._logger.error(
                 "GitHub webhook invalid JSON",
-                {
-                    "contentType": self._get_header(request, "content-type"),
-                    "bodyPreview": body[:200],
-                },
+                {**webhook_metadata, "jsonParseStatus": "error"},
             )
             return self._make_response(
                 "Invalid JSON. Make sure webhook Content-Type is set to application/json",