Repository navigation
Conversation
28c31dd to
cc43574
Compare
The gateway is OpenAI-compatible, so upstream provider errors arrive as
{"error": {"message": "..."}}. extract_detail() pulled the "error" key but
never unwrapped the nested dict, so callers got a Python repr string like
"{'message': 'context length exceeded'}" instead of the actual message.
Mirrors the fix already shipped in the sibling SDKs: otari-sdk-ts's
detailFromObject() (PR mozilla-ai#41, closing mozilla-ai#40) and otari-sdk-rust's
extract_detail(), which already documents this exact shape. Brings Python
in line with both.
cc43574 to
ee0ca59
Compare
peteski22
left a comment
There was a problem hiding this comment.
Thanks for this. The unwrap is the right direction and it is strictly better than str(detail) in every shape I tried. On ee0ca59: uv run pytest tests/unit 147 passed, uv run ruff check . clean, uv run mypy src/ clean.
Two substantive things before this lands, both verified against the gateway at tag v0.12.1.
1. The shape this targets is not what 0.12.x returns over HTTP. A top-level {"error": {"message": ...}} is only emitted mid-stream by src/gateway/streaming.py, and _streaming.py never routes SSE payloads through extract_detail. The error bodies 0.12.x actually returns from /api/v1/messages nest the message one level deeper:
# gateway src/gateway/api/routes/messages.py:396
detail={"type": "error", "error": {"type": error_type, "message": message}}which reaches the SDK as {"detail": {"type": "error", "error": {"type": ..., "message": ...}}}. detail.get("message") is None there, so it still falls through to the JSON dump, and string matching on the message (the motivating case in the description) still fails.
2. The new branch narrows guardrail 403s. src/gateway/api/routes/_helpers.py:349 returns a dict that does have a top-level message, plus code and a guardrails array carrying the flagged profile, explanation and score. After this change the exception message is just "Request blocked by guardrail policy." and the diagnostics are only reachable via err.original_error.body.
So the rule cannot be "take any message". It needs to be "take the message only when the dict is nothing but an error envelope". Suggested shape:
_ENVELOPE_KEYS = frozenset({"type", "message", "param", "code"})
def _envelope_message(detail: dict[str, Any]) -> str | None:
"""Return the message from an OpenAI or Anthropic style error envelope.
The message sits one level deep on the OpenAI surface and two levels deep on
the Anthropic surface.
A dict carrying keys outside the envelope holds diagnostics the caller needs,
so it is not reduced to its message.
"""
for candidate in (detail, detail.get("error")):
if isinstance(candidate, dict) and not candidate.keys() - _ENVELOPE_KEYS:
message = candidate.get("message")
if isinstance(message, str):
return message
return Noneand in extract_detail:
if isinstance(detail, dict):
message = _envelope_message(detail)
if message is not None:
return message
if detail is not None:
return json.dumps(detail, ensure_ascii=False)Resulting behavior:
| body | result |
|---|---|
{"error": {"message": "context length exceeded"}} |
context length exceeded |
{"detail": {"type": "error", "error": {"message": "Rate limit exceeded"}}} |
Rate limit exceeded |
| guardrail 403 | full JSON, code and guardrails preserved |
| 422 validation list | full JSON, non-ASCII intact |
Worth covering with tests: the two-level envelope, the guardrail dict keeping its diagnostics, and a non-ASCII message surviving the fallback path.
One thing not called out in the description: the 422 path changes from str(list) to json.dumps(list), so the message text of every validation error changes. It is an improvement, but it is wire-visible.
Nothing here is a merge blocker for safety. The concern is that the PR's own goal is only partly met against 0.12.x.
| Recognizes the FastAPI/gateway ``{"detail": "..."}`` shape and the | ||
| OpenAI-style ``{"error": {"message": "..."}}`` / ``{"error": "..."}`` | ||
| shapes, mirroring ``extract_detail`` in the Rust SDK (``core.rs``) and | ||
| ``detailFromObject`` in the TS SDK (``mapError.ts``). |
There was a problem hiding this comment.
nit: AGENTS.md asks that comments and docstrings not cite other repositories, and the commit body cites PR numbers too. Those references rot once the sibling SDKs move. The shapes themselves are the useful part; suggest keeping those and dropping the cross-SDK pointers.
There was a problem hiding this comment.
Removed the external SDK references from the touched source/test docstrings and PR description. The new commit message also has no cross-repository references. I preserved the maintainer-rebased commit history, including its original message.
| if isinstance(detail, str): | ||
| return detail | ||
| if isinstance(detail, dict): | ||
| nested = detail.get("message") |
There was a problem hiding this comment.
This only reaches one level down, but the Anthropic surface nests the message twice.
Gateway src/gateway/api/routes/messages.py:396 builds every /api/v1/messages error as:
detail={"type": "error", "error": {"type": error_type, "message": message}}so the SDK sees {"detail": {"type": "error", "error": {"type": "rate_limit_error", "message": "Rate limit exceeded"}}}. detail.get("message") is None, and the raised RateLimitError still carries the whole JSON blob rather than Rate limit exceeded.
Needs one more hop through detail["error"].
There was a problem hiding this comment.
Added detail.error.message extraction for the HTTP envelope and regression tests through /messages in both authentication modes. Those cases failed on ee0ca590 and now pass.
| if isinstance(detail, dict): | ||
| nested = detail.get("message") | ||
| if isinstance(nested, str): | ||
| return nested |
There was a problem hiding this comment.
Behavior change worth a second look: this returns the bare message for any dict that has one, including the guardrail 403 from src/gateway/api/routes/_helpers.py:349:
{"detail": {"message": "Request blocked by guardrail policy.",
"code": "guardrail_violation",
"guardrails": [{"profile": "...", "explanation": "...", "score": 0.9}]}}Before this PR str(err) showed the flagged profile, explanation and score, so a caller could tell why they were blocked. Now they get a generic sentence, and the rest is only on err.original_error.body.
Guarding on "the dict carries no keys outside the error envelope" keeps both this and the case above working.
There was a problem hiding this comment.
The helper now reduces only recognized envelopes. It checks the outer wrapper before following error, so diagnostics beside the nested error are preserved too. Tests cover unknown fields at both levels and the full guardrail code, profile, explanation, and score through both authentication modes.
| return nested | ||
| if detail is not None: | ||
| return str(detail) | ||
| return json.dumps(detail) |
There was a problem hiding this comment.
json.dumps defaults to ensure_ascii=True. Because of the missed nesting above, both the /messages path and the 422 validation path ({"detail": [{...}]}) land here, so an upstream provider message in Japanese or French now renders as \u30a8\u30e9\u30fc... where str(detail) rendered it readably.
return json.dumps(detail, ensure_ascii=False)There was a problem hiding this comment.
The fallback now uses ensure_ascii=False. A 422 validation-list test checks readable French and Japanese text plus a JSON round trip. The PR description now calls out the change from Python list text to JSON.
| result = extract_detail(err) | ||
| # Must be valid JSON round-tripping to the nested object, not a | ||
| # Python repr (which would contain single-quoted keys). | ||
| assert "'" not in result |
There was a problem hiding this comment.
nit: this is a proxy for "not a Python repr" and would misfire on any payload containing an apostrophe. The json.loads(result) == {"code": 400} on the next line already carries the real assertion, so this one can go.
There was a problem hiding this comment.
Removed the apostrophe assertion. The fallback test now includes an apostrophe in the value and checks only that parsing the JSON returns the original detail object.
Gateway HTTP errors with a nested
detail.error.messagenow expose the message directly.error.messageenvelopes and existing string details remain supported.Only recognized error envelopes are reduced to their message. Guardrail responses keep their
codeand completeguardrailsdiagnostics as JSON. Unknown fields on either the outer wrapper or nested error also keep the full selected detail object.Structured fallback uses
json.dumps(detail, ensure_ascii=False). This changes exception text for 422 validation lists from Python'sstr(list)representation to JSON. Non-ASCII text remains readable. Plain text bodies and string details are unchanged.Validation on Python 3.13.15:
ee0ca590, covering nested HTTP errors, guardrail diagnostics, and Unicode fallback.uv run pytest tests/unit -q: 159 passed.uv run ruff check .: passed.uv run mypy src/: passed.The transport tests exercise
/messageserror extraction and guardrail failures in both platform and self-hosted modes. These are mocked unit tests, not live gateway integration tests.