feat(server): register Provider::Cohere on Hub for chat-compat (#332) by moonming · Pull Request #341 · api7/aisix

moonming · 2026-05-18T15:17:41Z

Summary

Cohere exposes an OpenAI-compatible chat endpoint at https://api.cohere.com/compatibility/v1/chat/completions. Until this PR Provider::Cohere was intentionally NOT Hub-registered (per #213 Phases 1-2 rerank-only plan), so customers who selected Cohere in the dashboard and POSTed /v1/chat/completions got 503 ProviderUnavailable. The rerank surface at /v1/rerank continues to bypass the Bridge via aisix-proxy::rerank — this PR only adds the chat-compat dispatch.

Surfaced by AISIX-Cloud's source-blind E2E matrix audit on PR #349 (D3.1 OpenAI-adapter long-tail), which held back Cohere chat-compat scenarios pending this Hub registration.

Fix

`OpenAiBridge` — `aisix-provider-openai/src/bridge.rs`

COHERE_DEFAULT_BASE = "https://api.cohere.com/compatibility/v1" constant for the with_name("cohere") variant's fallback base.
default_base() arm: "cohere" => COHERE_DEFAULT_BASE.
normalize_canonical_cohere: operators who paste the bare canonical host https://api.cohere.com (the rerank path / dashboard placeholder) get /compatibility/v1 synthesized for chat. Non-canonical hosts pass through verbatim.

`build_hub` — `aisix-server/src/main.rs`

hub.register(Provider::Cohere, Arc::new(OpenAiBridge::new().with_name("cohere"))).
Comment block updated: Jina alone stays rerank-only.

NOT touched

Provider::Cohere.default_base_url() in aisix-core stays as https://api.cohere.com (bare host) — the rerank URL builder appends /v1/rerank. The bridge handles chat-compat in its own resolve_base().

Tests

Three new bridge tests:

Test	Pins
`cohere_default_base_targets_compatibility_v1`	empty `api_base` → bridge falls back to `/compatibility/v1`
`cohere_api_base_tolerance_bare_host_synthesizes_compatibility_prefix`	bare host, trailing slash, full chat URL all normalize; custom host passes through
`cohere_chat_compat_round_trips_openai_envelope`	end-to-end chat through `with_name("cohere")` returns the OpenAI envelope verbatim

All 79 aisix-provider-openai tests pass; clippy clean; aisix-server builds clean.

References (per CLAUDE.md §7)

Cohere chat-compat docs — /compatibility/v1/chat/completions shape
LiteLLM's Cohere chat-compat — same /compatibility/v1 namespace
Portkey routes Cohere chat through /compatibility/v1 too

Test plan

CI green (cargo test + clippy + fmt)
AISIX-Cloud dashboard provider list update: defaultBase: 'https://api.cohere.com/compatibility/v1' (separate AISIX-Cloud PR; bridge tolerance covers bare-host PKs migration-free)
Once merged, AISIX-Cloud held-back D3.1 cohere matrix scenarios flip on

Closes #332

Summary by CodeRabbit

New Features
- Added Cohere provider support for chat/completions and embeddings via an OpenAI-compatible bridge; hub now registers Cohere for chat-compat.
Documentation
- Expanded provider docs and schema description to clarify Cohere routes and compatibility coverage.
Tests
- Extended test coverage for Cohere routing, API base normalization variations, and an end-to-end chat round-trip.

Cohere exposes an OpenAI-compatible chat endpoint at `https://api.cohere.com/compatibility/v1/chat/completions` per <https://docs.cohere.com/reference/chat>. Before this commit `Provider::Cohere` was intentionally NOT Hub-registered (per #213 Phases 1-2 rerank-only plan), so a customer who selected Cohere in the dashboard and POSTed /v1/chat/completions got 503 ProviderUnavailable. The rerank surface at `/v1/rerank` continues to bypass the Bridge via aisix-proxy::rerank — this commit only adds the chat-compat dispatch. Fix 1. `OpenAiBridge` (`aisix-provider-openai/src/bridge.rs`) - New `COHERE_DEFAULT_BASE = "https://api.cohere.com/compatibility/v1"` constant for the `with_name("cohere")` variant's fallback base. - `default_base()` arm: `"cohere" => COHERE_DEFAULT_BASE`. - `normalize_api_base` arm + `normalize_canonical_cohere`: operators who paste the bare canonical host `https://api.cohere.com` (the rerank path / dashboard placeholder) get `/compatibility/v1` synthesized for chat. Non-canonical hosts pass through verbatim — operator's path on a custom host wins. 2. `build_hub()` (`aisix-server/src/main.rs`) - `hub.register(Provider::Cohere, Arc::new(OpenAiBridge::new().with_name("cohere")))` - Comment block updated: Jina alone stays rerank-only; Cohere now serves chat-compat via the bridge. `Provider::Cohere.default_base_url()` in `aisix-core` stays as `https://api.cohere.com` (bare host) because the rerank URL builder appends `/v1/rerank` to it. The bridge handles chat-compat in its own resolve_base(). Tests Three new bridge tests under `aisix-provider-openai/src/bridge.rs::tests`: - `cohere_default_base_targets_compatibility_v1` — empty `api_base` falls back to `/compatibility/v1`. - `cohere_api_base_tolerance_bare_host_synthesizes_compatibility_prefix` — bare host, trailing slash, full chat URL all normalize; a corporate-proxy host passes through unchanged. - `cohere_chat_compat_round_trips_openai_envelope` — end-to-end chat through `with_name("cohere")` returns the OpenAI envelope verbatim. Pins the contract Hub.register relies on. All 79 `aisix-provider-openai` tests pass; clippy clean; `aisix-server` builds clean. References (per CLAUDE.md §7) - Cohere chat-compat docs: https://docs.cohere.com/reference/chat - LiteLLM's Cohere chat-compat handler at https://github.com/BerriAI/litellm/blob/main/litellm/llms/cohere/chat/transformation.py uses the same `/compatibility/v1` namespace. - Portkey's Cohere routing: similar — points at `/compatibility/v1`. Closes #332

coderabbitai · 2026-05-18T15:17:52Z

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Free

Run ID: 7b3bc6ea-948a-4c41-ba18-0a9dfe9c2793

📥 Commits

Reviewing files that changed from the base of the PR and between 848a1a0 and 09e8f81.

📒 Files selected for processing (3)

crates/aisix-core/src/models/model.rs
crates/aisix-server/src/main.rs
schemas/resources/model.schema.json

✅ Files skipped from review due to trivial changes (2)

schemas/resources/model.schema.json
crates/aisix-core/src/models/model.rs

🚧 Files skipped from review as they are similar to previous changes (1)

crates/aisix-server/src/main.rs

📝 Walkthrough

Walkthrough

Adds Cohere support for OpenAI-compatible chat by normalizing Cohere canonical bases (synthesizing /compatibility/v1 when appropriate), registering Provider::Cohere with OpenAiBridge, and adding tests verifying base normalization and end-to-end chat compatibility.

Changes

Cohere OpenAI-Compatible Bridge Support

Layer / File(s)	Summary
Cohere base and normalization in OpenAiBridge `crates/aisix-provider-openai/src/bridge.rs`	Introduces `COHERE_DEFAULT_BASE`, extends `default_base()` and `normalize_api_base()` to dispatch `"cohere"`, and implements `COHERE_CANONICAL_HOSTS` with `normalize_canonical_cohere()` to append `/compatibility/v1` only for bare canonical hosts while preserving custom `api_base` values.
Hub registration and Cohere bridge testing `crates/aisix-provider-openai/src/bridge.rs`, `crates/aisix-server/src/main.rs`, `crates/aisix-core/src/models/model.rs`, `schemas/resources/model.schema.json`	Registers `Provider::Cohere` in `build_hub()` via `OpenAiBridge::with_name("cohere")`, updates `Provider::Cohere` documentation and schema description, and adds tests covering default base routing, operator `api_base` tolerance, and an end-to-end chat compatibility round-trip.

🎯 3 (Moderate) | ⏱️ ~20 minutes

Note

🎁 Summarized by CodeRabbit Free

Your organization has reached its limit of developer seats under the Pro Plan. For new users, CodeRabbit will generate a high-level summary and a walkthrough for each pull request. For a comprehensive line-by-line review, please add seats to your subscription by visiting https://app.coderabbit.ai/login.If you believe this is a mistake and have available seats, please assign one to the pull request author through the subscription management page using the link above.

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

Copilot

Pull request overview

This PR enables Cohere chat-completions dispatch by registering Provider::Cohere with the OpenAI-compatible bridge and teaching that bridge to target Cohere’s /compatibility/v1 namespace.

Changes:

Adds Cohere default base URL and canonical host normalization in OpenAiBridge.
Registers Cohere in the server Hub using OpenAiBridge::with_name("cohere").
Adds bridge tests covering Cohere default base resolution, normalization, and chat round-trip behavior.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File	Description
`crates/aisix-server/src/main.rs`	Registers Cohere on the Hub and updates provider-registration comments.
`crates/aisix-provider-openai/src/bridge.rs`	Adds Cohere base URL handling, normalization, and unit coverage for chat compatibility.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

+/// `Provider::Cohere` is registered against the OpenAI-compatible
+/// chat endpoint at `https://api.cohere.com/compatibility/v1` (per
+/// <https://docs.cohere.com/reference/chat>). Cohere's rerank surface
+/// at `/v1/rerank` continues to bypass the Bridge via
+/// `aisix-proxy::rerank` — the bridge here only serves `chat/completions`,
+/// `embeddings`, and the other OpenAI-shape endpoints the bridge
+/// supports. The chat-compat namespace gives an exact OpenAI envelope
+/// shape so `OpenAiBridge::with_name("cohere")` can serve it directly
+/// (closes #332).


…nt + build_hub registration tests Independent audit (CLAUDE.md §7) of PR #341 surfaced: - HIGH-1: `crates/aisix-core/src/models/model.rs:28-31` still documented `Provider::Cohere` as "exposed for /v1/rerank only" and "chat / generate APIs are not OpenAI-compatible". Both statements contradict the new Hub registration. Updated to describe the chat-compat path (https://api.cohere.com/compatibility/v1 per https://docs.cohere.com/reference/chat) and the model coverage caveat (command-r / command-a family on chat-compat; legacy command* not yet bridged). - MEDIUM-1: nothing test-side ties `Provider::Cohere` to `OpenAiBridge::with_name("cohere")` — the wiremock test in bridge.rs uses an override api_base so a regression where build_hub registered `OpenAiBridge::new()` (default name "openai") would still pass. Added two tests on `build_hub()`: - `build_hub_registers_cohere_chat_compat_variant` — asserts `hub.get(Provider::Cohere).unwrap().name() == "cohere"`. A `with_name("openai")` fallback would silently route Cohere chat to OpenAI's host via the bridge's `default_base()`. - `build_hub_does_not_register_jina_for_chat` — companion check so a future Jina chat-compat enablement is a deliberate change, not a silent regression. MEDIUM-2 (cohere streaming e2e), MEDIUM-3 (tool-use forwarding), MEDIUM-4 (model coverage gate) — scoped out of this PR per audit's recommendation, will land as separate enhancements after the chat-compat baseline merges. LOW findings deferred (passthrough comment staleness; endpoint- suffix corner case).

…comment update (#332)

Copilot

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

+    hub.register(
+        Provider::Cohere,
+        Arc::new(OpenAiBridge::new().with_name("cohere")),
+    );


Integrate origin/main (commit 2c1d485 = post-PR-#326 / #348 plus #330 / #341 / #343 / #345 / #346) into this branch via `git merge --squash` to clear PR #344's lingering `mergeable: dirty` state. Conflict on `docs/quickstart/self-hosted.md` was a 3-way-merge-base artifact: base (3596c0a) read `- a reachable etcd instance`, main changed `a` → `A` (via #326), this branch additionally inserted the glossary link. Both changes are wanted; resolution per Umar's approved plan was `git checkout --ours`, which preserves the branch's self-hosted.md state (already integrates capital A + glossary link + first-time-build paragraph + keep-running framing). Other 4 overlapping doc files auto-merged cleanly (`bootstrap-config.md`, `core-concepts.md`, `first-model-first-key-first-request.md`, `openai-sdk.md`). Code files all auto-merged cleanly. Additional Copilot review (post-`167196a` cycle) addressed: - `docs/index.md:7` — change link display text from `[data-plane]` to `[data plane]` to match the canonical glossary term. The URL anchor `#data-plane` stays kebab-case (matches the glossary heading's auto-anchor); only the display text changes. Comment id 3271145422. - `docs/quickstart/openai-sdk.md:43` — change `All three steps below` to `All commands below`. The Install-the-SDK section has two command blocks (mkdir+cd, npm install), not three; the prior wording originated from a mental model (mkdir, cd, install) that doesn't match the typographic count of code blocks under the heading. Comment id 3271145458. Copilot's third comment on `docs/overview/core-concepts.md` Observability Exporter wording (id 3271145444) auto-resolves via this merge — main's #326 rewrite supersedes the branch's pre-#326 wording at that location ("ships per-request span telemetry… OTLP/HTTP-compatible backend…" replaces "Use this concept when documenting…"). No separate edit needed; the merge IS the fix.

…ickstart-polish Resolve PR #344's lingering mergeable: dirty state by linking the branch history to origin/main (2c1d485 = post-#326 / #348 / #330 / #341 / #343 / #345 / #346). The squash-merge commit landed earlier (e2af197) integrated main's content into the branch tree but did not link the histories, so GitHub's mergeable computation still saw the 3-way-merge-base artifact conflict on docs/quickstart/self-hosted.md (a vs A + the glossary link / "In another terminal" vs "Keep the gateway running" framing). This explicit merge commit ties the branch to main's history. Self-hosted.md conflict resolved by taking OUR side — the branch's edits already contain main's substantive changes (capital A, first-time-build paragraph) plus this PR's additions (glossary link, keep-running framing, YOUR_ADMIN_KEY note, config.yaml location anchor). The auto-merge of first-model-first-key-first-request.md duplicated the :::warning callout that was already integrated via the squash commit; removed the duplicate.

Copilot AI review requested due to automatic review settings May 18, 2026 15:17

Copilot started reviewing on behalf of moonming May 18, 2026 15:18 View session

Copilot AI reviewed May 18, 2026

View reviewed changes

moonming added 2 commits May 18, 2026 23:22

fix(schema): regenerate model.schema.json after Provider::Cohere doc-…

09e8f81

…comment update (#332)

Copilot AI review requested due to automatic review settings May 18, 2026 15:31

Copilot started reviewing on behalf of moonming May 18, 2026 15:32 View session

Copilot AI reviewed May 18, 2026

View reviewed changes

Comment thread crates/aisix-server/src/main.rs

Comment on lines +795 to +798

hub.register(

Provider::Cohere,

Arc::new(OpenAiBridge::new().with_name("cohere")),

);

moonming merged commit 9750da0 into main May 18, 2026
12 checks passed

jarvis9443 deleted the fix/cohere-chat-hub-register branch June 25, 2026 06:26

Navigation Menu

Uh oh!

feat(server): register Provider::Cohere on Hub for chat-compat (#332) - #341

feat(server): register Provider::Cohere on Hub for chat-compat (#332)#341
moonming merged 3 commits into
mainfrom
fix/cohere-chat-hub-register

moonming commented May 18, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

coderabbitai Bot commented May 18, 2026 •

edited

Loading

Walkthrough

Changes

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

moonming commented May 18, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Fix

OpenAiBridge — aisix-provider-openai/src/bridge.rs

build_hub — aisix-server/src/main.rs

NOT touched

Tests

References (per CLAUDE.md §7)

Test plan

Summary by CodeRabbit

Uh oh!

coderabbitai Bot commented May 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

moonming commented May 18, 2026 •

edited by coderabbitai Bot

Loading

`OpenAiBridge` — `aisix-provider-openai/src/bridge.rs`

`build_hub` — `aisix-server/src/main.rs`

coderabbitai Bot commented May 18, 2026 •

edited

Loading