Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
60 changes: 60 additions & 0 deletions docs/agent-model-catalog.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
# Agent model catalog

`packages/shared/src/agent-models.ts` owns the curated launch models used by
agent settings, workspace creation, and host-side CLI validation.
`SUPERSET_CHAT_MODELS` in the same file supplies the cloud chat model list.
Model names are provider identifiers and are not translated.

## Verification: September 26, 2026

| Agent | Result | Evidence |
| --- | --- | --- |
| Claude | Opus 5.5, Fable 5.1, and Sonnet 5 were already present | [Anthropic model IDs](https://platform.claude.com/docs/en/models/overview) |
| Codex | GPT-6 Astra/Sol/Luna and their effort levels were already current | Live `codex app-server` → `model/list`; [OpenAI models](https://developers.openai.com/api/docs/models) |
| Cursor | Added Opus 5.5, Grok 4.7, and Muse Spark 1.3 with their effort variants | Live `cursor-agent --list-models` |
| Copilot | Added current Claude, GPT-6, GPT-5.6, Gemini, Grok, and Kimi choices, plus Auto | [GitHub supported models](https://docs.github.com/en/copilot/reference/ai-models/supported-models), `copilot --help`, and the `github-copilot` catalog in [models.dev](https://models.dev/api.json) |
| Gemini | Added Gemini 3.8 Flash and Gemini 3.1 Pro Preview | [Google model guidance](https://ai.google.dev/gemini-api/docs/latest-model), [CLI model selection](https://geminicli.com/docs/cli/model/), and the installed CLI's bundled 3.1 Pro identifier |
| OpenCode | Added Opus 5.5, Sonnet 5/4.6, and Gemini 3.8 Flash; GPT-6 already present | `opencode models` for Google/OpenAI; [models.dev](https://models.dev/api.json) for Anthropic |
| OMP | Added Opus 5.5 and Sonnet 5; GPT-6 already present | Published `@oh-my-pi/pi-catalog` 18.3.4 `src/models.json` |
| Vibe | Existing Medium 3.5 and Devstral Small choices remain current | [Mistral Vibe configuration](https://docs.mistral.ai/vibe/code/cli/configuration) |
| Polygraph | Harness choices remain Claude, Codex, and OpenCode | This picker chooses a harness, not an underlying model |
| Superset chat | Added Sonnet 5; Opus 5.5 and GPT-6 already present | Anthropic model IDs above |

Agents without a curated model picker continue to use their own configured
model selection. This audit does not add launch flags to those agents.

Copilot uses dotted Claude release IDs (`claude-opus-5.5`,
`claude-fable-5.1`). Anthropic uses dashed IDs (`claude-opus-5-5`,
`claude-fable-5-1`). Cursor encodes effort in the model ID: Opus 5.5's
base picker choice is `claude-opus-5-5-medium`. Its live catalog did not expose
GPT-6, so GPT-6 was not added to Cursor.

Validation covers catalog discovery, exact launch arguments, supported effort
variants, host validation, shared-package types, and strict locale compilation.
It does not establish successful inference on every provider or account.
Available models still depend on account access, provider configuration, and
installed CLI version. Existing saved choices and default model behavior are
preserved.

## Live smoke tests: September 27, 2026

Checked 33 agent/model configurations in empty temporary directories using:
`Reply with exactly MODEL_SMOKE_OK. Do not use tools, read files, or change anything.`
Each subprocess had a 75-second timeout. No project files were supplied.

| Agent | Models checked | Live result |
| --- | --- | --- |
| Claude | Opus 5.5, Sonnet 5 | Both returned `MODEL_SMOKE_OK`; successful result metadata confirmed the requested model |
| Codex | GPT-6 Astra, Sol, Luna | All returned `MODEL_SMOKE_OK` and completed their turn successfully |
| Copilot | All 17 added choices, including Auto | All rejected by account policy before inference: `Access denied by policy settings` |
| Cursor | Opus 5.5, Grok 4.7, Muse Spark 1.3 | All rejected: `Named models unavailable`; this account's free plan only permits Auto |
| Gemini | Gemini 3.8 Flash, Gemini 3.1 Pro Preview | Both rejected with `IneligibleTierError` / `UNSUPPORTED_CLIENT`; retried with `--skip-trust` to eliminate the temporary-directory trust error |
| OpenCode | Opus 5.5, Sonnet 5, Sonnet 4.6 | All failed with a generic server error; a diagnostic retry of Opus 5.5 exposed `ProviderModelNotFoundError` for `anthropic/claude-opus-5-5`. Anthropic OAuth credentials exist, but Anthropic is absent from this installation's active `opencode models` listing |
| OpenCode | Gemini 3.8 Flash | Rejected with `ProviderAuthError`: Google Generative AI API key missing |
| OMP | Opus 5.5, Sonnet 5 | Not run: `omp` is not installed on this machine |

Five configurations passed inference; 26 failed before a successful response;
two could not run. Account policy, subscriptions, credentials, and installed
agents were not changed. Sonnet 5 passed through Claude's CLI; this does not
constitute an end-to-end Superset chat UI test. No model is marked working
solely because it appears in a provider catalog or passes a unit test.
68 changes: 68 additions & 0 deletions packages/shared/src/agent-models.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -592,3 +592,71 @@ describe("buildAgentModelEnv (vibe)", () => {
]);
});
});

describe("September 2026 model launches", () => {
it.each([
["claude", "claude-opus-5-5"],
["codex", "gpt-6-astra"],
["codex", "gpt-6-sol"],
["codex", "gpt-6-luna"],
["copilot", "claude-opus-5.5"],
["copilot", "claude-fable-5.1"],
["copilot", "gpt-6-astra"],
["copilot", "gpt-6-sol"],
["copilot", "gpt-6-luna"],
["gemini", "gemini-3.8-flash"],
["gemini", "gemini-3.1-pro-preview"],
["opencode", "anthropic/claude-opus-5-5"],
["opencode", "anthropic/claude-sonnet-5"],
["opencode", "google/gemini-3.8-flash"],
["omp", "anthropic/claude-opus-5-5"],
["omp", "anthropic/claude-sonnet-5"],
])("launches %s with its exact model id %s", (preset, model) => {
expect(isCuratedAgentModel(preset, model)).toBe(true);
expect(buildAgentModelArgs(preset, model)).toEqual(["--model", model]);
});

it("keeps Copilot's dotted ids separate from Anthropic's ids", () => {
expect(buildAgentModelArgs("copilot", "claude-opus-5-5")).toEqual([]);
expect(buildAgentModelArgs("claude", "claude-opus-5.5")).toEqual([]);
});

it.each([
[
"claude-opus-5-5-medium",
"claude-opus-5-5",
["low", "medium", "high", "xhigh", "max"],
],
["grok-4.7-high", "grok-4.7", ["low", "medium", "high", "xhigh"]],
[
"muse-spark-1.3-high",
"muse-spark-1.3",
["minimal", "low", "medium", "high", "xhigh", "max"],
],
] as const)("launches every supported Cursor effort for %s", (model, family, levels) => {
expect(getAgentEfforts("cursor-agent", model).map(({ id }) => id)).toEqual([
...levels,
]);
for (const effort of levels) {
const variant = `${family}-${effort}`;
expect(buildAgentModelArgs("cursor-agent", model, effort)).toEqual([
"--model",
variant,
]);
expect(isCuratedAgentModel("cursor-agent", variant)).toBe(true);
}
});

it("offers Sonnet 5 in the cloud chat catalog", () => {
expect(
SUPERSET_CHAT_MODELS.find(({ id }) => id === "anthropic/claude-sonnet-5")
?.provider,
).toBe("Anthropic");
});

it("has unique ids within every agent catalog", () => {
for (const { models } of AGENT_MODEL_SUPPORT) {
expect(new Set(models.map(({ id }) => id)).size).toBe(models.length);
}
});
});
58 changes: 57 additions & 1 deletion packages/shared/src/agent-models.ts
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,7 @@ export const SUPERSET_CHAT_MODELS: readonly SupersetChatModel[] = [
provider: "Anthropic",
},
{ id: "anthropic/claude-fable-5", label: "Fable 5", provider: "Anthropic" },
{ id: "anthropic/claude-sonnet-5", label: "Sonnet 5", provider: "Anthropic" },
{
id: "anthropic/claude-sonnet-4-6",
label: "Sonnet 4.6",
Expand Down Expand Up @@ -160,6 +161,8 @@ export const AGENT_MODEL_SUPPORT: readonly AgentModelSupport[] = [
presetId: "gemini",
modelFlag: "--model",
models: [
{ id: "gemini-3.8-flash", label: "Gemini 3.8 Flash" },
{ id: "gemini-3.1-pro-preview", label: "Gemini 3.1 Pro Preview" },
{ id: "gemini-2.5-pro", label: "Gemini 2.5 Pro" },
{ id: "gemini-2.5-flash", label: "Gemini 2.5 Flash" },
],
Expand All @@ -168,6 +171,23 @@ export const AGENT_MODEL_SUPPORT: readonly AgentModelSupport[] = [
presetId: "copilot",
modelFlag: "--model",
models: [
{ id: "auto", label: "Auto" },
{ id: "claude-opus-5.5", label: "Claude Opus 5.5" },
{ id: "claude-opus-5", label: "Claude Opus 5" },
{ id: "claude-fable-5.1", label: "Claude Fable 5.1" },
{ id: "claude-sonnet-5", label: "Claude Sonnet 5" },
{ id: "claude-sonnet-4.6", label: "Claude Sonnet 4.6" },
{ id: "claude-haiku-4.5", label: "Claude Haiku 4.5" },
{ id: "gpt-6-astra", label: "GPT-6 Astra" },
{ id: "gpt-6-sol", label: "GPT-6 Sol" },
{ id: "gpt-6-luna", label: "GPT-6 Luna" },
{ id: "gpt-5.6-sol", label: "GPT-5.6 Sol" },
{ id: "gpt-5.6-terra", label: "GPT-5.6 Terra" },
{ id: "gpt-5.6-luna", label: "GPT-5.6 Luna" },
{ id: "gpt-5.3-codex", label: "Codex 5.3" },
{ id: "gemini-3.8-flash", label: "Gemini 3.8 Flash" },
{ id: "grok-4.7", label: "Grok 4.7" },
{ id: "kimi-k3", label: "Kimi K3" },
{ id: "claude-fable-5", label: "Claude Fable 5" },
{ id: "claude-sonnet-4.5", label: "Claude Sonnet 4.5" },
{ id: "gpt-5.1", label: "GPT-5.1" },
Expand All @@ -180,14 +200,19 @@ export const AGENT_MODEL_SUPPORT: readonly AgentModelSupport[] = [
// cursor-agent has no effort flag: every effort level is its own
// model id, and these are each family's default level. The effort
// picker swaps in the sibling id (`CURSOR_EFFORT_VARIANTS`). Ids
// verified against a live account's `--list-models` (2026-09-04);
// verified against a live account's `--list-models` (2026-09-26);
// the list is account-dependent and unknown ids are rejected by
// the CLI, not silently ignored. "auto" is the only id free-plan
// accounts can use (besides composer) — named models fail there
// with "Named models unavailable", so keep an explicit working
// choice in the picker.
{ id: "auto", label: "Auto" },
{ id: "composer-2.5", label: "Composer 2.5" },
{
id: "claude-opus-5-5-medium",
label: "Opus 5.5",
group: CURSOR_ANTHROPIC_GROUP,
},
{
id: "claude-fable-5-1-thinking-high",
label: "Fable 5.1",
Expand Down Expand Up @@ -251,6 +276,12 @@ export const AGENT_MODEL_SUPPORT: readonly AgentModelSupport[] = [
label: "Grok 4.6",
group: CURSOR_OTHER_GROUP,
},
{ id: "grok-4.7-high", label: "Grok 4.7", group: CURSOR_OTHER_GROUP },
{
id: "muse-spark-1.3-high",
label: "Muse Spark 1.3",
group: CURSOR_OTHER_GROUP,
},
{ id: "kimi-k3-max", label: "Kimi K3", group: CURSOR_OTHER_GROUP },
{ id: "glm-5.2-high", label: "GLM 5.2", group: CURSOR_OTHER_GROUP },
],
Expand All @@ -265,10 +296,14 @@ export const AGENT_MODEL_SUPPORT: readonly AgentModelSupport[] = [
// appear in that listing; `claude-fable-5-1` was checked against
// models.dev directly (2026-09-01), the gpt-6 family likewise
// (2026-09-23).
{ id: "anthropic/claude-opus-5-5", label: "Claude Opus 5.5" },
{ id: "anthropic/claude-sonnet-5", label: "Claude Sonnet 5" },
{ id: "anthropic/claude-opus-5", label: "Claude Opus 5" },
{ id: "anthropic/claude-fable-5-1", label: "Claude Fable 5.1" },
{ id: "anthropic/claude-fable-5", label: "Claude Fable 5" },
{ id: "anthropic/claude-sonnet-4-6", label: "Claude Sonnet 4.6" },
{ id: "anthropic/claude-sonnet-4-5", label: "Claude Sonnet 4.5" },
{ id: "google/gemini-3.8-flash", label: "Gemini 3.8 Flash" },
{ id: "openai/gpt-6-astra", label: "GPT-6 Astra" },
{ id: "openai/gpt-6-sol", label: "GPT-6 Sol" },
{ id: "openai/gpt-6-luna", label: "GPT-6 Luna" },
Expand All @@ -289,6 +324,8 @@ export const AGENT_MODEL_SUPPORT: readonly AgentModelSupport[] = [
{ id: "@smol", label: "Configured fast model" },
{ id: "@slow", label: "Configured slow model" },
{ id: "@plan", label: "Configured plan model" },
{ id: "anthropic/claude-opus-5-5", label: "Claude Opus 5.5" },
{ id: "anthropic/claude-sonnet-5", label: "Claude Sonnet 5" },
{ id: "anthropic/claude-opus-5", label: "Claude Opus 5" },
{ id: "anthropic/claude-fable-5-1", label: "Claude Fable 5.1" },
{ id: "anthropic/claude-fable-5", label: "Claude Fable 5" },
Expand Down Expand Up @@ -403,6 +440,24 @@ const CURSOR_GPT_LEVELS = [
const CURSOR_EFFORT_VARIANTS: Readonly<
Record<string, Readonly<Record<string, string>>>
> = {
"claude-opus-5-5-medium": cursorEffortVariants(
"claude-opus-5-5",
CURSOR_CLAUDE_LEVELS,
),
"grok-4.7-high": cursorEffortVariants("grok-4.7", [
"low",
"medium",
"high",
"xhigh",
]),
"muse-spark-1.3-high": cursorEffortVariants("muse-spark-1.3", [
"minimal",
"low",
"medium",
"high",
"xhigh",
"max",
]),
"claude-fable-5-1-thinking-high": cursorEffortVariants(
"claude-fable-5-1-thinking",
CURSOR_CLAUDE_LEVELS,
Expand Down Expand Up @@ -579,6 +634,7 @@ export const AGENT_EFFORT_SUPPORT: readonly AgentEffortSupport[] = [
effortFlag: null,
efforts: [
{ id: "none", label: "None" },
{ id: "minimal", label: "Minimal" },
{ id: "low", label: "Low" },
{ id: "medium", label: "Medium" },
{ id: "high", label: "High" },
Expand Down
Loading