Skip to content

[Bug]: Orchestrator ignores toggled-off models: orchestrator_capabilities lists them and delegate_task runs them #15716

Closed
Closed
[Bug]: Orchestrator ignores toggled-off models: orchestrator_capabilities lists them and delegate_task runs them#15716
Labels
bugSomething is broken or behaving incorrectly.via-triageFiled through npx t3 triage

Description

@AlexisGrrd
AlexisGrrd
opened on Oct 4, 2026

Before submitting

  • I searched existing issues and did not find a duplicate.
  • I included enough detail to reproduce or investigate the problem.

Area

apps/server

Steps to reproduce

  1. In Settings → Providers, toggle off one or more models for a provider (e.g. Cursor: leave only composer-2.5, grok-4.7, grok-4.6 on; turn off gpt-5.4-nano).
  2. Confirm they are stored as hidden in ~/.t3/userdata/client-settings.json → providerModelPreferences.cursor.hiddenModels, and that the composer model picker no longer shows them.
  3. From an agent thread, call the T3 MCP tool orchestrator_capabilities.
  4. Call delegate_task with target: { providerInstanceId: "cursor", model: "gpt-5.4-nano" }.

Expected behavior

A model toggled off in settings is not offered to or usable by orchestration, matching the composer:

  • orchestrator_capabilities omits hidden models (or at least marks them, e.g. hidden: true). Its own description says it uses "the same live catalog as the composer".
  • delegate_task rejects a hidden model with a clear error (optionally overridable with an explicit flag), instead of silently running it.

Actual behavior

  • orchestrator_capabilities returns every model the provider supports and ignores hiddenModels. For Cursor it listed 44 models when only 3 are enabled; for OpenCode it listed ~500 when 9 are enabled.
  • Agents pick from that list, so they choose models the user turned off. In my case an agent picked cursor/gpt-5.4-nano and opencode/big-pickle, both toggled off.
  • delegate_task accepted and ran the hidden model. The Cursor run then failed on an account usage limit for that model, surfaced in the thread only as Provider turn failed.

Likely cause: the toggle is persisted in client settings (client-settings.json → providerModelPreferences.*.hiddenModels), while the orchestrator builds its catalog server-side from settings.json, so it never sees the preference.

Impact

Major degradation or frequent failure

Delegation is supposed to be safe to hand to agents. Today a toggle-off has no effect on what agents use, which can mean unexpected cost, usage-limit failures, or models the user deliberately excluded.

Version or commit

T3 Code (Nightly) 0.0.46-nightly.20261004.2648

Environment

macOS (Darwin 25.5, arm64), desktop app. Parent thread: Claude Agent (claude-opus-5-5), runtime mode auto. Reproduced with Cursor (cursor-agent 2026.09.15) and OpenCode 1.18.34 providers.

Logs or stack traces

# orchestrator_capabilities vs. client-settings hiddenModels (summarized)
cursor    catalog: 44 models | enabled (not hidden): grok-4.7, grok-4.6, composer-2.5
opencode  catalog: 500 models | enabled (not hidden): 9 opencode-go/* models

# delegate_task target {providerInstanceId: "cursor", model: "gpt-5.4-nano"} -> accepted, ran
# provider log, run.completed:
"status": "error", "error": {"message": "You've hit your usage limit ... Switch to a different model ..."}, "model": {"id": "gpt-5.4-nano"}
# thread surface: "Provider turn failed."

Workaround

Before delegating, have the agent read ~/.t3/userdata/client-settings.json → providerModelPreferences.<providerInstanceId>.hiddenModels and exclude those models itself.

Related: #11031 (Android thread settings show disabled models and select one as default). Same underlying gap: hidden or disabled model preferences aren't honored outside the desktop composer.

Activity

  1. juliusmarminge commented on Oct 4, 2026

    @juliusmarminge
    juliusmarminge
    on Oct 4, 2026
    Member

    Note

    Grok responding on behalf of Julius.

    Triage

    Thanks for the detailed counts and the provider log, @AlexisGrrd. They made the mismatch easy to confirm.

    What I found

    Confirmed on current main. orchestrator_capabilities and delegate_task read the server's provider catalog, and they never see the hide toggles under Settings → Providers.

    • Those toggles are a per-client picker filter ("Show {model} in the model picker" in ProviderModelsSection.tsx). They're stored in ClientSettings.providerModelPreferences.<instance>.hiddenModels, and the composer applies them in applyInstanceModelPreferences (apps/web/src/modelSelection.ts).
    • Where they're stored depends on the client. Desktop writes them to userdata/client-settings.json over IPC, the browser keeps them in localStorage, and mobile has its own list. apps/server doesn't read ClientSettings.
    • OrchestratorMcpService.capabilities maps every model from providerRegistry.getProviders. delegate_task (via resolveTarget, which create_threads also uses) rejects a model only when the provider no longer advertises it. That explains why you saw 44 Cursor models and about 500 OpenCode models.
    • The tool descriptions in apps/server/src/mcp/toolkits/orchestrator/tools.ts say they use "the same live catalog as the composer." Agents read that as matching the picker, but the list they get is unfiltered.
    • "Provider turn failed." is the generic fallback in ProviderFailure.ts for unknown provider errors. It's separate from this issue.

    Related but not duplicates: #11031 (mobile thread settings ignore desktop hiddenModels) and closed #9350 (desktop hide doesn't change the environment catalog).

    Likely fix area

    There are a few options, and choosing between them is a product decision:

    • Correct the MCP tool text so it describes the provider catalog, not the picker.
    • Define an environment-level model policy on the server that orchestration respects. This would mean deciding which client's hide list it starts from.

    A maintainer will decide on the fix direction.

  2. juliusmarminge
    added
    bugSomething is broken or behaving incorrectly.
    via-triageFiled through npx t3 triage
    on Oct 4, 2026

  3. nkoynov commented on Oct 5, 2026

    @nkoynov
    nkoynov
    on Oct 5, 2026
    Contributor

    The unfiltered catalog also has a size cost. Some numbers from one setup: Claude (12 models), Cursor (44), and OpenCode 2 with cursor-opencode-provider (69).

    • orchestrator_capabilities returns 61,128 bytes of JSON on a single line. OpenCode accounts for 38 KB, Cursor 15 KB and Claude 6 KB, which works out to roughly 170–550 bytes per model. With the ~500 OpenCode models in this report, it would be roughly 85–275 KB (my estimate).
    • OpenCode 2 keeps a tool result inline only up to 50 KiB. Above that, the model gets [showing 0 lines of 1; full output saved to …/tool-output/tool_…], and the read tool truncated the file's one 61 KB line.
    • I ran the same prompt twice ("call orchestrator_capabilities and report the provider instance ids", Opus 5.5 under OpenCode 2, nightly 2648):

    So filtering out hidden models, or a compact listing (ids and option ids, with details on request), would also keep the result under these limits.

    Model: Claude Opus 5.5 (1M). Harness: Claude Code in T3 Code.

  4. newink commented on Oct 8, 2026

    @newink
    newink
    on Oct 8, 2026

    Another data point, from the Pi provider this time.

    • Pi advertises 548 models. 544 of them are in client-settings.json → providerModelPreferences.pi.hiddenModels, which leaves 4 visible in the picker.
    • orchestrator_capabilities returns 159,261 characters. Pi accounts for 147,772 of them, and the other six providers together for about 11K.
    • Under Claude Code the result goes over the MCP tool output limit and gets written to a file. The agent has to fall back to jq, which takes a few extra calls before it can delegate anything.
    • Delegation still works. A delegate_task to codex gpt-5.6-luna completed fine.

    With hiddenModels applied, this response would be around 12K and fit inline.

    T3 Code (Nightly) 0.0.46-nightly.20261005.2702, macOS arm64.

    Model: Claude Opus 5.5 (1M). Harness: Claude Code in T3 Code.

  5. t3dotgg commented on Oct 21, 2026

    @t3dotgg
    t3dotgg
    on Oct 21, 2026
    Member

    Note

    🤖 Claude Opus 5.5 responding on behalf of Theo

    Thanks for the report. One part of this is fixed on main in #16816. Please test on the nightly built from main on 2026-10-21, or on the next stable release. One part is still open: delegate_task can still run a hidden model when its ID is supplied directly. If that part still matters to you, please open a new issue for that part only.

  6. t3dotgg
    closed this as completedon Oct 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething is broken or behaving incorrectly.via-triageFiled through npx t3 triage

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions