Skip to content

refactor(llm): one model catalog, starting with the remote side - #1901

Merged
CREDO23 merged 22 commits into
MODSetter:devfrom
CREDO23:refactor/model-catalog
Sep 23, 2026
Merged

CREDO23 merged 22 commits into
MODSetter:devfrom
CREDO23:refactor/model-catalog

Conversation

@CREDO23

@CREDO23 CREDO23 commented Sep 23, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

  • A selection is keyed by model type (text_gen, image_gen, …), not a role, and one rule in selectable.py decides which model fills a slot for selection and every picker alike.
  • Remote models are classified offline from a reviewed models.dev manifest keyed provider/model: embedders stop reading as chat, video models are video_gen, and each provider says how it is reached.
  • A connection names the manifest provider it reaches, the remote catalog is served under /llm/catalog/remote, and the connection form picks providers from it instead of a hard-coded list.
  • Also on this branch: the plugins design now writes through the app's API and specifies how its surface grows, AGENTS.md lists the plugins tree, and comments and docstrings are intent only.

Steps 1–3 of docs/proposals/model-catalog.md; the local side follows.

🤖 Generated with Claude Code

High-level PR Summary

Refactors the LLM module to use a unified model catalog system keyed by model type (text_gen, image_gen, etc.) rather than roles. Remote models are now classified offline from a reviewed manifest that maps providers to models with their capabilities. A connection stores which manifest provider it reaches, enabling scoped lookups. The remote catalog is served via /llm/catalog/remote routes that return providers first, then models when a provider is opened. The connection form now picks providers from the manifest instead of using hard-coded presets. This implements steps 1-3 of the model-catalog proposal, with the local side to follow.

⏱️ Estimated Review Time: 1-3 hours

💡 Review Order Suggestion
Order File Path
1 docs/proposals/model-catalog.md
2 surfsense_local/backend/modules/llm/model_type.py
3 surfsense_local/backend/modules/llm/selectable.py
4 surfsense_local/backend/alembic/versions/0013_selection_by_model_type.py
5 surfsense_local/backend/alembic/versions/0014_connection_catalog_provider.py
6 surfsense_local/backend/modules/llm/models.py
7 surfsense_local/backend/modules/llm/catalog/remote/manifest/schema.py
8 surfsense_local/backend/modules/llm/catalog/remote/manifest/loader.py
9 surfsense_local/backend/modules/llm/catalog/remote/manifest/lookup.py
10 surfsense_local/backend/modules/llm/catalog/remote/classifier.py
11 surfsense_local/backend/modules/llm/catalog/remote/support.py
12 surfsense_local/backend/modules/llm/catalog/remote/not_text_gen.py
13 surfsense_local/backend/modules/llm/catalog/remote/rows.py
14 surfsense_local/backend/modules/llm/catalog/remote/catalog.py
15 surfsense_local/backend/modules/llm/catalog/remote/router.py
16 surfsense_local/backend/scripts/refresh_remote_manifest.py
17 surfsense_local/backend/scripts/remote_manifest/translate.py
18 surfsense_local/backend/scripts/remote_manifest/endpoints.py
19 surfsense_local/backend/modules/llm/connections/service.py
20 surfsense_local/backend/modules/llm/selection.py
21 surfsense_local/backend/modules/llm/resolution.py
22 surfsense_local/backend/modules/llm/router.py
23 surfsense_local/backend/modules/llm/schemas.py
24 surfsense_local/backend/modules/llm/connections/router.py
25 surfsense_local/backend/modules/artifacts/formats.py
26 surfsense_local/backend/modules/artifacts/service.py
27 surfsense_local/backend/worker/studio/job.py
28 surfsense_local/frontend/src/features/model-selection/model-type.ts
29 surfsense_local/frontend/src/features/model-selection/api.ts
30 surfsense_local/frontend/src/features/model-selection/connection-form.tsx
31 surfsense_local/frontend/src/features/model-selection/connection-card.tsx
32 surfsense_local/frontend/src/features/studio/api.ts
33 docs/architecture/connections.md
34 docs/architecture/data-model.md
35 docs/architecture/local-models/selection.md
36 docs/architecture/studio.md
37 AGENTS.md

Need help? Join our Discord

CREDO23 and others added 22 commits September 23, 2026 00:55
Plugin docs moved to docs/proposals/plugins on dev; resolve against the
loopback-API design here, keeping dev's per-host egress consent, trial
unlock and open questions. runtime/02-calls-and-results.md stays deleted.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… packaged manifests

Two catalogs sharing one vocabulary: a reviewed local GGUF manifest and a
models.dev-based remote manifest, each run through manifest → classifier →
support → catalog. Covers manifest shapes, the curation path, provider/model
identity, and the order of work.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…e lines

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…model fills a slot

ModelType replaces the two roles: selected_models is keyed by model_type
(migration 0013 renames generation to text_gen and image_generation to
image_gen), /llm/selection/{model_type} has a slot per type, and Studio
declares requires_model_types. A connection's model listing carries types
and selectable_for from selectable.py, so selection and every picker share
one rule, and an unknown model fills every slot. The database refuses a
local runtime in a slot it does not serve.

Step 1 of docs/proposals/model-catalog.md.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…fest

The remote manifest keeps models.dev's providers and the evidence the
classifier reads, one model per line, refreshed by hand with
scripts/refresh_remote_manifest.py behind a shrinkage guard. The classifier
and support move from taxonomy/ to catalog/remote/ and read the manifest.
Connections look a listed id up by maker, then by the types every provider
agrees on, so embedders no longer read as chat models and video models are
video_gen. The old capability snapshot and its script are gone.

Step 2 of docs/proposals/model-catalog.md.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The remote manifest gains connect per provider (ready, needs_url,
needs_account_details or unreachable, with the base URL, its origin, account
fields, key and loopback) and call per model when a model is served only on
/responses or through another protocol. A small reviewed table fills the
fixed URLs models.dev leaves to its SDKs and says why Bedrock and Vertex need
more than a key; it never overrides models.dev and never holds a local
server, whose URL is the user's.

Step 3a of docs/proposals/model-catalog.md.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
provider_connections gains catalog_provider (migration 0014): a remote
manifest provider id, or custom for anything the manifest does not list, which
every existing connection becomes. The connection API stores it and refuses an
id the manifest lacks, and discovery reads the named provider's entry first,
so the same model id means what the serving provider says it means.

The downgrade drops the column in place: a batch rebuild of the parent table
fires selected_models' ON DELETE CASCADE and deletes every remote selection.

Step 3b of docs/proposals/model-catalog.md.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
catalog/source.py gets its first reader, and catalog/remote/catalog.py turns
the manifest and the user's connections into rows: a provider summary with
counts per type, a provider's rows offline (to add a key for, or one row per
connection), and a connection's rows checked against its live listing. Each
row says whether it is available, not served, unchecked, could not be checked,
not connected or unusable, and an unusable model offers no slot. Pure
functions: whoever fetched a listing hands it in.

Step 3c of docs/proposals/model-catalog.md.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…m it

Three routes under /llm/catalog/remote: the providers with their type counts
and connections, one provider's rows offline, and a connection's rows checked
against its live listing, where a down endpoint leaves rows could_not_check
rather than failing. Deprecated models are hidden unless asked for.

The connection form picks a provider from the manifest instead of a
hard-coded list: a ready provider fills an editable URL, account details
build a templated one, an unreachable provider is listed but disabled with
its reason, a keyless one hides the key field, and a local or custom server's
URL is always typed. The shared Combobox gains disabled items.

Step 3d of docs/proposals/model-catalog.md.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…out to the code

Status moves to in-progress with the first implementation commits. The layout
now shows where the remote side actually lives: routes and schemas per side,
discovery kept in connections/, selectable.py at the llm root, and the refresh
script's parts. The old listing routes go in step 6 with their readers, and the
roadmap names the initiative.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@vercel

vercel Bot commented Sep 23, 2026

Copy link
Copy Markdown

@CREDO23 is attempting to deploy a commit to the Rohan Verma's projects Team on Vercel.

A member of the Team first needs to authorize it.

@coderabbitai

coderabbitai Bot commented Sep 23, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: 7cdd691e-5c58-49cf-8ee4-c67ec200b053

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@CREDO23
CREDO23 merged commit 2923cf7 into MODSetter:dev Sep 23, 2026
3 of 8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant