feat(llm): rework the local model catalog around pinned builds and one row shape by AnishSarkar22 · Pull Request #1902 · MODSetter/SurfSense

AnishSarkar22 · 2026-09-23T18:08:45Z

What

Speeds up GGUF header parsing: arrays past 4096 items keep only their length instead of being decoded, cutting a Qwen3 header's parse time from 3.5 s to 0.05 s.
Groups a repo's file listing into local builds, keeping the UD- prefix on quantization labels, listing every split part, and skipping imatrix files, drafters, big-endian builds and incomplete splits.
Adds the rule for a model's default build (first in the quantization preference order) and its recommended build (the default if it runs well here, else the largest smaller build that does, never a larger one).
Classifies local models into model types instead of a chat-or-refuse denylist: diffusion is image_gen, video is video_gen, speech output is audio_gen, embedders and projectors are known but typeless, and anything unnamed is text.
Reads local model support from GGUF header keys: a build reads images when its projector's header has a vision encoder whose width matches the model, and tools/reasoning come from the chat template.
Adds the pinned local manifest: seven models, most preferred first, including Gemma 3 4B with vision, every build in the preference order pinned with its revision, size and sha256.
Adds the local manifest refresh script, which reads each repo at a pinned commit, pins its builds, and reads the default build's header and projector over range requests, refusing unreadable builds, mismatched projectors, and unnamed losses.
Builds local catalog rows from the manifest and the models folder into one shared row shape for curated and downloaded models, pricing builds with their projector and keeping an install record of each model's projector pairing.
Opens a searched repo from its listing alone with no header read, showing exact sizes and an over-charging estimated fit; one build's header is read before download, and search recommends nothing.
Fixes local model typing and selection: llama.cpp claimed completion for every file on disk, letting a downloaded embedder fill the text_gen slot; models now carry types from their headers and selection uses the same rule as remote.
Adds the local catalog service and its routes: installs fetch a build's whole file set, pinned and hash-verified, after a searched build passes its exact check, and deleting a model removes its projector too.
Mounts the new routes at /llm/catalog/local and removes the old curated manifest, service, search, the not_chat gate and the lone-projector pairing.
Bundles the local manifest into the packaged build and drops the old curated refresh script.
Renders local models as rows of builds in the frontend: each row leads with its in-use, installed or recommended build, with the rest behind a disclosure.
Fixes the star and the fit badge disagreeing: a badge now warns only when there's something to warn about, and the recommendation never drops below four bits, so the star no longer lands on a spilling three-bit build next to a smaller model running at full speed.
Moves the decision of which build a row leads with onto the server (lead_build()), replacing a frontend heuristic; rows sort by the lead build's fit state.
Adds a --warning amber color token for the reduced-speed badge, replacing a plain outline.
Has a searched repo say whether it reads images from its file names alone (shared preferred_projector() rule), with no extra request, confirmed later by the header at install.
Shows install progress under the specific build that's downloading, via a shared BuildAction component used by curated and searched builds alike; renames "Reads images" to "Vision".
Names Hugging Face in the search copy instead of llama.cpp.
Updates the docs to describe the local model catalog as built and fixes links to the files it replaced.

Why

Fixes #

How to test

High-level PR Summary

This pull request implements a redesigned local model catalog for SurfSense's desktop app, consolidating curated, downloaded, and searched models into a unified row format with enhanced build selection logic. The new catalog supports multi-build recommendations (including a default and recommended build per model), vision model support (including projector pairing and verification), and improved type classification. The manifest now pins every build in a quantization preference order rather than just one, allowing the system to recommend smaller builds of preferred models when the default cannot run well. The catalog operates entirely offline for curated and downloaded models, with search as an opt-in network feature, and badges now function as warnings shown only when necessary rather than status indicators for every model.

⏱️ Estimated Review Time: 3+ hours

💡 Review Order Suggestion

Order	File Path
1	`docs/proposals/model-catalog.md`
2	`docs/adr/0014-two-tier-model-catalog.md`
3	`docs/adr/0026-curated-order-is-list-position.md`
4	`surfsense_local/backend/modules/llm/catalog/local/manifest/schema.py`
5	`surfsense_local/backend/modules/llm/catalog/local/builds.py`
6	`surfsense_local/backend/modules/llm/catalog/local/build_choice/preference.py`
7	`surfsense_local/backend/modules/llm/catalog/local/build_choice/default_build.py`
8	`surfsense_local/backend/modules/llm/catalog/local/build_choice/recommended_build.py`
9	`surfsense_local/backend/modules/llm/catalog/local/classifier.py`
10	`surfsense_local/backend/modules/llm/catalog/local/support.py`
11	`surfsense_local/backend/modules/llm/catalog/local/pricing.py`
12	`surfsense_local/backend/modules/llm/catalog/local/catalog.py`
13	`surfsense_local/backend/modules/llm/catalog/local/recommendation.py`
14	`surfsense_local/backend/modules/llm/catalog/local/lead_build.py`
15	`surfsense_local/backend/modules/llm/catalog/local/rows.py`
16	`surfsense_local/backend/modules/llm/catalog/local/downloaded.py`
17	`surfsense_local/backend/modules/llm/catalog/local/installs.py`
18	`surfsense_local/backend/modules/llm/catalog/local/search/hits.py`
19	`surfsense_local/backend/modules/llm/catalog/local/search/listing.py`
20	`surfsense_local/backend/modules/llm/catalog/local/search/repo_row.py`
21	`surfsense_local/backend/modules/llm/catalog/local/search/exact_check.py`
22	`surfsense_local/backend/modules/llm/catalog/local/search/tickets.py`
23	`surfsense_local/backend/modules/llm/catalog/local/service.py`
24	`surfsense_local/backend/modules/llm/catalog/local/router.py`
25	`surfsense_local/backend/modules/llm/catalog/local/schemas.py`
26	`surfsense_local/backend/scripts/local_manifest/entries.py`
27	`surfsense_local/backend/scripts/local_manifest/hub.py`
28	`surfsense_local/backend/scripts/local_manifest/recorded.py`
29	`surfsense_local/backend/scripts/local_manifest/assemble.py`
30	`surfsense_local/backend/scripts/local_manifest/guard.py`
31	`surfsense_local/backend/scripts/refresh_local_manifest.py`
32	`surfsense_local/backend/modules/llm/fit/copy.py`
33	`surfsense_local/backend/modules/llm/gguf/header_prefix.py`
34	`surfsense_local/backend/modules/llm/providers/llamacpp/provider.py`
35	`surfsense_local/backend/modules/llm/router.py`
36	`surfsense_local/backend/modules/llm/selection.py`
37	`surfsense_local/frontend/src/features/model-catalog/api.ts`
38	`surfsense_local/frontend/src/features/model-catalog/build-action.tsx`
39	`surfsense_local/frontend/src/features/model-catalog/fit-badge.tsx`
40	`surfsense_local/frontend/src/features/model-catalog/model-card.tsx`
41	`surfsense_local/frontend/src/features/model-catalog/model-search.tsx`
42	`surfsense_local/frontend/src/features/model-catalog/model-catalog-page.tsx`
43	`surfsense_local/frontend/src/features/model-catalog/use-model-catalog.ts`

The vocabulary walk cost 3.4 s of a 3.5 s parse. Arrays past 4096 items keep only their length, so a Qwen3 header parses in 0.05 s with n_vocab unchanged.

Quantization labels keep the UD- prefix. A build lists every split part and the preferred projector. Imatrix files, drafters, big-endian builds and incomplete splits are skipped.

The default is the first build in the quantization preference order. The recommendation is that default when it runs well here, else the largest smaller build that does, never a larger one.

Replaces the chat-or-refuse denylist with groups that each answer a type: diffusion is image_gen, video is video_gen, speech out is audio_gen, and embedders and projectors are known and typeless. Anything unnamed is text.

A build reads images when its projector's header carries a vision encoder and its width matches the model. Tools and reasoning come from the chat template.

Seven models, most preferred first, including Gemma 3 4B with vision. Every build in the preference order is pinned, each file with its revision, size and sha256, and projectors carry their GGUF keys.

Reads each repo at a pinned commit, pins its builds, and reads the default build's header and projector over range requests. It refuses unreadable builds, mismatched projectors and losses nobody named.

One row shape for curated and downloaded models. Builds are priced with their projector, the star goes to the first model with a recommended build, and an install record keeps each model's projector pairing.

No header is read to draw a repo's builds: sizes are exact and fit is an estimate that over-charges. One build's headers are read before download. Search recommends nothing.

llama.cpp claimed completion for every file on disk, so a downloaded embedder could fill the text_gen slot. Models now carry types from their headers, and local selection uses the same rule as remote.

Installs fetch a build's whole file set, pinned and hash-verified, after a searched build passes its exact check. Deleting a model removes its projector too, and the preset pairs only a recorded projector.

Mounts the new routes and removes the old curated manifest, service, search, the not_chat gate and the lone-projector pairing. The gate audit reads the new classifier.

…ript

Each curated row leads with its in-use, installed or recommended build, with the rest behind a disclosure. "Reads images" comes from one field on every row, estimated fits show ~, and model filters use selectable_for.

A badge now warns only when there is something to warn about: a build that runs fully, or spills only a little, carries none, since those are exactly the tiers a build can be recommended at. Moderate and heavy spill, and won't fit, still warn. The recommendation also stops below four bits, since a three or two bit build of a larger model is not clearly better than a four bit build of a smaller one; the star now moves to the smaller model instead. Together these fix the screenshot case: an 8 GB Mac starred Qwen3 8B at a spilling three bit build next to Qwen3 4B running at full speed. It now stars Qwen3 4B, with no warning beside the star.

Which build a curated row shows and its Download fetches now comes from the backend, not a frontend heuristic. lead_build() picks the build in use, else the installed one, else the recommendation, else the largest build at four bits or above that installs, else the default to say the model will not fit, and the reason travels with it as row.lead. Rows sort by the lead build's fit state, so a row's position matches what it actually shows. The frontend's lead-build.ts is removed; the card looks the choice up among the builds the server already sent.

Adds a --warning token, amber and distinct from --destructive, contrast checked at 4.5:1 for text on its own tint in both themes. The notice level fit badge (Reduced speed) now uses it instead of a plain outline, so a build that runs slower reads as a caution rather than a refusal.

preferred_projector() is now shared by builds_in() and search: the one rule that decides whether a repo ships a vision projector, so a search hit and its opened builds can never disagree about it. A hit's siblings, which the listing already carries, are enough to answer reads_images with no extra request; opening the repo and installing a build still confirm it against the projector's own header. The "Reads images" badge moves from outline to secondary everywhere it appears, and now shows on a search hit as well as on its builds.

…ding BuildAction moves into its own file, shared by a curated row, its other builds and a searched repo's builds. Each build now carries its own progress bar rather than one bar for the whole row, so a download on an other build is visible without collapsing the list, and the lead build's row stays expanded while one of its other builds is in flight. Search gains the same Download/Use, phase and progress bar every curated build already had. "Reads images" is renamed to "Vision" everywhere it appears.

….cpp

…replaced files

vercel · 2026-09-23T18:08:50Z

@AnishSarkar22 is attempting to deploy a commit to the Rohan Verma's projects Team on Vercel.

A member of the Team first needs to authorize it.

coderabbitai · 2026-09-23T18:08:53Z

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: 580b26a0-c973-4cc5-8991-04380e49ed9a

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

Check the architecture pages the catalog PRs (#1899-#1902) touched against the code, correct what no longer matches, and add the Known gaps found on the way. Fix the model catalog proposal's lines about the present, note its revisions on ADRs 0011 and 0015 and in the ADR index, update the SEO fact table for the new manifest, and mark the plugins tree in AGENTS.md as planned until it lands. Co-authored-by: Cursor <cursoragent@cursor.com>

AnishSarkar22 added 21 commits September 23, 2026 22:12

perf(gguf): count long header arrays instead of decoding them

ccedfff

The vocabulary walk cost 3.4 s of a 3.5 s parse. Arrays past 4096 items keep only their length, so a Qwen3 header parses in 0.05 s with n_vocab unchanged.

feat(llm): group a repo listing into local builds

6fa156a

Quantization labels keep the UD- prefix. A build lists every split part and the preferred projector. Imatrix files, drafters, big-endian builds and incomplete splits are skipped.

feat(llm): choose a model's default and recommended build

ec969f7

The default is the first build in the quantization preference order. The recommendation is that default when it runs well here, else the largest smaller build that does, never a larger one.

feat(llm): classify local models into model types

196ae9a

Replaces the chat-or-refuse denylist with groups that each answer a type: diffusion is image_gen, video is video_gen, speech out is audio_gen, and embedders and projectors are known and typeless. Anything unnamed is text.

feat(llm): read local model support from GGUF keys

786e0bd

A build reads images when its projector's header carries a vision encoder and its width matches the model. Tools and reasoning come from the chat template.

feat(llm): add the pinned local manifest

5c0133a

Seven models, most preferred first, including Gemma 3 4B with vision. Every build in the preference order is pinned, each file with its revision, size and sha256, and projectors carry their GGUF keys.

feat(llm): add the local manifest refresh script

84bd5e6

Reads each repo at a pinned commit, pins its builds, and reads the default build's header and projector over range requests. It refuses unreadable builds, mismatched projectors and losses nobody named.

feat(llm): build local rows from the manifest and the models folder

06bfcbd

One row shape for curated and downloaded models. Builds are priced with their projector, the star goes to the first model with a recommended build, and an install record keeps each model's projector pairing.

feat(llm): open a searched repo from its listing alone

cb90419

No header is read to draw a repo's builds: sizes are exact and fit is an estimate that over-charges. One build's headers are read before download. Search recommends nothing.

fix(llm): type local models and select them with selectable_for

f31a871

llama.cpp claimed completion for every file on disk, so a downloaded embedder could fill the text_gen slot. Models now carry types from their headers, and local selection uses the same rule as remote.

feat(llm): add the local catalog service and routes

fda441f

Installs fetch a build's whole file set, pinned and hash-verified, after a searched build passes its exact check. Deleting a model removes its projector too, and the preset pairs only a recorded projector.

refactor(llm): serve the local catalog at /llm/catalog/local

070ab48

Mounts the new routes and removes the old curated manifest, service, search, the not_chat gate and the lone-projector pairing. The gate audit reads the new classifier.

build(llm): bundle the local manifest and drop the curated refresh sc…

dd5c5cc

…ript

feat(local-ui): render local models as rows of builds

edd62da

Each curated row leads with its in-use, installed or recommended build, with the rest behind a disclosure. "Reads images" comes from one field on every row, estimated fits show ~, and model filters use selectable_for.

docs(local-ui): name Hugging Face in the search copy instead of llama…

e1890c6

….cpp

docs: describe the local model catalog as built and fix links to its …

de3a504

…replaced files

AnishSarkar22 requested review from CREDO23 and MODSetter as code owners September 23, 2026 18:08

AnishSarkar22 changed the title ~~Feat/local models catalog~~ feat(llm): rework the local model catalog around pinned builds and one row shape Sep 23, 2026

AnishSarkar22 merged commit e7b119c into MODSetter:dev Sep 23, 2026
3 of 8 checks passed

CREDO23 mentioned this pull request Sep 26, 2026

[Local|Release] Release desktop 2.0.3 #1932

Merged

Navigation Menu

Repository navigation

feat(llm): rework the local model catalog around pinned builds and one row shape - #1902

feat(llm): rework the local model catalog around pinned builds and one row shape#1902
AnishSarkar22 merged 21 commits into
MODSetter:devfrom
AnishSarkar22:feat/local-models-catalog

AnishSarkar22 commented Sep 23, 2026 •

edited

Loading

Uh oh!

vercel Bot commented Sep 23, 2026

Uh oh!

coderabbitai Bot commented Sep 23, 2026

Review skipped

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

AnishSarkar22 commented Sep 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

Why

How to test

High-level PR Summary

Uh oh!

vercel Bot commented Sep 23, 2026

Uh oh!

coderabbitai Bot commented Sep 23, 2026

Review skipped

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

AnishSarkar22 commented Sep 23, 2026 •

edited

Loading