Repository navigation
Description
The gap
docs/architecture/local-models/catalog.md lists this under Known gaps:
POST /llm/installsdoes not refuse a curated build that will not fit; only the screen's disabled Download does.
Why it matters
The whole catalog is built on the server deciding and the screen rendering — the doc says so twice, and every fit, badge and recommendation is computed in the backend for exactly that reason. This one refusal is the exception: the only thing standing between a user and a multi-gigabyte download of a model their machine cannot load is a disabled attribute in the renderer.
Anything that is not that button gets through. Onboarding's model step, a catalog the page fetched before the user unplugged an external GPU, a second window, or a retry of a job whose id is still warm — all of them reach POST /llm/installs, and the job runs to completion, writes an install record, and leaves a model that fails when it is chosen. The searched path already refuses this properly with a sentence; the curated path, the one the app actually recommends from, does not.
Where
surfsense_local/backend/modules/llm/catalog/local/engines/llamacpp/engine.py—check(), and itsif not plan.needs_check: return planfirst line. Everything below it, including theFitState.TOO_BIGrefusal and its sentence, is what a curated build skips.surfsense_local/backend/modules/llm/catalog/local/install/plan.py—InstallPlan.needs_check, false for a curated build because its header was read at refresh time. That is true and is not the same question as whether it fits here.surfsense_local/backend/modules/llm/catalog/local/service.py—resolve_install()builds the curated plan from_curated_ids, andcheck()is where the disk's own refusal already lives.refuse_without_room()is the pattern for a server-side refusal that the job turns into an error event.surfsense_local/backend/modules/llm/catalog/local/engines/llamacpp/pricing.py—price()is what the rows path already calls with the entry's committedshape. Reuse it; do not price a curated build a second way.
What done looks like
POST /llm/installsfor a curated build whose exact fit isTOO_BIGon this machine ends the job with the same kind of refusal the searched path gives, before any bytes move, and the screen keeps disabling the button as it does now.- The one real decision is plumbing:
LlamaCppEngine.check()receives aBuild, not theCuratedModelthat carriesshape, so either the plan learns the shape when it is resolved or the check moves where the manifest is in scope. Pick one and say why. - Delete the Known gaps line from
docs/architecture/local-models/catalog.mdin the same PR, and update Install ids or Install jobs there if the refusals they list change.
What not to change
Only a refusal blocks. A build that spills — PARTIAL, "Reduced speed" — installs with no confirmation and must keep doing so; the screen's own rule says the same. Refusing anything short of TOO_BIG would make a slower-but-working model unreachable.
How to verify
surfsense_local/backend/tests/unit/llm/catalog/local/test_service.py already builds a service over the shipped manifest with a budget it controls and asserts on InstallRefusedError, and test_install_jobs.py covers a job ending in an error event. The repo is TDD: write the failing test first, on a tiny budget against a real curated build.
cd surfsense_local/backend
uv run pytest -m unit
uv run pytest -m integration
uv run ruff check .Before you start
Read CONTRIBUTING.md, the local model catalog doc and the fit doc, which owns what each FitState means. Open your PR against dev, not main. Comment here to claim this.
Activity
ybai08 commentedon Sep 29, 2026
I'd like to take this one.
Plan: a failing test first in
test_service.py, a curated build resolved on a tiny budget, assertingInstallRefusedErrorbefore any bytes move, plus a job-level case intest_install_jobs.pyending in the error event. Then the plumbing: the curated plan learns its entry's committedshapewhenresolve_install()builds it, socheck()can price it through the sameprice()the rows path uses and refuse onlyTOO_BIG; anything that spills still installs with no confirmation. Known gaps line deleted and the install sections updated in the same PR, againstdev.Cedric921 commentedon Sep 29, 2026
I'd like to take this one — PR is up: #2055. A curated id now resolves with its manifest entry's shape, and llama.cpp's
check()prices it with the row's ownprice()and refusesTOO_BIG; spills still install. Why the plan learns the shape rather than the check moving is in the PR.fix(local): refuse a curated install that will not fit before any byt…
MODSetter commentedon Sep 29, 2026
Fixed by #2056,
fix(local): refuse a curated install that will not fit before any bytes move, by @ybai08. Merged intodev.Closing by hand rather than by keyword:
Fixes #1976only auto-closes on a merge to the default branch, and this repo merges throughdev(CONTRIBUTING), so this would have stayed open and claimable while the work was already done. It reachesmainwith the next release.Nothing left to pick up here.