Skip to content

fix(local): refuse a curated install that will not fit before any bytes move - #2056

Merged
MODSetter merged 1 commit into
MODSetter:devfrom
ybai08:fix/refuse-curated-install-too-big
Sep 29, 2026
Merged

MODSetter merged 1 commit into
MODSetter:devfrom
ybai08:fix/refuse-curated-install-too-big

Conversation

@ybai08

@ybai08 ybai08 commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

What

  • InstallPlan gains shape: ModelShape | None. resolve_install() fills it for a curated build from the manifest, keyed the same way the id was minted (weights repo and path), so the plan carries the entry's committed shape without the engine seam changing.
  • LlamaCppEngine.check() no longer returns a curated plan untouched. It prices it through the same price() the rows path uses, with the entry's shape, the build's weights bytes and its projector bytes, against the capacity budget, and raises InstallRefusedError only for FitState.TOO_BIG, with the sentence the searched path already gives (now one _TOO_BIG constant for both). A build that spills, PARTIAL, still installs with no confirmation, as the screen's own rule says. The searched path is unchanged.
  • install_steps() already turns the refusal into the job's error event before install() runs, so the job ends "the same kind of refusal the searched path gives" and no bytes move. The screen keeps disabling the button as it does now.
  • catalog.md: the error event row and the Install jobs prose describe the refusal; the Known gaps line is deleted.

The plumbing decision. The plan learns the shape when it is resolved, rather than moving the check to where the manifest is in scope. check() is the engine's seam for "what may install here", and the disk's refusal already runs beside it in service.check(); giving the plan the one fact it lacked keeps that seam and every caller as they are. The alternative would have pulled fit pricing into the service or the route, which know nothing else about llama.cpp.

Why

The whole catalog is built on the server deciding and the screen rendering, and this refusal was the exception: onboarding's model step, a catalog fetched before an external GPU was unplugged, a second window, or a retry of a warm job id all reached POST /llm/installs past the disabled button, downloaded gigabytes, wrote an install record, and left a model that fails when chosen.

Fixes #1976

How to test

Tests first, in test_service.py, over the shipped manifest with the budget controlled through available_bytes:

  • test_a_curated_build_that_will_not_fit_is_refused_before_download: at 256 MiB, resolving qwen3-32b's leading build the way the route does and calling service.check() raises InstallRefusedError matching "too big", and the faked hub was never asked for a byte. Failed before the change ("DID NOT RAISE").
  • test_a_curated_build_that_merely_spills_installs_with_no_confirmation: at 64 GiB, qwen3-0.6b passes check() unchanged.
  • test_install_steps.py (new file): a service whose check() refuses ends the job with exactly [{"type": "error", "message": ...}] and its install() is never called. This one passed already; it pins the "before any bytes move" guarantee at the job level.
cd surfsense_local/backend
uv run pytest -m unit
uv run pytest -m integration -k "install or catalog"
uv run ruff check .
cd ../.. && python scripts/check_docs.py

Locally: 2,844 LLM unit tests pass (3,062 across the backend), 52 install and catalog integration tests pass, ruff clean, 102 docs checked.

High-level PR Summary

This PR prevents wasting bandwidth by refusing to download curated model builds that won't fit on the machine before any bytes are transferred. Previously, only the UI's disabled button prevented oversized downloads—users could bypass this through onboarding, external GPU changes, or job retries, resulting in multi-gigabyte downloads that produced unusable models. The fix adds shape information to InstallPlan at resolution time and prices curated builds through the same capacity check that searched builds use, raising InstallRefusedError for builds that are TOO_BIG while still allowing builds that merely spill to VRAM to install (as they can run, albeit slower).

⏱️ Estimated Review Time: 15-30 minutes

💡 Review Order Suggestion
Order File Path
1 surfsense_local/backend/modules/llm/catalog/local/install/plan.py
2 surfsense_local/backend/modules/llm/catalog/local/service.py
3 surfsense_local/backend/modules/llm/catalog/local/engines/llamacpp/engine.py
4 surfsense_local/backend/tests/unit/llm/catalog/local/test_install_steps.py
5 surfsense_local/backend/tests/unit/llm/catalog/local/test_service.py
6 docs/architecture/local-models/catalog.md

Need help? Join our Discord

…es move

POST /llm/installs accepted a curated build whose exact fit was TOO_BIG
here: the only thing in the way was the screen's disabled Download, and
onboarding, a stale catalog, a second window or a warm job id reached
the route past it. The job then downloaded gigabytes, wrote a record,
and left a model that fails when chosen.

The plan learns its entry's committed shape when resolve_install()
builds it, where the manifest is in scope, and LlamaCppEngine.check()
prices a curated build through the same price() its row uses. Only
TOO_BIG refuses, with the sentence the searched path gives; a build
that spills installs with no confirmation, as the screen's rule says.
The searched path is unchanged.

Fixes MODSetter#1976
@coderabbitai

coderabbitai Bot commented Sep 29, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: c43a38ac-414d-4283-8371-1c4cfe32d380

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@vercel

vercel Bot commented Sep 29, 2026

Copy link
Copy Markdown

@ybai08 is attempting to deploy a commit to the Rohan Verma's projects Team on Vercel.

A member of the Team first needs to authorize it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants