Repository navigation
fix(local): refuse a curated install that will not fit before any bytes move - #2056
Conversation
…es move POST /llm/installs accepted a curated build whose exact fit was TOO_BIG here: the only thing in the way was the screen's disabled Download, and onboarding, a stale catalog, a second window or a warm job id reached the route past it. The job then downloaded gigabytes, wrote a record, and left a model that fails when chosen. The plan learns its entry's committed shape when resolve_install() builds it, where the manifest is in scope, and LlamaCppEngine.check() prices a curated build through the same price() its row uses. Only TOO_BIG refuses, with the sentence the searched path gives; a build that spills installs with no confirmation, as the screen's rule says. The searched path is unchanged. Fixes MODSetter#1976
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Advanced Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
@ybai08 is attempting to deploy a commit to the Rohan Verma's projects Team on Vercel. A member of the Team first needs to authorize it. |
What
InstallPlangainsshape: ModelShape | None.resolve_install()fills it for a curated build from the manifest, keyed the same way the id was minted (weights repo and path), so the plan carries the entry's committed shape without the engine seam changing.LlamaCppEngine.check()no longer returns a curated plan untouched. It prices it through the sameprice()the rows path uses, with the entry's shape, the build's weights bytes and its projector bytes, against the capacity budget, and raisesInstallRefusedErroronly forFitState.TOO_BIG, with the sentence the searched path already gives (now one_TOO_BIGconstant for both). A build that spills,PARTIAL, still installs with no confirmation, as the screen's own rule says. The searched path is unchanged.install_steps()already turns the refusal into the job'serrorevent beforeinstall()runs, so the job ends "the same kind of refusal the searched path gives" and no bytes move. The screen keeps disabling the button as it does now.catalog.md: theerrorevent row and the Install jobs prose describe the refusal; the Known gaps line is deleted.The plumbing decision. The plan learns the shape when it is resolved, rather than moving the check to where the manifest is in scope.
check()is the engine's seam for "what may install here", and the disk's refusal already runs beside it inservice.check(); giving the plan the one fact it lacked keeps that seam and every caller as they are. The alternative would have pulled fit pricing into the service or the route, which know nothing else about llama.cpp.Why
The whole catalog is built on the server deciding and the screen rendering, and this refusal was the exception: onboarding's model step, a catalog fetched before an external GPU was unplugged, a second window, or a retry of a warm job id all reached
POST /llm/installspast the disabled button, downloaded gigabytes, wrote an install record, and left a model that fails when chosen.Fixes #1976
How to test
Tests first, in
test_service.py, over the shipped manifest with the budget controlled throughavailable_bytes:test_a_curated_build_that_will_not_fit_is_refused_before_download: at 256 MiB, resolvingqwen3-32b's leading build the way the route does and callingservice.check()raisesInstallRefusedErrormatching "too big", and the faked hub was never asked for a byte. Failed before the change ("DID NOT RAISE").test_a_curated_build_that_merely_spills_installs_with_no_confirmation: at 64 GiB,qwen3-0.6bpassescheck()unchanged.test_install_steps.py(new file): a service whosecheck()refuses ends the job with exactly[{"type": "error", "message": ...}]and itsinstall()is never called. This one passed already; it pins the "before any bytes move" guarantee at the job level.Locally: 2,844 LLM unit tests pass (3,062 across the backend), 52 install and catalog integration tests pass, ruff clean, 102 docs checked.
High-level PR Summary
This PR prevents wasting bandwidth by refusing to download curated model builds that won't fit on the machine before any bytes are transferred. Previously, only the UI's disabled button prevented oversized downloads—users could bypass this through onboarding, external GPU changes, or job retries, resulting in multi-gigabyte downloads that produced unusable models. The fix adds shape information to
InstallPlanat resolution time and prices curated builds through the same capacity check that searched builds use, raisingInstallRefusedErrorfor builds that areTOO_BIGwhile still allowing builds that merely spill to VRAM to install (as they can run, albeit slower).⏱️ Estimated Review Time: 15-30 minutes
💡 Review Order Suggestion
surfsense_local/backend/modules/llm/catalog/local/install/plan.pysurfsense_local/backend/modules/llm/catalog/local/service.pysurfsense_local/backend/modules/llm/catalog/local/engines/llamacpp/engine.pysurfsense_local/backend/tests/unit/llm/catalog/local/test_install_steps.pysurfsense_local/backend/tests/unit/llm/catalog/local/test_service.pydocs/architecture/local-models/catalog.md