[AnishSarkar22]

What

• Replaces Ollama with llama-server running in router mode as an Electron sidecar; a pinned llama.cpp build is staged at package time and the release workflow smoke-tests llama-server instead of Ollama, on a pinned ubuntu-22.04.
• Adds the llamacpp provider (modules/llm/providers/llamacpp/): presets, model download, vision projector loading, thinking-block handling, capability detection, and a router client that lets the router own model loads rather than the backend racing it.
• Deletes the Ollama provider and its catalog, and drops the llmfit recommendation path; ModelStore and other now-unused protocol surface go with them.
• Adds modules/llm/gguf/: reads GGUF headers through llama.cpp's own key table and derives model shape from a local or remote source, with file-kind detection.
• Adds modules/llm/hardware/: probes ggml devices out of process, reports whether the runtime can reach the GPU at all, budgets unified memory as a single pool, and covers per-OS GPU discovery for darwin/linux/windows.
• Adds modules/llm/fit/: estimates KV cache (including sliding-window and latent/MLA caches, priced per layer), sizes compute buffers from the widest tensor a layer builds, itemises what a load allocates, counts offload in layers, and separates residency from refusal so a CPU-only machine is still judged fairly.
• Picks one KV cache precision that the UI badge and the loader both read, so the estimate shown and the load performed agree.
• Adds modules/llm/catalog/: a curated curated-models.json manifest at schema 4 carrying the widened model shape, Hugging Face–backed search with eligibility filtering and non-chat-model rejection, recommendation gating on speed, and rows priced at the precision the loader will actually choose.
• Gates installs on the GGUF file's own header via a single denylist, instead of a generated allowlist or repo-summary/name heuristics.
• Warns when a searched repo ships no chat template, and serves a GPU diagnosis that is warmed off the boot path.
• Splits the ollama_pull egress grant into model_download and model_* grants; Alembic revision 0012_llamacpp_provider migrates selected models and existing egress grants off Ollama.
• Chat: budgets the prompt from the model's real context window rather than a fixed number, prices history with the model's own tokenizer and caches /props, and budgets the first token separately from subsequent ones.
• Secrets: raises UnreadableSecretError instead of leaking InvalidToken, and chat classifies an unreadable stored key as a provider auth error.
• Frontend: model catalog shows fit badges, model search, the GPU diagnosis, and the chat-template warning; Ollama-era copy and tests renamed for the llama.cpp path.
• Adds a curated-model refresh script that authors entries from the header's tensor table, and ships the curated manifest from its catalog path in the PyInstaller bundle.
• Docs/plans: new phase 7 (07-llamacpp-runtime.md) and phase 8 (08-cuda-backend.md) specs — Vulkan ships everywhere now, CUDA deferred to phase 8 — plus updates to the umbrella plan, diagrams, data model, 05a/05c, install UX, and the local README.

Why

Fixes #

How to test

[vercel]

@AnishSarkar22 is attempting to deploy a commit to the Rohan Verma's projects Team on Vercel.

A member of the Team first needs to authorize it.

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: 74426afb-1847-4b33-8422-fc72dd792c8d

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}