Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/adr/0007-bundled-embeddings.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# ADR 0007: Embeddings come from a bundled bge-small model run in process on the CPU

- **Status:** Accepted; amended by [ADR 0036](0036-the-index-records-its-embedder.md): the width comes from the index's own record, and bge-small is the default rather than the only embedder
- **Status:** Accepted; amended by [ADR 0036](0036-the-index-records-its-embedder.md): the width comes from the index's own record, and bge-small is the default rather than the only embedder; the bundled file is FP16, not the int8 the Decision names (66,465,124 bytes, measured in [embedding](../architecture/embedding.md#choosing), [#1996](https://github.com/MODSetter/SurfSense/issues/1996))
- **Date:** 2026-09-04
- **Source:** [Umbrella plan L106](https://github.com/MODSetter/SurfSense/blob/431914fae066e0c42a38b1fdbbab64e8f92d3d00/plans/community-local/00-umbrella-plan.md#L106), [Data model L220–224](https://github.com/MODSetter/SurfSense/blob/431914fae066e0c42a38b1fdbbab64e8f92d3d00/plans/community-local/00c-data-model.md#L220-L224)

Expand Down
2 changes: 1 addition & 1 deletion docs/adr/0036-the-index-records-its-embedder.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,4 +27,4 @@ The [proposal](../proposals/embedding-model-choice.md) lets a new install choose
- A same-width replacement is now detected, because the row names the model, closing the gap ADR 0007 recorded.
- Locking is the one place outside migrations that emits DDL, safe only because it runs on an empty table.
- Workers wait until the API has migrated the database before taking a job. Electron starts them together, and a job queued before an update would otherwise run against the old schema.
- The bundled bge-small file can never change without a re-embed. Its FP16 weights stay, int8 or not ([search](../architecture/search.md), Known gaps).
- The bundled bge-small file can never change without a re-embed. Its FP16 weights stay, int8 or not ([embedding](../architecture/embedding.md#choosing)).
4 changes: 4 additions & 0 deletions docs/architecture/embedding.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,10 @@ Onboarding's last step lists the choices; finishing onboarding locks the one cho
| curated | the local manifest, run by onnxruntime ([catalog](local-models/catalog.md)); ranking weight measured by the retrieval eval | measured |
| Hugging Face | any ONNX embedder, found by search | declared, or inferred |

The bundled bge-small is `model_optimized.onnx` from `Qdrant/bge-small-en-v1.5-onnx-Q`. The repo is named for quantization, but the file stores its 33.2 million weights as FLOAT16 and is 66,465,124 bytes (66.5 MB). Measured for [#1996](https://github.com/MODSetter/SurfSense/issues/1996) on an Apple M6 with onnxruntime 1.29.0's CPU provider at default threads, through the app's own recipe (the bundled tokenizer, truncation at 512 tokens, CLS pooling, normalised, batches of 32), it embeds about 72 short passages a second (a median of 53 tokens each) and about 14.5 chunk-sized ones (512 tokens each).

An int8 export of the same model (`model_quantized.onnx` from `Xenova/bge-small-en-v1.5`, 34.0 MB) is half the size and embeds chunk-sized text 1.6 times as fast on that machine, but it is not the same vector space: against the bundled file its vectors have a mean cosine of 0.997 and a minimum of 0.978 over the retrieval eval's 66 queries (0.998 and 0.992 over 119 short passages of its corpus), and one of those queries changes rank by one place, both on a fresh index and when only the query side changes. The top-5 rate is 91% for both, which that eval cannot tell apart. Every library that embeds with bge-small was built with the bundled file, which is pinned by hash, so another build of it can replace it there only through a re-embed ([proposal](../proposals/embedding-model-change.md)).

## Hugging Face

`GET /embedding/huggingface/search?q=` lists repos tagged `sentence-similarity` or `feature-extraction`, most downloaded first. `GET /embedding/huggingface/repo/{repo}` opens one, without downloading weights, as a catalog row with one build, or none and why. Both answer in the GGUF search's shapes (`SearchRead`, `RepoRead`), so onboarding renders them through the same search as the chat step:
Expand Down
1 change: 0 additions & 1 deletion docs/architecture/search.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,5 +49,4 @@ Two things it cannot currently show. LIMIT-small is 200 of its 266 queries, so t
## Known gaps

- A question in one language does not find its answer in another. Cosine alone puts the answering passage in the top 5 for 1 of 8 such queries, and the blend for 2 of 8: bge-small is English-only, so no ranking recovers it. Same-language retrieval is unaffected, at 100% in all four non-English slices. A multilingual embedder is the fix, designed in the [retrieval proposal](../proposals/retrieval.md).
- The bundled model is FP16, not the int8 [ADR 0007](../adr/0007-bundled-embeddings.md) specifies: `Qdrant/bge-small-en-v1.5-onnx-Q` is named for quantization but its only ONNX file stores FLOAT16 weights. An int8 export would roughly halve the 63 MB and speed embedding, but every library was built with this exact file, which is now pinned by hash, so a different build is a different vector space: it can replace this one only through a re-embed ([proposal](../proposals/embedding-model-change.md)).
- FTS5's tokenizer keeps a Japanese or Chinese clause as one token, so the keyword leg finds nothing in those scripts and the blend runs on meaning alone there. `trigram` would segment them at the cost of what BM25 means everywhere else ([ADR 0032](../adr/0032-one-tokenizer-for-index-and-question.md)).
Loading