Repository navigation
Description
The gap
docs/architecture/chat.md lists this under Known gaps:
With no hits, the system message is the instructions alone, which still ask for
[n]labels. In the chat eval, Qwen3 1.7B wrote a[1]in all three such answers, which chat drops, and one of them never said the sources did not cover the question.
Why it matters
When retrieval returns nothing — an empty workspace, a question about something the user never saved, or every source unticked in the sources panel — the model is still handed instructions that tell it to put a [n] label after each claim and cite only what the sources back. There are no sources. The small local models this app is built around do as they are told, and the eval caught Qwen3 1.7B inventing a [1] in every one of those answers.
resolve_citations() drops the invented number, so nothing broken reaches the screen. The damage is upstream of that: a model spending its instruction budget obeying a rule that cannot apply is a model not following the rule that matters, which is to say plainly that the knowledge base did not cover the question. One of those three answers never said it.
Where
surfsense_local/backend/modules/chat/prompt.py—build_context()returnsinstruction, []for the no-hits case. That early return is the branch this issue is about.surfsense_local/backend/modules/chat/prompts/—compact.md,capable.md,frontier.md, the three files that carry the citation rules today.surfsense_local/backend/modules/llm/prompting/—load(package, tier, case="", **slots)already supports a second prompt per package throughcase, which is the mechanism to reuse if you go the separate-file route. Do not add a loader.
What done looks like
- With no hits, the model is not asked to label claims it cannot source, and is asked to say the knowledge base did not cover the question before answering from its own knowledge.
- How you get there is yours to choose, and the two options differ in cost. A
casesubfolder means three more prompt files and full control per tier; appending or swapping a paragraph inbuild_context()means one place to change and no new files. The house style favours the shorter diff, butcompact.mdwas tuned against measured eval behaviour, so do not quietly rewrite prose the eval is holding steady. - The prompt-tier contract is unchanged: whatever you add exists for all three tiers, because
tests/unit/llm/prompting/test_prompts_ship.pyasserts a case ships every tier and a missing file fails the job on the user's machine. - Delete the Known gaps line from
docs/architecture/chat.md, and update the Grounding shape bullet that says "No hits, no context block: the system message is the instruction alone".
What not to change
If you add prompt files in a subfolder, the PyInstaller specs will not pick them up. Both bundling/api.spec and bundling/worker.spec collect chat's prompts with includes=["prompts/*.md"], which is one level deep and will not match prompts/<case>/<tier>.md. Nothing imports a prompt, so nothing fails at build time — it fails on a packaged install, at the moment a user asks a question with no hits. Widen both globs if you take that route.
How to verify
surfsense_local/backend/tests/unit/chat/test_prompt.py covers build_context() and is where the failing test goes: with no hits, the system message does not ask for [n]. tests/unit/llm/prompting/test_prompts_ship.py covers the tier completeness half.
cd surfsense_local/backend
uv run pytest -m unit
uv run ruff check .
# from the repo root, because the PR edits a doc
cd ../.. && python scripts/check_docs.pyBefore you start
Read CONTRIBUTING.md, the chat doc and the chat eval proposal, which is where the measurement in this gap line came from and where a claim about prompt behaviour gets settled. Open your PR against dev, not main. Comment here to claim this.
Activity
MONISMALIK1 commentedon Sep 29, 2026
I'd like to take this one. Going with the shorter-diff route in
build_context(): a no-sources instruction for every tier, no new prompt files,compact.mduntouched. PR incoming.fix(chat): send no citation rules when retrieval finds nothing
MODSetter commentedon Sep 29, 2026
Fixed by #2028,
fix(chat): send no citation rules when retrieval finds nothing, by @MONISMALIK1. Merged intodev.Closing by hand rather than by keyword:
Fixes #1947only auto-closes on a merge to the default branch, and this repo merges throughdev(CONTRIBUTING), so this would have stayed open and claimable while the work was already done. It reachesmainwith the next release.Nothing left to pick up here.