feat(local): add Canary 180M Flash as a second speech model by AnishSarkar22 · Pull Request #2273 · MODSetter/SurfSense

AnishSarkar22 · 2026-10-15T21:51:04Z

What

Adds Canary 180M Flash as a second curated speech recognition model, next to Nemotron 3.5 ASR Streaming 0.6B. It is the light one, for English (also German and French).

	Nemotron 3.5 (existing)	Canary 180M Flash (new)
Download (`Q8_0`)	930.6 MB	249.5 MB
Peak, first request with the load	1,776 MiB (Ryzen 7 5800X)	783 MiB (Apple M2)
Languages listed	26, detected	en, de, fr; must be named
English, Open ASR leaderboard avg	n/a	7.12% WER
Licence	OpenMDW-1.1	CC-BY-4.0

Changes:

Build: canary_asr joins FAMILIES in scripts/audiocpp/pins.mjs. It is in the pinned audio.cpp v0.8.2 and already in upstream's macOS archive; Windows and Linux compile it.
Catalog: a manifest entry written by refresh_local_manifest.py --only canary-180m-flash (sha256-pinned, validated: v0.8.2), its family's locales and request options, the classifier, and cc-by-4.0 on the licence allowlist.
Language: Canary cannot detect the language, and told none it translates into English. A model that cannot detect now starts on its first listed language (English) and refuses auto, so a request always names one. Settings already hid "Detect automatically" for such a model.
Cleanup: a comma or full stop written apart from its word is joined (zeigt , dass → zeigt, dass), as Canary writes German.
UI: the speech model rows in Settings and the set-up popover no longer show the memory while transcribing. Admission still reads peak_mb.
Docs: catalog.md, selection.md, admission.md, packaging.md, and the speech proposal (Which model, Order of work, Measurements, and a note that naming a lighter model in the memory refusal is not built yet).

Spanish is in Canary's family but not listed: audio.cpp v0.8.2 drops every "ñ" it writes ("Mañana el niño" → "Ma ana el ni o").

Why

Nemotron needs about 2.9 GB free before it loads (its peak plus 1 GiB), which an 8 GB Mac in everyday use rarely has. Canary is the lightest recogniser in the pin with near state-of-the-art English, at a quarter of Nemotron's file and under half its peak.

On its own this does not make dictation work on a tight 8 GB Mac: with 1.07 GiB free, both the app's check (783 MiB plus 1 GiB) and audio.cpp's --min-free-memory-mb 1024 guard still refuse Canary. That needs a separate fix to the memory check on macOS.

Fixes # (no issue)

How to test

Automated:

cd surfsense_local/backend && uv run pytest -m unit
cd surfsense_local/backend && uv run pytest -m integration tests/integration/llm tests/integration/worker
cd surfsense_local/electron && node --test scripts/audiocpp/*.test.mjs
cd surfsense_local/frontend && pnpm test src/features/speech-setup src/features/settings/models

Run locally: backend unit 4,416 passed; LLM and worker integration 411 passed; audio.cpp script tests 19 passed; frontend speech tests 19 passed; translations:verify and check_docs.py clean.

On the staged server, with the model file:

cd surfsense_local/backend
SURFSENSE_TEST_AUDIO_MODELS=<folder with canary-180m-flash-q8_0.gguf> \
  uv run pytest -m packaging tests/packaging/test_audiocpp_transcription_config.py

This needs about 1.5 GiB free for audio.cpp's own guard; on the M2 with 1.07 GiB free it was refused with insufficient_memory.

In the app:

Delete electron/audiocpp/ once so staging picks up canary_asr, then pnpm dev.
Settings › Speech recognition › Add model: Canary 180M Flash shows "Q8_0 · 249 MB".
Download it. Spoken language offers English, German and French, no "Detect automatically", and starts on English.
Dictate in English, then switch to German or French and dictate again.

Not run: the Windows and Linux audio.cpp builds, which compile the new family in CI.

🤖 Generated with Claude Code

High-level PR Summary

This PR adds Canary 180M Flash as a second speech recognition model alongside the existing Nemotron model. Canary is significantly lighter (249 MB vs 931 MB) and uses less memory during operation (783 MiB vs 1,776 MiB), making it more suitable for systems with limited RAM. It supports English, German, and French, but cannot auto-detect languages and must be explicitly told which language to transcribe. The PR includes catalog updates, language handling changes for non-detecting models, text cleanup for German punctuation spacing, UI simplifications removing memory display during transcription, and comprehensive documentation updates.

⏱️ Estimated Review Time: 15-30 minutes

💡 Review Order Suggestion

Order	File Path
1	`docs/proposals/local-speech-to-text.md`
2	`docs/architecture/local-models/catalog.md`
3	`docs/architecture/local-models/selection.md`
4	`docs/architecture/local-models/admission.md`
5	`docs/architecture/packaging.md`
6	`surfsense_local/backend/scripts/local_manifest/entries.py`
7	`surfsense_local/backend/modules/llm/catalog/local/manifest/models.json`
8	`surfsense_local/backend/modules/llm/catalog/local/engines/audiocpp/manifest_fields.py`
9	`surfsense_local/backend/modules/llm/catalog/local/classifier.py`
10	`surfsense_local/electron/scripts/audiocpp/pins.mjs`
11	`surfsense_local/backend/modules/llm/transcription/language_intent.py`
12	`surfsense_local/backend/modules/llm/transcription/language_router.py`
13	`surfsense_local/backend/modules/llm/transcription/cleanup.py`
14	`surfsense_local/frontend/src/features/models/local/transcription/describe-speech-model.ts`
15	`surfsense_local/frontend/src/features/speech-setup/speech-model-card.tsx`
16	`surfsense_local/frontend/src/features/settings/models/speech-models-settings.test.tsx`
17	`surfsense_local/frontend/translations/en.json`
18	`surfsense_local/backend/scripts/local_manifest/audiocpp/refresh.py`
19	`surfsense_local/backend/scripts/local_manifest/licence.py`
20	`surfsense_local/backend/tests/integration/llm/test_audio_catalog_routes.py`
21	`surfsense_local/backend/tests/integration/llm/test_speech_language_routes.py`
22	`surfsense_local/backend/tests/integration/llm/test_transcription_routes.py`
23	`surfsense_local/backend/tests/packaging/test_audiocpp_transcription.py`
24	`surfsense_local/backend/tests/packaging/test_audiocpp_transcription_config.py`
25	`surfsense_local/backend/tests/packaging/test_recording_transcription.py`
26	`surfsense_local/backend/tests/unit/llm/catalog/local/engines/audiocpp/rows/test_catalog.py`
27	`surfsense_local/backend/tests/unit/llm/catalog/local/test_classifier.py`
28	`surfsense_local/backend/tests/unit/llm/transcription/test_cleanup.py`
29	`surfsense_local/backend/tests/unit/llm/transcription/test_languages.py`
30	`surfsense_local/backend/tests/unit/scripts/test_local_manifest.py`
31	`surfsense_local/electron/scripts/audiocpp/recipe.test.mjs`
32	`surfsense_local/electron/scripts/audiocpp/server-files.test.mjs`
33	`surfsense_local/frontend/src/features/speech-setup/speech-setup-popover.test.tsx`
34	`surfsense_local/frontend/translations/de.json`
35	`surfsense_local/frontend/translations/es.json`
36	`surfsense_local/frontend/translations/fr.json`
37	`surfsense_local/frontend/translations/hi.json`
38	`surfsense_local/frontend/translations/ja.json`
39	`surfsense_local/frontend/translations/ko.json`
40	`surfsense_local/frontend/translations/pt-BR.json`
41	`surfsense_local/frontend/translations/ru.json`
42	`surfsense_local/frontend/translations/zh-CN.json`

A light recogniser for English, German and French: 249.5 MB and a 783 MiB peak on its first request (Apple M2), against Nemotron's 931 MB and 1,776 MiB. - Build canary_asr into audio.cpp and curate it in the manifest, with cc-by-4.0 on the licence allowlist. - A model that cannot detect the language is always told one: Canary told nothing translates into English. Its intent starts on English and auto is refused. - The cleanup joins a comma or full stop written apart from its word, as Canary writes German. - Spanish is not listed: audio.cpp v0.8.2 drops every "ñ" Canary writes. - Settings' speech rows no longer show the memory while transcribing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

The card shows the download size and where the model works well. The memory figure is gone from the UI; admission still reads peak_mb. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

vercel · 2026-10-15T21:51:08Z

@AnishSarkar22 is attempting to deploy a commit to the Rohan Verma's projects Team on Vercel.

A member of the Team first needs to authorize it.

coderabbitai · 2026-10-15T21:51:40Z

Note

Currently processing new changes in this PR. This may take a few minutes, please wait...

⚙️ Run configuration

Configuration used: Repository: MODSetter/SurfSense/.coderabbit.yaml
Review profile: CHILL
Plan: Advanced
Run ID: f78ad506-6de4-4987-be15-9f33c94b8903

📥 Commits

Reviewing files that changed from the base of the PR and between c45d0fb and cef357a.

📒 Files selected for processing: 42

docs/architecture/local-models/admission.md
docs/architecture/local-models/catalog.md
docs/architecture/local-models/selection.md
docs/architecture/packaging.md
docs/proposals/local-speech-to-text.md
surfsense_local/backend/modules/llm/catalog/local/classifier.py
surfsense_local/backend/modules/llm/catalog/local/engines/audiocpp/manifest_fields.py
surfsense_local/backend/modules/llm/catalog/local/manifest/models.json
surfsense_local/backend/modules/llm/transcription/cleanup.py
surfsense_local/backend/modules/llm/transcription/language_intent.py
surfsense_local/backend/modules/llm/transcription/language_router.py
surfsense_local/backend/scripts/local_manifest/audiocpp/refresh.py
surfsense_local/backend/scripts/local_manifest/entries.py
surfsense_local/backend/scripts/local_manifest/licence.py
surfsense_local/backend/tests/integration/llm/test_audio_catalog_routes.py
surfsense_local/backend/tests/integration/llm/test_speech_language_routes.py
surfsense_local/backend/tests/integration/llm/test_transcription_routes.py
surfsense_local/backend/tests/packaging/test_audiocpp_transcription.py
surfsense_local/backend/tests/packaging/test_audiocpp_transcription_config.py
surfsense_local/backend/tests/packaging/test_recording_transcription.py
surfsense_local/backend/tests/unit/llm/catalog/local/engines/audiocpp/rows/test_catalog.py
surfsense_local/backend/tests/unit/llm/catalog/local/test_classifier.py
surfsense_local/backend/tests/unit/llm/transcription/test_cleanup.py
surfsense_local/backend/tests/unit/llm/transcription/test_languages.py
surfsense_local/backend/tests/unit/scripts/test_local_manifest.py
surfsense_local/electron/scripts/audiocpp/pins.mjs
surfsense_local/electron/scripts/audiocpp/recipe.test.mjs
surfsense_local/electron/scripts/audiocpp/server-files.test.mjs
surfsense_local/frontend/src/features/models/local/transcription/describe-speech-model.ts
surfsense_local/frontend/src/features/settings/models/speech-models-settings.test.tsx
surfsense_local/frontend/src/features/speech-setup/speech-model-card.tsx
surfsense_local/frontend/src/features/speech-setup/speech-setup-popover.test.tsx
surfsense_local/frontend/translations/de.json
surfsense_local/frontend/translations/en.json
surfsense_local/frontend/translations/es.json
surfsense_local/frontend/translations/fr.json
surfsense_local/frontend/translations/hi.json
surfsense_local/frontend/translations/ja.json
surfsense_local/frontend/translations/ko.json
surfsense_local/frontend/translations/pt-BR.json
surfsense_local/frontend/translations/ru.json
surfsense_local/frontend/translations/zh-CN.json

+-----------------------------------------+
| Digging deeper than a Minecraft player. |
+-----------------------------------------+
  \
   \     /)/)
        ( '')
        /  |
      o(___)

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create a new PR

Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

The 2.2.2 entry for the website's changelog: sources ready in seconds (MODSetter#2289), speech recognition for recordings and voice input (MODSetter#2264, MODSetter#2273), plugin search, permissions, licensing and speed (MODSetter#2256, MODSetter#2260, MODSetter#2261, MODSetter#2262, MODSetter#2270, MODSetter#2291), ChatGPT plan sign-in (MODSetter#2259), MP3 podcasts (MODSetter#2183), and the fixes, known issues and developer notes since v2.2.1.

AnishSarkar22 and others added 2 commits October 16, 2026 03:14

refactor(local): drop the speech set-up card's memory while transcribing

cef357a

The card shows the download size and where the model works well. The memory figure is gone from the UI; admission still reads peak_mb. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

AnishSarkar22 requested review from CREDO23 and MODSetter as code owners October 15, 2026 21:51

AnishSarkar22 merged commit 24fc3dc into MODSetter:dev Oct 15, 2026
32 of 37 checks passed

This was referenced Oct 15, 2026

fix(local): stage audio.cpp again when it lacks a curated family #2279

Merged

docs(changelog): v2.2.2 #2292

Merged

[Local|Release] Release desktop 2.2.2 #2293

Merged

Navigation Menu

Repository navigation

feat(local): add Canary 180M Flash as a second speech model - #2273

feat(local): add Canary 180M Flash as a second speech model#2273
AnishSarkar22 merged 2 commits into
MODSetter:devfrom
AnishSarkar22:feat/speech-model-choice

AnishSarkar22 commented Oct 15, 2026 •

edited by recurseml Bot

Loading

Uh oh!

vercel Bot commented Oct 15, 2026

Uh oh!

Uh oh!

coderabbitai Bot commented Oct 15, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

AnishSarkar22 commented Oct 15, 2026 • edited by recurseml Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

Why

How to test

High-level PR Summary

Uh oh!

vercel Bot commented Oct 15, 2026

Uh oh!

Uh oh!

coderabbitai Bot commented Oct 15, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

AnishSarkar22 commented Oct 15, 2026 •

edited by recurseml Bot

Loading