Skip to content

Linux: voice capsule state bridge + GNOME Shell extension (fix no-capsule on GNOME Wayland) + client resilience - #1155

Open
questionjie-max wants to merge 15 commits into
Open-Less:betafrom
questionjie-max:linux-capsule-and-resilience
Open

questionjie-max wants to merge 15 commits into
Open-Less:betafrom
questionjie-max:linux-capsule-and-resilience

Conversation

@questionjie-max

Copy link
Copy Markdown

提交标题

Linux: voice capsule state bridge + GNOME Shell extension (fix "no capsule on GNOME Wayland") + client resilience fixes

背景 / 问题

  1. GNOME Wayland 上没有胶囊:Linux 端的录音状态目前只通过 fcitx5 辅助区
    纯文本展示(emit_capsule_with_context_locked 的 aux 分支),因为 GNOME
    Wayland 不提供「第三方窗口置顶且不抢焦点」能力(无 layer-shell),且
    GNOME 50 起移除了 X11 会话,悬浮胶囊窗口路线在该平台不可行。
  2. 听写热键依赖焦点应用:fcitx5 未注册 Wayland 原生 IM 协议
    (fcitx5-diagnose 报 Using Wayland native input method protocol: 0),
    插件热键只在焦点应用恰好以 fcitx IM 模块启动时可达;焦点切到非 fcitx
    应用(空 IM 环境变量启动)后 toggle 丢失,会话卡在 Listening 持续录音
    (实测卡死 18 分钟,ASR 额度被无谓消耗)。
  3. 服务端抖动时的韧性缺口(2026-10-07 火山故障窗口实测):
    • 自动静默重试建立新连接后,上一局 audio worker 持有的共享 writer 槽位
      已被新会话接管,旧 worker 把积压的旧 seq 帧(1630)泄漏进新连接
      (seq 1..369 之后突然 1630)→ 服务端 45000000 sequence mismatch,
      重试必败;
    • 重试最终失败后向已关闭 WS 的推帧循环不停止(实测 1910 条
      websocket not open 刷屏);
    • 每次录音首块为设备启动瞬间的削波垃圾(inRMS≈0.999)且作为首包上行;
    • ASR 返回空转写(多为服务端问题或真的没说话)以红色 Error 胶囊呈现,
      用户会误以为是自己出错。

方案

1. 状态桥(Rust,新文件 capsule_bridge.rs)

  • emit_capsule_with_context_locked(胶囊状态唯一出口)向
    ~/.cache/openless/capsule-state.sock(Unix 流式 socket)广播 JSON 行:
    {"state":"recording","level":0.42}(状态变更即发,电平 30Hz);
  • 多客户端广播、无客户端静默零开销、对端断开自动剔除;
  • 暴露 has_clients():有 GUI 客户端订阅时自动抑制 fcitx 辅助区文字
    (避免双重指示),扩展退出自动恢复。

2. GNOME Shell 扩展(GJS/ESM,源码在本分支 gnome-shell-extension/ 目录;

构建说明见目录内 README)

  • Main.layoutManager.addTopChrome({reactive:false}) 挂载:天然置顶全部
    窗口、鼠标穿透、不抢焦点;
  • Clutter.ShaderEffect(legacy set_shader_source 路线;mutter 50 GI 已移除
    ShaderType 枚举,构造用字面量 shader_type:1)渲染 Siri 风格流体光球:
    6-metaball + smin 软融合 + 光谱色带 + RGB 色差,音量驱动幅度(振幅主通道,
    亮度增益 ≤40%,对齐行业惯例:Wispr Flow/Echo/macOS 听写均以振幅而非亮度
    表达说话程度);终态图标走 Cairo;
  • 支持 ~/.cache/openless/capsule-shader.frag 着色器热替换(2 秒生效),
    便于视觉调参与社区共创;
  • Main.wm.addKeybinding 全局听写热键(合成器抓取,任意窗口可用),
    通过 DBus 向应用发 DictationKeyEvent;扩展禁用时 fcitx 插件热键路径
    自动兜底;
  • OpenLess 不在时整体隐藏、零存在感;socket 断开自动重连。

3. 韧性修复(客户端)

  • volcengine.rs:audio worker 持 session_gen 代数快照,open_session 推进
    代数后旧 worker 自行退出(杜绝旧 seq 帧泄漏进新连接);发送失败 break
    终止本会话推送;
  • recorder.rs:丢弃每会话首个音频回调块(设备启动瞬间的削波垃圾,
    inRMS≈0.999);
  • dictation.rs:emptyTranscript 改走 CapsuleState::Cancelled(中性灰),
    仅真正的 ASR/网络错误保留红色 Error。

4. 电平动态化(dictation.rs LevelDynamics)

recorder 的 LEVEL_RMS_GAIN=4 直出导致底噪 0.3~0.5、说话顶格 1.0,动态被
压扁。level_handler 内新增噪声门(自适应基线只从低处学习)→ 基线归一化 →
pow1.35 扩张 → 快攻慢放包络(45ms/250ms)。插入点在 silence_auto_stop 消费
原始电平之后,不影响自动停录。实测:静音环境 189 帧电平全部为 0。

已验证(Ubuntu 26.04 · GNOME 50 · Wayland · ThinkPad E14)

  • 真机听写全链路(热键→录音→火山流式→DeepSeek 润色→fcitx5 上屏)稳定运行;
  • 静音 189 帧电平全 0;说话时光球幅度随音量涨落、无爆亮;
  • 扩展热替换通道工作正常(frag 文件修改 2 秒生效)。

兼容性 / 隐私

  • 全部新增代码 #[cfg(target_os = "linux")] 或独立 GJS 文件,不影响
    macOS/Windows/Android;
  • 状态桥只在本机 Unix socket 广播,无网络、无遥测;不包含任何凭据;
  • 扩展 AGPL-3.0 与上游一致,全部为原创代码。

备注

  • 本补丁基于 v1.3.18-tauri 标签生成;若套用到 beta(v2/eframe)分支,
    capsule_bridge.rs 与扩展可整体平移(事件出口同为
    emit_capsule_with_context_locked),volcengine.rs 建议在 beta 上重新对照;
  • 本地开发环境无法直连 GitHub,tauri-nspanel 暂改为 path 依赖 + vendor——
    合并时请保留原 git 依赖(该 hunk 可剔除);
  • qwen3-asr-rs 依赖行按 scripts/ci-disable-macos-qwen3.mjs 惯例移除
    (Linux 构建不需要)。

Duyi-Wang and others added 15 commits July 14, 2026 18:37
* fix(llm): make custom temperature configurable

Allow custom OpenAI-compatible LLM providers to omit or override the temperature parameter instead of always sending the default 0.3.

- Send temperature only when configured in OpenAI-compatible chat bodies
- Keep built-in OpenAI-compatible providers on the existing 0.3 default
- Store active custom LLM temperature via ark.temperature
- Expose a Temperature field for custom LLM providers in Settings
- Reuse the same temperature config for provider validation and real polish
- Add tests for omitted/configured temperature and input validation

* docs(settings): show custom temperature range

---------

Co-authored-by: Chris233 <h-chris233@outlook.com>
chore(release): prepare 1.3.17 stable
* [cask] openless 1.3.17 [skip ci]

* fix(settings): keep experimental state consistent

* fix: reconcile preference writes with external updates

---------

Co-authored-by: sim <sim@local>
Co-authored-by: Chris233 <h-chris233@outlook.com>
…amics, client resilience

- capsule_bridge.rs: Unix socket state broadcast (state/level JSON lines)
  + has_clients() to suppress fcitx aux text when a GUI client is attached
- GNOME Shell extension (separate zip artifact): Wayland floating Siri-style
  fluid orb (GLSL), global dictation hotkey via Mutter grab, aux suppression
- LevelDynamics: noise gate + adaptive baseline normalization + expansion +
  attack/release envelope for the UI level (silence 189 frames all 0.000)
- volcengine.rs: audio worker session-generation guard (stale worker must not
  leak old-sequence frames into a new connection) + break on send failure
- recorder.rs: drop the first audio callback chunk (device startup garbage)
- dictation.rs: emptyTranscript renders neutral Cancelled instead of red Error
- capsule_focus.rs: aux emission gated on !x11 && !has_clients()
- Wayland floating Siri-style fluid orb (GLSL ShaderEffect, Cairo fallback)
- State bridge client (Unix socket JSON lines, 30Hz level)
- Global dictation hotkey via Mutter grab (schema: Alt_R, rebindable)
- Aux text suppression handshake via state-bridge client count
- Shader hot-swap support (~/.cache/openless/capsule-shader.frag, 2s live)
- AGPL-3.0, original code, no credentials/telemetry
…doubled high-frequency jitter, safety margin
With a compositor-level global dictation hotkey (GNOME Shell extension
grabbing Alt_R), hotkey key events never transit fcitx5, so savedIc_ —
snapshotted at the last key event that did — goes stale indefinitely.
CommitText then delivers text to whatever window was focused at that
old moment (mis-delivery into unrelated apps, observed as 'dictating
in Firefox, text lands in the terminal').

Prefer the live focused IC (GTK/Qt IM modules report focus reliably
for all regular apps); keep savedIc_ only as a fallback when no IC is
focused. Also log the target program name at Info level to make
mis-routing diagnosable from logs.

Verified on GNOME 50 Wayland: dictation into Firefox / text editor /
terminal each delivers to the focused window; zero fallback warnings.
…new effect instances)

- null guard in the 60fps animation callback (teardown race during
  extension disable produced 'this._area is null' errors)
- shader hot-swap now rebuilds the ShaderEffect instance: set_shader_source
  on a live effect does not trigger a Cogl pipeline rebuild, so the old
  same-object path silently kept the previous shader
- enable() seeds the effect from the frag override file when present
@H-Chris233 H-Chris233 added area:asr ASR area area:hotkey Hotkey area area:recording Recording area bug Something isn't working enhancement New feature or request large-change Large or risky change needs-review Needs review needs-tests Needs tests needs-work Needs additional work P1 High priority platform:linux Linux-specific labels Oct 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:asr ASR area area:hotkey Hotkey area area:recording Recording area bug Something isn't working enhancement New feature or request large-change Large or risky change needs-review Needs review needs-tests Needs tests needs-work Needs additional work P1 High priority platform:linux Linux-specific

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants