Repository navigation
feat: voice prompting — push-to-talk dictation into the composer (web first) #36
Description
Activity
Closing this — same reasoning upstream gave in pingdotgg/t3code#653, and having now costed it out, I think they were right.
Why building it isn't worth it
The category that makes an in-app feature redundant isn't Wispr Flow (~$12–15/mo) — it's the free, system-wide, offline tools. They paste into whatever app has focus, so they already dictate into T3's composer without a line of T3 code, and they work in the terminal, the editor, and everywhere else at the same time:
- Windows Voice Typing (
Win+H) — already installed, $0, ~85–90% accuracy with Enhanced Speech Recognition enabled. - Handy — MIT, fully offline, push-to-talk, Windows/macOS/Linux. Whisper on GPU, or Parakeet V3 on CPU. (README notes Whisper crashes on some Windows configs; Parakeet V3 is the safe CPU default but weaker on accents/noise.)
- OpenWhispr, whisper-local — same shape, also free and offline.
On phones, iOS and Android keyboard dictation covers the Tailscale web-app workflow that originally motivated this issue, also at $0.
The cost that actually killed it
The API bill was never the problem. At
20 min of dictation/day, Groq$0.30/month**, comfortably inside budget — though its 10-second minimum billing per request bites for push-to-talk, where most clips are shorter than that. OpenAI is $0.003–0.006/min, i.e. ~$1.30–2.60/month for the same usage. Local models are $0.whisper-large-v3-turbo($0.04/hr) runs **The real cost is seam maintenance. Per
docs/t3x/SEAMS.md, the composer is the hottest surface the fork touches —ChatView.tsxat risk 9540,ChatComposer.tsxat 612,ComposerPrimaryActions.tsxat 276 — and the outbox work has already drifted against a new upstream gate once. Adding a mic button means a permanent, recurring rebase tax on the highest-churn file in the tree, paid every sync, forever, to duplicate somethingWin+Hdoes for free. The ledger's own tripwire says re-isolate rather than add row 35; this is a case for not adding the row at all.What would reopen this
Not "we want voice" — that's solved. Only this: if system dictation turns out to drop or mangle text in T3's composer specifically, because the composer is a custom editor (
ComposerPromptEditor.tsxon web,T3ComposerEditorView.swifton iOS) rather than a plain<textarea>. That would be an input-handling bug in the editor, filed as such and fixed there — not a voice feature to build.Worth a two-minute test with
Win+Hagainst the composer before this is fully settled.- Windows Voice Typing (
Summary
T3 Code has no voice input today — there is no way to dictate a prompt or attach a voice note on any surface (web, mobile, desktop). This issue proposes adding push-to-talk voice prompting, transcribed to text and dropped into the composer.
Motivation: the primary phone workflow here is the web app over the Tailscale HTTPS origin (see #19). Typing a long prompt on a phone keyboard is the main friction point in that flow; speaking it is not.
Current state (verified against
main)There is no microphone capture, speech-to-text, or audio-attachment path anywhere in the tree. What exists is audio-adjacent but unrelated:
apps/web/src/browser/browserRecording.tsMediaRecorderovercanvas.captureStream(12)— video capture of the browser-preview tool. No audio track, nogetUserMedia.thread/realtime/*inCodexAdapter.ts:1163-1241,CodexSessionRuntime.ts:530-537outputAudio/delta,transcript/delta,sdp). Translated to canonicalthread.realtime.*events inpackages/contracts/src/providerRuntime.ts:157-161— but no client subscribes to them and there is no RPC to start a realtime session. Dead-ends at the contract boundary.NSMicrophoneUsageDescriptionin the built.appInfo.plistComposer affordances today are attachments / line-break / model / config / stop — no mic button (
apps/mobile/src/features/threads/ThreadComposer.tsx:881-916).Upstream context — this has to be fork-owned
Upstream pingdotgg/t3code#653 "[Discussion] Voice support" was closed as NOT PLANNED (2026-03-15), despite a working demo PR (local NVIDIA Parakeet 0.6B v2 STT + Rust sidecar). Maintainer response: "out of scope, there are apps that do this already." Commenters asked for it to be opt-in if it ever landed.
So there is no upstream implementation to wait for or rebase onto — if we want it, the fork builds and maintains it.
Related upstream issues, still open, all output-side rather than input-side:
Proposed scope (v1)
Surface first: web, because that is the phone-over-Tailscale path.
getUserMedia+MediaRecorder(opus/webm). Secure-context requirement is already satisfied by the Tailscale HTTPS origin.notifyOnNeedsInputpattern inSettingsPanels.tsx).Deliberately out of scope for v1: TTS/read-aloud, realtime bidirectional voice sessions, storing the audio as a durable "voice note" artifact, and mobile-native capture. Mobile app parity is a follow-up once the web shape is settled.
Open decisions
SpeechRecognitionWeb Speech API — free, but effectively unusable on iOS Safari, which is the target device; (c) local model + sidecar, as the upstream demo did — no network, but ships a binary and burns CPU/RAM. Leaning (a) for v1, behind an opt-in setting, with the provider abstracted so (c) can land later.Seam discipline
The composer is one of the hottest fork seams we have —
docs/t3x/SEAMS.mdalready carries rows forComposerPrimaryActions.tsx(risk 276),ChatComposer.tsx(612),ChatView.tsx(9540), andThreadComposer.tsx(480), and the outbox work already drifted against an upstream gate once.Per the ledger's tripwire rule: put the capture/transcription logic in fork-owned files registered through
apps/server/src/t3x/index.tsand the per-surface aggregator. Budget for exactly one new upstream edit — the mic button mount point in the composer footer — and update theSEAMS.mdrow and header totals in the same commit that adds it.