Skip to content

feat(audio): loopback feeder mode — all app audio under exclusive/ASIO#865

Merged
byrongamatos merged 2 commits into
mainfrom
fix/asio-all-app-audio
Jul 11, 2026
Merged

feat(audio): loopback feeder mode — all app audio under exclusive/ASIO#865
byrongamatos merged 2 commits into
mainfrom
fix/asio-all-app-audio

Conversation

@OmikronApex

@OmikronApex OmikronApex commented Jul 11, 2026

Copy link
Copy Markdown
Contributor

Stacked on #852 (diag branch). Fixes the tester-confirmed gaps from the 2026-07-11 log analysis:

Loopback mode (new preferred)

One getDisplayMedia frame-audio capture covers every sound the app makes — song, previews, UI — for the whole exclusive/ASIO session (engages even with no song loaded). Desktop main answers with the app's own frame (frame-scoped). Local playback silenced via suppressLocalAudioPlayback, page-mute IPC fallback otherwise. Sticky fallback to the existing stems/element surface modes when capture is unavailable (old desktop main, permission denied; Docker sphere never reaches the feeder).

Element-capture poisoning fix (tester log, lines 985/997)

createMediaElementSource(#audio) collides with highway_3d's analyser tap (one-shot per element → InvalidStateError); _elCtx was assigned before the throw, so every later tick died with TypeError: Cannot set properties of null while the song kept playing on the default WASAPI device. Now: module state assigned only after the whole chain succeeds, context closed on failure, failed engage disables the bus + tears down loopback.

Companion

Desktop PR fix/asio-all-app-audio (display-media handler, page-mute IPC, streamer-mix engine fix).

Verification

12/12 sandbox tests (5 new: loopback engage without song, loopback preferred over stems, page-mute fallback + unmute on disengage, sticky fallback on denial, collision retry without poisoning). Needs tester run on real ASIO hardware.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features

    • Added whole-app loopback audio capture for exclusive output sessions.
    • Automatically mutes and restores page playback when required by the capture environment.
    • Added fallback handling to continue using alternative audio capture modes when loopback is unavailable.
  • Bug Fixes

    • Improved recovery from audio capture failures and prevented incomplete audio sessions.
    • Ensured audio capture resources and renderer state are cleaned up during disengagement.

@coderabbitai

coderabbitai Bot commented Jul 11, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

The renderer-bus feeder adds whole-application loopback capture for exclusive output, falls back to stems or element capture when unavailable, hardens element initialization, expands teardown handling, and adds coverage for loopback, fallback, mute, retry, and collision scenarios.

Changes

Renderer bus routing

Layer / File(s) Summary
Capture path setup and teardown
static/app.js
Element capture now commits state only after complete setup succeeds. Loopback capture uses display media, a worklet tap, renderer-bus forwarding, track stopping, context closure, and page-mute restoration.
Exclusive-session engagement and fallback
static/app.js
Exclusive running sessions prefer loopback, record failed loopback capture as unavailable, fall back to stems or element capture, and clean up renderer-bus state after failures.
Loopback and fallback validation
tests/js/renderer_bus_feeder.test.js
Tests cover media-stream setup, fallback probing, native transport, teardown, mute restoration, sticky failures, retries, graph replacement, and element-capture collisions.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Engine
  participant RendererBusFeeder
  participant MediaDevices
  participant RendererBus
  Engine->>RendererBusFeeder: report exclusive running state
  RendererBusFeeder->>MediaDevices: request whole-app loopback capture
  MediaDevices-->>RendererBusFeeder: stream or capture error
  alt Loopback succeeds
    RendererBusFeeder->>RendererBus: forward captured audio
  else Loopback fails
    RendererBusFeeder->>RendererBus: engage stems or element fallback
  end
Loading

Possibly related PRs

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 20.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: adding a loopback feeder mode for all-app audio during exclusive/ASIO sessions.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/asio-all-app-audio

Comment @coderabbitai help to get the list of available commands.

@byrongamatos
byrongamatos changed the base branch from diag/asio-routing-debug to main July 11, 2026 11:41
OmikronApex and others added 2 commits July 11, 2026 13:43
Tester-confirmed (2026-07-11 log): song previews and other
plugin-private audio bypass the per-surface feeder taps and leak to the
default WASAPI device under ASIO output. Also confirmed: the element
capture path poisons itself when highway_3d already owns #audio's
one-shot MediaElementSource (InvalidStateError with _elCtx assigned
pre-throw → TypeError every later tick).

- New preferred mode 'loopback': one getDisplayMedia frame-audio capture
  (desktop main answers with the app's own frame) covers song, previews,
  and UI sounds for the whole exclusive session — engages even with no
  song loaded. Local playback silenced via suppressLocalAudioPlayback,
  page-mute IPC fallback otherwise.
- Sticky fallback to the existing stems/element surface modes when
  capture is unavailable (old desktop main, denied, Docker sphere).
- Element capture: assign module state only after the whole chain
  succeeds; close the context on failure — collision now retries clean.
- Failed engage now disables the bus and tears down loopback (no more
  bus-enabled-with-no-producer stranding).
- Tests: 12 (5 new — loopback engage/preference/mute-fallback/sticky
  fallback, collision retry).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…orklet)

The loopback context was reused across engages (_lbCtx || new), but teardown
only stopped the stream + deactivated the tap — never closing the context or
detaching the worklet node. Each exclusive<->shared switch orphaned a live
tap worklet on the long-lived context. Use a fresh context per session and
close it on disengage. Adds a test asserting the context is closed on teardown.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@byrongamatos
byrongamatos force-pushed the fix/asio-all-app-audio branch from 1db9acd to 8f53ea1 Compare July 11, 2026 11:43

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@static/app.js`:
- Around line 5603-5610: Update the failure handling around the surface-mode
engage flow, including _engageStems() and _engageElement(), so any error after
switching to the null sink restores the original surface sink before leaving the
outer catch. Ensure failed engages cannot leave the context muted, while
preserving the existing _mode reset, renderer-bus cleanup, and loopback teardown
behavior.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 613060c1-d2a2-4e7e-be7e-c40dad3c1df4

📥 Commits

Reviewing files that changed from the base of the PR and between 1db9acd and 8f53ea1.

📒 Files selected for processing (2)
  • static/app.js
  • tests/js/renderer_bus_feeder.test.js
🚧 Files skipped from review as they are similar to previous changes (1)
  • tests/js/renderer_bus_feeder.test.js

Comment thread static/app.js
Comment on lines 5603 to +5610
} catch (e) {
console.warn('[renderer-bus] reevaluate failed (will retry):', e);
_mode = 'off';
// A partial engage may have left the bus enabled with no producer
// and the page muted — undo both so a failed tick can't strand
// audio in silence until the next successful engage.
try { await api.setRendererBus(false, 0); } catch (_) { /* engine gone */ }
await _teardownLoopback().catch(() => {});

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Inspect surface-mode engage functions for sink flips that precede possible throws
fd -t f 'app.js' static | xargs -I{} ast-grep outline {} --items all --match '_engage(Stems|Element)|_setSink' 2>/dev/null
rg -nP -C6 'async function _engage(Stems|Element)\s*\(' static/app.js

Repository: got-feedBack/feedBack

Length of output: 1301


🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Read the engage/disengage and failure-handling sections with line numbers.
sed -n '5465,5535p' static/app.js | cat -n
printf '\n-----\n'
sed -n '5580,5625p' static/app.js | cat -n

printf '\n-----\n'
# Inspect sink helper and any self-cleanup around element capture.
rg -n -C4 '_setSink|_ensureElementCapture|_teardownLoopback|_disengage' static/app.js

Repository: got-feedBack/feedBack

Length of output: 11077


🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Inspect the exact failure path and the surrounding state transitions.
sed -n '5478,5528p' static/app.js | cat -n
printf '\n-----\n'
sed -n '5590,5615p' static/app.js | cat -n

Repository: got-feedBack/feedBack

Length of output: 3947


🏁 Script executed:

#!/bin/bash
set -euo pipefail

python3 - <<'PY'
from pathlib import Path
p = Path('static/app.js')
text = p.read_text()
for needle in ['async function _disengage', 'async function _engageStems', 'async function _engageElement', 'async function _ensureElementCapture', '_mode = \'off\'', '_teardownLoopback']:
    idx = text.find(needle)
    if idx != -1:
        start = max(0, idx - 300)
        end = min(len(text), idx + 1200)
        print(f"\n=== {needle} ===\n")
        print(text[start:end])
PY

Repository: got-feedBack/feedBack

Length of output: 9378


Restore the sink on failed surface-mode engage
_engageStems() and _engageElement() flip their context to the null sink before later awaits can throw. If that happens, the outer catch only sets _mode = 'off' and tears down loopback, so the surface context can stay stranded on the null sink until a later successful engage. Clean up the surface-mode sink in the failure path, or make the engage helpers self-revert on error.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@static/app.js` around lines 5603 - 5610, Update the failure handling around
the surface-mode engage flow, including _engageStems() and _engageElement(), so
any error after switching to the null sink restores the original surface sink
before leaving the outer catch. Ensure failed engages cannot leave the context
muted, while preserving the existing _mode reset, renderer-bus cleanup, and
loopback teardown behavior.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants