Skip to content

Trial an in-app debug bridge so agents can inspect the running Tauri app #3739

Closed

Description

@malibio

Overview

Add a debug-only bridge plugin to the Tauri app so an AI agent can inspect and drive the real running app (DOM, console, JS evaluation, input, IPC traffic) from a CLI. This is a trial: it is purely additive, and nothing in browser mode is removed.

Problem Statement

The only way an agent can look at the running UI today is bun run dev:browser: the frontend in Chrome, driven by Playwright or chrome-devtools. Chrome is not a Tauri process, so that mode needs a second backend path that imitates the Tauri commands:

  • packages/dev-tools/ (the HTTP dev-proxy, about 2,000 lines)
  • HttpAdapter in packages/desktop-app/src/lib/services/backend-adapter.ts
  • BrowserSyncService in packages/desktop-app/src/lib/services/browser-sync-service.ts

All of it is hand-kept in step with the Tauri command layer, and it is a path no shipped user is ever on. It also means the agent inspects Chrome, not the WKWebView the app ships with.

Playwright and chrome-devtools cannot attach to the Tauri app directly: WKWebView exposes no remote debugging protocol. The working alternative on macOS is a plugin inside the app that evaluates JavaScript in the webview and returns results over a local socket.

Proposed Solution

Wire hypothesi/mcp-server-tauri into development builds:

  • Rust plugin tauri-plugin-mcp-bridge, registered only in development builds
  • Driven through its CLI (@hypothesi/tauri-mcp-cli, the tauri-mcp command), with its MCP server (@hypothesi/tauri-mcp-server) as the alternative interface

It was chosen over the other community options because it is the most adopted, is built for an agent driving a live app rather than for test files, has both a CLI and MCP, and can monitor and invoke the real Tauri commands. None of the options is official or mature, which is why this is a trial with explicit proof criteria.

If it fails a proof criterion below on our build, try P3GLEG/tauri-plugin-mcp (token-authenticated Unix socket, native macOS input; git dependency, not on crates.io) against the same criteria before concluding.

Documentation

  • ADR-035 - agents reach the knowledge graph through the skill and CLI, not MCP. This bridge is a different surface (debugging the UI, not the graph), so it does not reverse the ADR, but prefer the tool's CLI over its MCP server for the same reasons.
  • ADR-048 - describes the two adapter paths and why the HTTP path is not the production path. Context only; the contract tests it requires are untouched.
  • ADR-082 - nodespace-app-lib is shared by every app crate built on core, so the plugin registration must be generic and carry no product-specific wording.
  • browser-dev-mode.md - the current agent-debugging flow this trial is measured against.
  • Updates: if the trial succeeds, development/browser-dev-mode.md gains a section describing the Tauri-app path next to the browser one. No ADR: retiring browser mode is a later decision and is not made here.

Acceptance Criteria

Wiring

  • The bridge plugin is an optional dependency behind a Cargo feature that only bun run dev:tauri enables (extend the existing devtools feature in packages/desktop-app/src-tauri/Cargo.toml, or add a sibling feature), with exact versions pinned for the crate and the npm packages
  • A release or bundle build does not compile or register the plugin, pinned by a test or a build-script check
  • The plugin's permission is granted only in the development build, without weakening the production window capability (packages/desktop-app/app-lib/src/window-capability.json)
  • withGlobalTauri stays false in the committed tauri.conf.json. If the plugin needs it, it is turned on only for the dev run (for example a --config override in dev:tauri)
  • The production CSP in tauri.conf.json is unchanged

Proof on macOS (WKWebView), against bun run dev:tauri

  • DOM snapshot of a page with rendered nodes
  • Console output read back, including an error thrown by the page
  • JavaScript evaluated in the webview, with the result returned
  • A click and typed text land in a text node's editor and the content persists
  • Screenshot captured
  • A real Tauri command observed through IPC monitoring, and one invoked directly
  • The app under inspection talks to a daemon on an isolated NODESPACE_HOME and a short NODESPACED_SOCKET path, never the user's real database (same rule as the run-app skill)
  • Two worktrees can each run an inspected app at the same time, or the limitation and its cause (the default WebSocket port is 9223) are written down

Outcome

  • A comment on this issue records what worked, what did not, and how it compares with dev:browser plus Playwright for day-to-day agent debugging, so the decision to retire browser mode can be made from it
  • If it works: .claude/skills/run-app/SKILL.md gains the launch-and-inspect recipe for the Tauri path, and browser-dev-mode.md gains the matching section
  • bun run test:changed passes, including the sync-boundary check
  • Code passes bun run quality:fix

Technical Specifications

Reference Files

  • Plugin registration: packages/desktop-app/app-lib/src/lib.rs - register_core_plugins builds the plugin list; run adds the window capability first (window_capability::add)
  • Capability: packages/desktop-app/app-lib/src/window_capability.rs and window-capability.json - the capability is added at runtime, not from a file. An app crate must depend directly on every plugin crate the capability names (window_capability::plugin_crates), because tauri_build reads permission files from direct dependencies
  • App crate: packages/desktop-app/src-tauri/Cargo.toml - the devtools feature and the direct plugin dependencies
  • Dev script: packages/desktop-app/package.json - dev:tauri runs bunx tauri dev --features devtools -- --bin nodespace-app
  • Config: packages/desktop-app/src-tauri/tauri.conf.json - withGlobalTauri: false, strict CSP (script-src 'self', connect-src 'self' ipc: http://ipc.localhost)
  • Launch recipe to extend: .claude/skills/run-app/SKILL.md - daemon isolation rules

Implementation Details

  • The bridge's MCP server or CLI connects to a WebSocket opened by the Rust plugin; the webview itself should not need a new connect-src entry. Confirm this rather than loosening the CSP.
  • The upstream docs say withGlobalTauri must be true. Confirm whether that still holds for the pinned version before changing anything.
  • The upstream README states the project was built with agentic coding tools and may contain bugs, and documents no authentication on its socket. Treat it as a development-only tool: it must be impossible to enable in a shipped build.
  • The registration lives in nodespace-app-lib, which other app crates build on. Name the feature and any comments for what the bridge does; no product names.
  • tauri-plugin-mcp-bridge is a new Rust dependency in every dev build. Note the effect on a cold dev:tauri build in the outcome comment.

Non-Goals

  • Removing or changing dev:browser, the dev-proxy, HttpAdapter, BrowserSyncService, or the e2e tier that spawns the proxy. bun run dev keeps pointing at dev:browser.
  • Moving any test onto the bridge.
  • Adopting the Tauri 3 CEF runtime (Chromium webview with real CDP). It is alpha and a separate decision.

Activity

  1. malibio commented on Oct 13, 2026

    @malibio
    CollaboratorAuthor

    Frontend: 326 files / 5899 tests passed

  2. self-assigned this
    on Oct 13, 2026
  3. malibio commented on Oct 13, 2026

    @malibio
    CollaboratorAuthor

    Trial outcome (PR #3776)

    Verdict: works for querying and driving the real WKWebView; a complement to dev:browser, not yet a replacement for passive watching.

    Worked (macOS, dev:tauri, tauri-plugin-mcp-bridge 0.13.0, scratch HOME + isolated daemon)

    • DOM snapshot (accessibility tree), JS eval, console logs including a thrown error with its source, screenshot
    • Click and typed text into a node's editor, persisted across a full reload (daemon reported the scratch DB path)
    • A real command (list_databases) captured by the IPC monitor and invoked directly
    • Bound to 127.0.0.1; permission is its own capability; committed withGlobalTauri: false and CSP unchanged; no new connect-src needed

    Did not work / caveats

    • IPC monitor only sees window.__TAURI__.core.invoke; the app's own calls (@tauri-apps/api -> __TAURI_INTERNALS__) are invisible. A reload resets it.
    • Newest plugin (0.13.1) requires tauri ~2.12.2 and moved tauri/wry for every build in Cargo.lock; pinned 0.13.0 (tauri ^2).
    • --config set TAURI_CONFIG and disabled the unstaged-bundle-entry drop, failing fresh worktrees; fixed in the helper.
    • Launching the app is not isolated by NODESPACE_HOME: it writes ~/.nodespace/bin, kills the default-socket daemon and registers the per-user debug launchd label. Needs a scratch HOME with a pre-seeded daemon. Documented.
    • Two apps at once: not run. The plugin takes the first free port from 9223 (100 ports); CLI needs --port.
    • Cold build: not measured. The plugin adds ~35 packages to Cargo.lock (objc2 family, tokio-tungstenite, image/zune-jpeg, windows-*), built only with the feature.

    Compared with dev:browser + Playwright

    • Better: inspects the shipped WKWebView and the real Tauri commands with no dev-proxy/HttpAdapter path to keep in step.
    • Worse: no passive network/IPC watching of the app's own traffic, more fragile launch (home/daemon/launchd), one element-targeting model (CSS/XPath/text) with less tooling than Playwright, no test runner.
    • Decision input: retiring browser mode would lose passive IPC visibility and the cheap isolated launch; neither is recoverable with this plugin alone.
  4. malibio commented on Oct 13, 2026

    @malibio
    CollaboratorAuthor

    Addendum: hover and right-click through the bridge dispatch synthetic events (isTrusted: false). JS handlers fire, but CSS :hover does not apply (matches(':hover') stayed false), so hover styles can't be verified this way. Click and typing are fine. focus was not tested. Playwright against dev:browser does not have this limit; P3GLEG's plugin (native macOS input) is the one to try if real pointer input matters.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions