feat(terminal): add xterm.js as a selectable backend

[Ayman Bagabas (aymanbagabas)]

tui-test can already run a session on alacritty, ghostty, or rio. This adds the emulator behind VS Code's terminal as a fourth, so a suite can be run against what a large share of users are actually looking at. A test that passes on one emulator and fails on another is telling you something real about the program under test.

$ tui-test open --backend xtermjs
$ tui-test run --backend xtermjs -- vim notes.md

Selectable everywhere the other backends are: the CLI's --backend, and both the Python and Node bindings.

No Node at runtime

@xterm/headless and the unicode11 addon are vendored and evaluated into a QuickJS context per session, so the backend depends on nothing installed on the machine. shim.js is our own code; the bundles are dropped in unchanged at pinned versions.

They are committed rather than fetched, so a clone builds without a network and there is no build script. .github/scripts/vendor-xtermjs.sh --latest bumps pinned.json to the newest releases and re-fetches in one command, and .github/workflows/xtermjs-update.yml runs that same script weekly and opens a pull request when either package has a newer release.

The grid crosses that boundary packed rather than a cell at a time. Reading an 80x30 screen through the per-cell getters is 2,400 calls with ten property reads each, which costs milliseconds per poll; the shim flattens a row span into one string and one integer array, so a whole screen crosses as two values and this side decodes rather than traverses.

What the shim has to supply

Five things the Emulator contract asks for are missing from, or wrong in, the headless bundle's public surface. The shim supplies them rather than the Rust side pretending they are absent:

Contract	How
Window title	onTitleChange, with windowOptions.pushTitle/popTitle enabling the CSI 22/23 t stack the bundle implements but leaves off by default
Cursor visibility	coreService.isCursorHidden
Cursor shape	coreService.decPrivateModes.cursorStyle, which is absent until DECSCUSR sets one and so reads as a block until then
OSC 4/10/11/12 set, query, and reset	parser.registerOscHandler, answering out of the same reply queue the terminal's own replies use, so answers keep the order they were asked in
SGR 59	xterm.js stores the reset as an explicit white, indistinguishable from a real 58;2;255;255;255 once written; the shim clears it where the parser sets it, while the two are still apart

A colour reply has to echo the terminator its query used, and an OSC handler is handed its payload but not that terminator. The shim scans the incoming bytes for it, carrying state between calls because a PTY read splits wherever it likes — including between the two bytes of an ST — and keying what it finds by OSC code so a title arriving between two queries cannot put the wrong terminator on a reply.

Verified against a real session rather than only in unit tests. The same probe run on both backends, querying the background, setting it, resetting it, and querying once more with each terminator:

alacritty:  <E>]11;rgb:0000/0000/0000<BEL> | <E>]11;rgb:6565/4343/2121<BEL> | <E>]11;rgb:0000/0000/0000<BEL> | ST:<E>]11;rgb:0000/0000/0000<E>\
xtermjs:    <E>]11;rgb:0000/0000/0000<BEL> | <E>]11;rgb:6565/4343/2121<BEL> | <E>]11;rgb:0000/0000/0000<BEL> | ST:<E>]11;rgb:0000/0000/0000<E>\

Byte-identical, terminator included.

Conformance

The backend passes the conformance suite, which is what makes swapping emulators safe, with one declared exception: xterm.js records a cell's underline colour only when that cell also has an underline style, so SGR 58 on its own — or a colour that outlives an SGR 24 — is not readable back off the cell.

Nothing renders differently, since a cell with no underline draws no underline colour either way. What is lost is the colour surviving in the cell vocabulary. I confirmed this against the bundle rather than inferring it: the cell reports isAttributeDefault(), and the colour is gone from the line's extended attributes while remaining in the current SGR state, so it is xterm.js's per-cell storage rather than anything this mapping does.

The suite grew a Divergences declaration for it, so the exception is a visible claim a reviewer can check rather than a quietly failing case or a backend that skips conformance altogether. It is deliberately narrow: a field earns a place there only when the limitation is in the emulator itself.

Also here

Backend::ALL becomes a slice, so each backend adds one line rather than doubling the number of cfg-gated definitions, and the list of names in a parse error is derived from it instead of repeated in prose that can drift out of step.

Notes for review

• The backend is compiled in by default for the CLI and both bindings, matching how ghostty is wired, so --backend xtermjs works out of the box. Measured cost: 229.6 KiB of embedded JavaScript, and a release binary going 9,208,800 -> 10,506,064 bytes, so about 1.30 MB once QuickJS is compiled in. Happy to put it behind an off-by-default feature instead if the size matters more than the availability.
• The xtermjs crate feature itself is off by default, so cargo build -p tui-test-rs is unaffected.
• Conformance for the optional backends does not run in CI today, since cargo test --workspace uses default features while only clippy passes --all-features. That predates this PR and applies to ghostty and rio equally, but it does mean these 53 cases are green locally rather than on a runner. Worth a follow-up.
• Unicode 11 is pinned deliberately, not incidentally: it is the only version that agrees with alacritty on every emoji width case measured. The table is in assets/xtermjs/README.md.

Review

Four reviewers went over this (shim correctness, Rust/FFI, architecture, supply chain). Three real divergences came back, all now fixed in the second commit and all reproduced before being touched:

• A split ST was answered with BEL. The scan that recovers a query's terminator waited for the \ of an ST, but xterm.js ends an OSC on the ESC before it, so a PTY read splitting between those two bytes answered the wrong terminator. Recording it on the ESC keeps the two in step.
• The terminator queue grew without bound. It recorded every OSC sequence while only colour ones claim what it records, so a shell that retitles the window on each prompt leaked an entry per prompt. Both the shim and supply-chain reviewers found this independently.
• OSC 4;1x wrote to slot 1. parseInt took the leading digit of an index that was not a number.

The first two are contract behaviour rather than quirks of this backend, so they went into the conformance suite: both fail on xterm.js without the fix and pass on alacritty, which is what makes them worth asserting of every backend.

Also fixed: a usize -> u32 scrollback cast that wrapped a deep request into a shallow one, and the vendored licence, which reproduced only one of the two packages' notices.

Verified clean by review, worth recording:

• Bundle provenance is byte-for-byte. Both vendored files match what npm publishes for @xterm/headless@6.0.0 and @xterm/addon-unicode11@0.9.0, reproducible with .github/scripts/vendor-xtermjs.sh.
• No path from terminal bytes to evaluated code. Only the three static assets are ever evaluated; PTY bytes reach JS as a byte array.

Considered and not done, happy to be overruled:

• A QuickJS memory cap. The one unbounded growth path was the queue above, and it is fixed; what remains is bounded by scrollback. A cap would need a number I have no evidence for, and one set too low breaks a legitimately deep session.
• Bell tracking. xterm.js exposes onBell, but build_with_bells falls through to build exactly as ghostty's does. Worth doing for both backends together rather than making them differ.

Review round two

Ten threads from cpendery (@cpendery), all addressed and resolved. Two of them landed the other way from the review; both are called out below with the reasoning, and I am happy to reopen either.

Two were real bugs, both reproduced before being touched:

• SGR 59 swallowed a genuine white underline. xterm.js stores the reset as an explicit RGB #ffffff, and every public getter then reports exactly what a real 58;2;255;255;255 reports, so reading the cell could only guess — and the old heuristic guessed "reset", meaning no cell could ever have a white underline. The shim now clears it where the parser sets it, while the two are still distinguishable. Verified the new conformance case fails on xterm.js with the old heuristic restored.
• JS exceptions were suppressed. process and resize discarded their Result entirely, so a throw froze the grid and every later read answered from it as fact. Faults are now recorded and reported by the next operation, through a new Emulator::fault checked in Engine::with_session — before the operation runs, since a wrong answer off a stale grid is worse than a failure. This is not hypothetical: the shim reads several private xterm.js fields, so a bundle upgrade moving one is exactly how it fires.

Two went against the review, so the reasoning is here rather than buried in a thread:

• Vendoring. I tried de-vendoring in 0fc1fbb and reverted it in 8ff376c: a build script put npm and a network on the path of anyone building from a clone, which is a heavier cost than keeping 264 KB out of git. The bundles stay committed, and .github/scripts/vendor-xtermjs.sh --latest bumps and re-fetches in one command.

• How a stale pin is noticed. A weekly pull request rather than a release-time check. A release gate fires on someone else's schedule — an upstream publish turns a ready tag into a failed release — and a version-string comparison cannot see the failure that actually matters: the shim reads several private xterm.js fields, so a release moving one is the real risk. The weekly job runs the conformance suite against the new bundles before opening the PR, which nothing else does, since CI builds with default features. It opens one only on a real new release, decided from the vendored bytes rather than pinned.json alone.

  I also tried and reverted a release-time provenance check (bcb3c65 → 50b5d7e): it re-asserted something already settled when the files were written, for a mismatch that could not reach the artifact, and put a network dependency on the release path. Nothing xterm.js-specific runs at release time now.

The rest:

• Added conformance cases for xterm.js's special cases: the underline reset above, and the isAttributeDefault() fast path where one call stands in for nineteen getters — that shortcut previously had a comment claiming it was verified with nothing checking it.
• Trimmed assets/xtermjs/README.md from 114 lines to 74, and moved the deferred-addons section to xterm.js backend: revisit addon-unicode-graphemes and addon-clipboard (OSC 52) #175.
• Dropped the Backend::ALL comment.
• Answered the raster.rs question (recent clippy lints, isolated in 9a68c1c) and the OSC 10-12 one (fixed earlier in 8f6520b, with two conformance cases).

Every new conformance case is a case, not a snapshot of current behaviour: each was verified to fail on xterm.js without its fix and pass on alacritty, ghostty, and rio.

Testing

53 conformance cases against the xterm.js backend, plus the full workspace suite (514 tests), cargo clippy --workspace --all-targets --all-features -- -D warnings, the no-default-features clippy pass, fmt, and the Python and Node binding suites. cargo publish --dry-run packages 94 files and compiles from the tarball.

Also verified that a clone builds with no network and no Node: cargo build -p tui-test-rs --features xtermjs succeeds with npm off PATH, which is the property the vendored bundles exist to preserve.

Driven end to end through a real PTY as well: title, cursor, colour assertions, and SVG screenshots all agree with alacritty.

[cpendery (cpendery)]

Overall looks good, I think we can put alacritty as the default & include the other backends as features. For the client libraries, I think it makes sense to include xtermjs & alacritty by default

[Ayman Bagabas (aymanbagabas)]

I think we can put alacritty as the default & include the other backends as features.

That's already how the crate is set up — tui-test-rs has default = [], so a plain cargo add tui-test-rs gets Alacritty only, and both ghostty and xtermjs are opt-in features. Nothing changed there in this PR.

For the client libraries, I think it makes sense to include xtermjs & alacritty by default

This PR does the additive half: xtermjs is now enabled for the CLI and both the Python and Node native packages, so --backend xtermjs and backend="xtermjs" work out of the box.

The other half — dropping Ghostty from the published bindings — I've deliberately left out, and I think it deserves its own PR. It's a change to what shipped packages contain rather than an addition, it touches a backend this PR doesn't otherwise go near, and it has a release-notes consequence for anyone already passing backend="ghostty". Folding it in here would put two unrelated stories in one diff.

It's also worth doing on its own merits, and the argument is stronger than just build times: Ghostty needs Zig 0.16 to build, and its upstream Zig build is the reason we don't publish Windows ARM64 artifacts at all today. xterm.js needs no external toolchain, so bindings carrying Alacritty + xterm.js could publish everywhere the CLI does. That looks like the real motivation for your suggestion, and it's a good one — I'd just rather land it where it can be reviewed as that change.

Happy to open it as a follow-up right after this, or to pull it in here if you'd rather have them together.

[cpendery (cpendery)]

I think we can put alacritty as the default & include the other backends as features.

That's already how the crate is set up — tui-test-rs has default = [], so a plain cargo add tui-test-rs gets Alacritty only, and both ghostty and xtermjs are opt-in features. Nothing changed there in this PR.

For the client libraries, I think it makes sense to include xtermjs & alacritty by default

This PR does the additive half: xtermjs is now enabled for the CLI and both the Python and Node native packages, so --backend xtermjs and backend="xtermjs" work out of the box.

The other half — dropping Ghostty from the published bindings — I've deliberately left out, and I think it deserves its own PR. It's a change to what shipped packages contain rather than an addition, it touches a backend this PR doesn't otherwise go near, and it has a release-notes consequence for anyone already passing backend="ghostty". Folding it in here would put two unrelated stories in one diff.

It's also worth doing on its own merits, and the argument is stronger than just build times: Ghostty needs Zig 0.16 to build, and its upstream Zig build is the reason we don't publish Windows ARM64 artifacts at all today. xterm.js needs no external toolchain, so bindings carrying Alacritty + xterm.js could publish everywhere the CLI does. That looks like the real motivation for your suggestion, and it's a good one — I'd just rather land it where it can be reviewed as that change.

Happy to open it as a follow-up right after this, or to pull it in here if you'd rather have them together.

based on our offline sync, I think we want to include all backends in the bindings.

[cpendery (cpendery)]

left some comments on some fixes, overall work looks really good.

nit: can we make sure the ci is actually running the xtermjs conformance tests? cargo test --workspace uses the default features so it'll skip the xterm tests even if clippy compiles them