Repository navigation
feat(tui): Hidden Ctrl+T HUD showing running/final LLM token cost - #783
Merged
Merged
Conversation
Adds an undiscoverable Ctrl+T toggle (not registered as a keyboard hint) that shows the current wizard run's accumulated token usage and USD cost estimate, updating live as the agent works and settling to the SDK's exact total_cost_usd once the run completes. Note: Ctrl+M (as originally asked) can't work here -- in terminal raw input Ctrl+M and Enter/Return are the same byte (0x0D), and this codebase never enables the Kitty keyboard protocol that could disambiguate them (checked start-tui.ts's render() call). Ctrl+T has no such collision and nothing else in the repo uses it. - token-pricing.ts: pricing table + cost formula extracted from the benchmark's CostTrackerPlugin into a shared module, so the live HUD's per-turn estimate and the --benchmark report can't drift apart. Behavior-identical refactor (cost-tracker.ts re-exports it locally). - WizardStore: tokenUsage/tokenHudVisible observation state. addTokenUsage accumulates each assistant turn's usage (mirrors TokenTrackerPlugin/CacheTrackerPlugin's extraction, minus SDK-retry dedup -- approximate by design for a debug HUD, not a billing record). setFinalTokenCostUsd reconciles to the SDK's authoritative total_cost_usd on completion, same trick CostTrackerPlugin.onFinalize uses. - agent-interface.ts: wires both into the existing assistant-message loop and completeWithSuccess -- no new code paths, just two more getUI() calls alongside the existing analytics capture. - ScreenContainer: the Ctrl+T useInput + TokenCostHud render live here (not on any screen), since it's the one component mounted for the whole process. contentHeight is budgeted for the HUD's one row so it can't push content past the terminal height. - Guarded all 6 existing 'any key dismisses/exits this screen' handlers (OutroScreen and friends) against key.ctrl/key.meta -- Ink calls every mounted useInput for a keypress with no stopPropagation, so without this, toggling the HUD on e.g. the outro screen would also dismiss it (and OutroScreen's dismissal chains into process.exit()). - exit-line.ts: the final tally is also echoed into post-exit scrollback unconditionally, so it's guaranteed visible even if the HUD was never opened or a screen transition raced the toggle.
🧙 Wizard CIRun the Wizard CI and test your changes against wizard-workbench example apps by replying with a GitHub comment using one of the following commands: Test all apps:
Test all apps in a directory:
Test an individual app:
Show more apps
Results will be posted here when complete. |
Separates the HUD from the screen content below it. Bumped contentHeight's budget from 1 to 2 rows to account for the extra line.
Two changes to the hidden Ctrl+T HUD:
1. Defaults visible for local/dev/test runs (WizardStore's
$tokenHudVisible now inits to IS_DEV instead of false), so
contributors see it without needing to know the shortcut. Still
defaults hidden in the published build. The panel now also names
its own shortcut ('Ctrl+T to hide'), so it's self-documenting
either way -- bumped ScreenContainer's contentHeight budget from 2
to 3 rows for the extra hint line.
2. Fixes a real pricing gap: the cost formula was a single flat table
(Sonnet rates) applied to every token regardless of which model
actually produced it. Two ways that's wrong in practice --
AgentConfig.modelOverride switches specific programs to Haiku (e.g.
source-map detection), and the SDK reports a model string on every
individual assistant turn because a subagent dispatched via the
Agent tool can run on a different model than the main session. A
Haiku turn priced as Sonnet would show a running estimate ~3-15x
too high.
token-pricing.ts now has Haiku/Sonnet/Opus price tables and
pricePerMtokForModel(), matched by substring so a dated version
suffix never breaks the match. agent-interface.ts reads
message.message.model (the BetaMessage's own model field) per turn
-- not the run's nominal model -- and threads it through a new
shared TokenUsageDelta type (WizardUI/InkUI/LoggingUI/store) into
computeTokenCostUsd. Unknown/missing model falls back to Sonnet
pricing, the same behavior as before this change.
Deliberately NOT threaded into the benchmark's CostTrackerPlugin --
MiddlewareContext has no model field to pass, and --benchmark always
measures the default flow, never a Haiku-overridden program. Left a
comment there; both paths still reconcile to the SDK's authoritative
total_cost_usd at the end regardless.
…seInput calls The 6 'any key dismisses/exits this screen' handlers each had their own inline if (key.ctrl || key.meta) return; guard, copy-pasted with the same comment. Centralizes that into a shared hook so: - The guard can't be forgotten on the next screen that needs this pattern (AuditAreaPane.tsx's unrelated 'o' key handler shows this mistake is easy to make -- it checks input but not key.ctrl, so a hypothetical Ctrl+O would trigger it; not fixed here, out of scope, but it's a live example of the risk). - isIgnoredForDismiss is a pure, directly unit-tested predicate, matching how keyboard-hints-utils.ts's matchesKey is tested -- Ink-wiring itself stays untested, consistent with the rest of this hooks directory. No behavior change for any of the 6 screens.
The exit-line tally previously showed unconditionally whenever a run produced any usage, regardless of whether the user ever saw the HUD. That meant a production run (where the HUD defaults hidden) would still get a cost number printed to scrollback out of nowhere. tokenCostLine now checks store.tokenHudVisible first -- the tally only survives into scrollback if the panel is actually showing at exit (which defaults on in dev/test via IS_DEV, off in production, and is always overridable with Ctrl+T either way). Also: 'Ctrl+T to hide' now shares the cost line's row when there's room, right-aligned with a minimum 2-column gap, falling back to its own row on a narrow terminal. tokenCostHudRowCount is exported so ScreenContainer's height budget (1 HUD row + spacer, or 2 + spacer) can never disagree with what TokenCostHud actually renders -- both call the exact same function with the exact same inputs.
The previous pricePerMtokForModel matched by family-name substring
(model.includes('opus')/('haiku')), which silently mis-prices any
model that shares a family name but not a price: Claude Opus 4.5 is
$5/$25 per Mtok, half of what the table charged (it had Opus 4.1's
$15/$75 rate hardcoded), and Claude Sonnet 5 is $2/$10, cheaper than
Sonnet 4.6's $3/$15 -- both would have been silently mis-priced by
the substring match.
Replaced with an exact-match table keyed by the *undated* model id
(e.g. 'claude-haiku-4-5', not 'claude-haiku-4-5-20251001') --
pricePerMtokForModel strips a trailing 8-digit release-date suffix
before falling back to the table, since Anthropic's dated and undated
ids for the same version always carry the same price, verified for
every entry against https://models.dev/api.json. A model id that's
given but not in the table now returns undefined (computeTokenCostUsd
treats that as $0 for the delta) rather than guessing from a family
name -- the live estimate can undercount for an unrecognized model,
but never mis-prices one the way the substring match did; the final
number is unaffected either way since it reconciles to the SDK's
total_cost_usd regardless.
Also dropped the unused PRICE_PER_MTOK export (nothing outside this
file referenced it) and rounded pricingFromInput's derived cache rates
to 6dp to avoid binary float noise (3 * 0.1 = 0.30000000000000004).
Considered adopting tokenlens (an existing npm package wrapping
models.dev) instead of hand-maintaining this table, but its published
static catalog doesn't have any of the three model ids this wizard
actually uses -- only models.dev's *live* API does, which would mean
a network dependency for what's a hidden debug HUD. Kept the local
table; decision documented in the module's doc comment.
Also removes a handful of tests added earlier in this branch that
were verification scaffolding rather than real regression coverage:
duplicate assertions of already-covered code paths, and a couple of
near-tautological checks (0 tokens costs $0; a longer string needs
more width).
No behavior change -- four cleanups across the branch's own diff: - agent-interface.ts: extracted the 'assistant' case's inline usage- extraction (type cast + field mapping) into a named extractTokenUsageDelta() helper, matching this file's existing pattern of small private helpers above handleSDKMessage. Shrinks the case block from ~30 lines to 2. - TokenCostHud.tsx: tokenCostLineParts() was being computed twice per render (once directly, once again inside the tokenCostHudRowCount() call used to decide inline vs. two-row layout), each independently recomputing lineLength. Consolidated into one layoutTokenCostLine() that both the exported row-count helper and the component now share. - store.ts / TokenCostHud.tsx / exit-line.ts: the "sum all four token counters" formula was duplicated verbatim in TokenCostHud and exit-line. Both already import from store.ts, so added totalTokenCount(usage) there instead of introducing a new shared module (which would've needed a type-only import back into token-pricing.ts, adding a cycle for no real benefit). - token-pricing.ts: computeTokenCostUsd took 7 positional parameters, several the same type -- an easy mis-ordering hazard at call sites. Converted to a single options object shaped like TokenUsageDelta, so store.ts's addTokenUsage can now just pass computeTokenCostUsd(delta) directly instead of re-listing its fields in a fixed order. cost-tracker.ts's two call sites and the test file's calls updated to match; also dropped its local computeCost alias in favor of calling computeTokenCostUsd directly, since it only had two call sites.
Sonnet 5's $2/$10 rate isn't its permanent price -- it's a promotional discount that reverts to Sonnet 4.6's rate ($3/$15) on 2026-09-01. Hardcoding $2/$10 as a static table entry would have silently undercharged every Sonnet 5 turn forever after the promo ends, exactly the kind of staleness this table is meant to avoid. Pulled 'claude-sonnet-5' out of the static PRICE_TABLE into a small sonnet5Price(now) helper that checks the cutoff date, wired through pricePerMtokForModel's new optional `now` param (defaults to the real clock; injectable so the promo-window tests are deterministic instead of depending on whichever date the suite happens to run on). Also fixed a comment that had crept 1-2 characters past this repo's 80-char printWidth.
edwinyjlim
approved these changes
Jul 1, 2026
edwinyjlim
left a comment
Member
There was a problem hiding this comment.
this is really slick!
ty for making it so isolated and excluding it from prod build ❤️
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Not a big problem, but in development, I'm curious about the costs of a wizard run. Haven't found a way to see it.
Changes
Proposing a token cost bar at the top, default on in development, hidden in prod. Although in both cases toggleable with Ctrl+T toggle. Shows the current wizard run's accumulated token usage and USD cost estimate.
Verification
Had Claude do a bunch of tests. Ran this locally.