Skip to content

feat: split TokenUsage output tokens - #22

Open
Miss-you wants to merge 1 commit into
mainfrom
you-17-tokenusage-output-split
Open

Miss-you wants to merge 1 commit into
mainfrom
you-17-tokenusage-output-split

Conversation

@Miss-you

Copy link
Copy Markdown
Owner

Summary

  • split provider.TokenUsage output accounting into output_other and output_reasoning
  • add TotalOutput() and update totals, providers, aggregators, and CLI tables to use the new shape
  • update package and end-to-end tests to assert the renamed JSON contract

Changes

  • provider/provider.go: rename Output to OutputOther, add OutputReasoning, add TotalOutput(), and route Total() through it
  • provider/claude, provider/codex, provider/cursor, provider/kimi: store parsed output tokens in OutputOther and update assertions
  • cmd, stats, and e2e: carry the split fields through aggregation/output and update JSON contract checks

Validation

  • make fmt
  • make vet
  • make test
  • make build
  • make lint skipped locally because golangci-lint is not installed in this environment
  • manual CLI: ./bin/codetok daily --json --all --kimi-dir $(pwd)/e2e/testdata/sessions --claude-dir $(pwd)/.tmp-empty --codex-dir $(pwd)/.tmp-empty --cursor-dir $(pwd)/.tmp-empty
  • manual CLI: ./bin/codetok session --json --kimi-dir $(pwd)/e2e/testdata/sessions --claude-dir $(pwd)/.tmp-empty --codex-dir $(pwd)/.tmp-empty --cursor-dir $(pwd)/.tmp-empty

Risks / Notes

  • Provider parsers still emit output_reasoning = 0 until the follow-up reasoning-parsing work lands
  • Existing CLI Output columns now read OutputOther; the separate reasoning column remains follow-up work

Copilot AI review requested due to automatic review settings March 27, 2026 13:18

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the token accounting model to split output tokens into output_other and output_reasoning, and propagates that shape through provider parsers, aggregation, and JSON/e2e contract tests.

Changes:

  • Renamed TokenUsage.Output to TokenUsage.OutputOther, added TokenUsage.OutputReasoning, and introduced TotalOutput() with Total() routing through it.
  • Updated providers (Claude/Codex/Cursor/Kimi) and stats aggregation to accumulate the split output fields.
  • Updated CLI-related logic and unit/e2e tests to validate the new JSON contract (output_other / output_reasoning) and reject legacy output.

Reviewed changes

Copilot reviewed 16 out of 16 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
provider/provider.go Splits output token fields; adds TotalOutput() and updates totals.
provider/provider_test.go Adds tests for new JSON contract and totals behavior.
provider/claude/parser.go Routes parsed output tokens into OutputOther.
provider/claude/parser_test.go Updates assertions to OutputOther.
provider/codex/parser.go Routes parsed output tokens into OutputOther.
provider/codex/parser_test.go Updates assertions to OutputOther.
provider/cursor/parser.go Routes parsed output tokens into OutputOther.
provider/cursor/parser_test.go Updates assertions to OutputOther.
provider/kimi/parser.go Routes wire output tokens into OutputOther.
provider/kimi/parser_test.go Updates assertions to OutputOther.
stats/aggregator.go Aggregates OutputOther + OutputReasoning.
stats/aggregator_test.go Updates aggregation assertions to OutputOther.
cmd/session.go Updates session table values/totals to new fields.
cmd/daily.go Updates token merging and group-share output value to OutputOther.
cmd/daily_test.go Updates dashboard test fixtures to new field names.
e2e/e2e_test.go Updates JSON contract assertions to require split output fields and reject legacy output.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread cmd/daily.go
Comment on lines 478 to 481
formatTokenByUnit(g.TokenUsage.InputOther, unit),
formatTokenByUnit(g.TokenUsage.Output, unit),
formatTokenByUnit(g.TokenUsage.OutputOther, unit),
formatTokenByUnit(g.TokenUsage.InputCacheRead, unit),
formatTokenByUnit(g.TokenUsage.InputCacheCreate, unit),

Copilot AI Mar 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This table column is labeled "Output" (see header setup earlier in this function), but the value printed is OutputOther. Once OutputReasoning is populated, totals will no longer reconcile with the displayed Output column. Rename the header to reflect "OutputOther" (or show TotalOutput() / add a reasoning column).

Copilot uses AI. Check for mistakes.
Comment thread provider/provider_test.go
Comment on lines +22 to +33
got := string(b)

if strings.Contains(got, `"output":`) {
t.Fatalf("json should not contain legacy output field: %s", got)
}
if !strings.Contains(got, `"output_other":40`) {
t.Fatalf("json missing output_other field: %s", got)
}
if !strings.Contains(got, `"output_reasoning":50`) {
t.Fatalf("json missing output_reasoning field: %s", got)
}

Copilot AI Mar 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These JSON contract assertions are string-based and can be brittle (they depend on exact marshaling formatting and can miss cases like different numeric values/types). Prefer unmarshaling into a map[string]any / struct and asserting keys and values directly (including verifying the legacy output key is absent).

Suggested change
got := string(b)
if strings.Contains(got, `"output":`) {
t.Fatalf("json should not contain legacy output field: %s", got)
}
if !strings.Contains(got, `"output_other":40`) {
t.Fatalf("json missing output_other field: %s", got)
}
if !strings.Contains(got, `"output_reasoning":50`) {
t.Fatalf("json missing output_reasoning field: %s", got)
}
var decoded map[string]any
if err := json.Unmarshal(b, &decoded); err != nil {
t.Fatalf("unmarshal TokenUsage json: %v", err)
}
if _, ok := decoded["output"]; ok {
t.Fatalf("json should not contain legacy output field: %v", decoded)
}
outputOther, ok := decoded["output_other"]
if !ok {
t.Fatalf("json missing output_other field: %v", decoded)
}
if v, ok := outputOther.(float64); !ok || v != 40 {
t.Fatalf("output_other = %v, want 40", outputOther)
}
outputReasoning, ok := decoded["output_reasoning"]
if !ok {
t.Fatalf("json missing output_reasoning field: %v", decoded)
}
if v, ok := outputReasoning.(float64); !ok || v != 50 {
t.Fatalf("output_reasoning = %v, want 50", outputReasoning)
}

Copilot uses AI. Check for mistakes.
Comment thread cmd/session.go
Comment on lines 128 to 132
truncate(s.Title, 40),
s.TokenUsage.TotalInput(),
s.TokenUsage.Output,
s.TokenUsage.OutputOther,
s.TokenUsage.Total(),
)

Copilot AI Mar 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The session table still labels the column as "Output", but the values printed here are OutputOther (and the total includes OutputReasoning via Total()). This makes the table misleading once OutputReasoning becomes non-zero. Update the header (e.g., "OutputOther" / "Output (other)") and/or print TotalOutput() in this column to match the label/total semantics.

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants