Background
The DP currently logs cache_status=hit|miss on every chat request. The dashboard /usage page can show "N hits today" but cannot answer the only question operators actually care about:
"How much did caching save me?"
There is no way today to attribute saved tokens or saved USD to cache hits. Without that number every other piece of cache work (TTL tuning, applies_to scoping, semantic cache, etc.) is unjustifiable to finance / leadership.
Proposed fix
On a cache hit, the DP already has the cached ChatResponse, which carries usage.prompt_tokens / completion_tokens (and the four detail counters added for cache pricing). Emit two extra fields on the cache-hit telemetry event:
tokens_saved_input = cached usage.prompt_tokens
tokens_saved_output = cached usage.completion_tokens
cost_saved_usd = price_table.lookup(model) applied to the saved tokens
The price table already exists in cp-api (internal/dpmgr/dpstore/pricing.go); the DP either receives a snapshot via the existing config watcher or computes from the resolved aisix_core::Model pricing fields, whichever is already wired up for non-cache cost reporting.
Where the fields land
- DP structured log: hit event gets
tokens_saved_input, tokens_saved_output, cost_saved_usd fields.
- Existing telemetry exporters (OTel, Langfuse) receive them automatically as part of the same span / event.
- Usage rollup table (
requests / usage_daily in cp-api) gains a cost_saved_usd column populated from the hit events; a follow-up dashboard PR adds the "Saved this week: $X" tile to /usage.
Why this delivers value
- Direct ROI dashboard — converts the abstract "hits" counter into a dollar number that sells the product internally.
- Operations decisions become data-driven — TTL tuning, policy
applies_to scope, semantic-cache adoption all become "did $ saved go up?" instead of guesswork.
- Per-tenant cost attribution — group
cost_saved_usd by api_key_id and the dashboard can show "we saved you $Y this month" on customer-facing billing pages.
Acceptance criteria
Out of scope
- Dashboard wiring of the new field into a "Saved this week" tile — separate cp-api + dashboard PR after the DP fields land.
- Per-token-class differential pricing (cached_prompt vs reasoning) — first cut uses base prompt + completion rates, refinement is a follow-up.
Background
The DP currently logs
cache_status=hit|misson every chat request. The dashboard/usagepage can show "N hits today" but cannot answer the only question operators actually care about:There is no way today to attribute saved tokens or saved USD to cache hits. Without that number every other piece of cache work (TTL tuning,
applies_toscoping, semantic cache, etc.) is unjustifiable to finance / leadership.Proposed fix
On a cache hit, the DP already has the cached
ChatResponse, which carriesusage.prompt_tokens/completion_tokens(and the four detail counters added for cache pricing). Emit two extra fields on the cache-hit telemetry event:tokens_saved_input= cachedusage.prompt_tokenstokens_saved_output= cachedusage.completion_tokenscost_saved_usd=price_table.lookup(model)applied to the saved tokensThe price table already exists in cp-api (
internal/dpmgr/dpstore/pricing.go); the DP either receives a snapshot via the existing config watcher or computes from the resolvedaisix_core::Modelpricing fields, whichever is already wired up for non-cache cost reporting.Where the fields land
tokens_saved_input,tokens_saved_output,cost_saved_usdfields.requests/usage_dailyin cp-api) gains acost_saved_usdcolumn populated from the hit events; a follow-up dashboard PR adds the "Saved this week: $X" tile to/usage.Why this delivers value
applies_toscope, semantic-cache adoption all become "did $ saved go up?" instead of guesswork.cost_saved_usdbyapi_key_idand the dashboard can show "we saved you $Y this month" on customer-facing billing pages.Acceptance criteria
tokens_saved_input,tokens_saved_output,cost_saved_usd(computed from the cached usage + the model's price entry).usage× pricing.Out of scope