Skip to content

feat(ai): Estimate LLM cache savings - #127023

Merged
vgrozdanic merged 2 commits into
feat/llm-cache-probe-queriesfrom
feat/llm-cache-pricing
Oct 13, 2026
Merged

vgrozdanic merged 2 commits into
feat/llm-cache-probe-queriesfrom
feat/llm-cache-pricing

Conversation

@vgrozdanic

@vgrozdanic vgrozdanic commented Oct 9, 2026 •

Copy link
Copy Markdown
Member

Adds savings estimates so LLM prompt-cache findings can be ranked by possible cost impact, not only token volume.

For call sites that are not caching, the estimate applies the difference between normal input price and cached input price to uncached tokens. This is an upper bound because aggregate telemetry cannot reveal exact shared prefix length. For cache thrash, the estimate applies the difference between cache write price and cache read price to cache creation tokens.

Pricing comes from Sentry's existing AI model metadata. Each estimate keeps the exact prices used so later reports remain understandable when provider prices change. Missing metadata, unknown models, absent prices, and findings with no recoverable cost produce explicit pricing gaps instead of invented savings.

validated finding + model metadata
→ select outcome formula
→ estimate savings for detection window
→ savings with price snapshot | pricing gap

This layer only calculates estimates. It does not fetch prompt content, schedule scans, emit shadow logs, or create issues.

@vgrozdanic
vgrozdanic added this pull request to stack #127016 October 9, 2026 12:23
@github-actions github-actions Bot added the Scope: Backend Automatically applied to PRs that change backend components label Oct 9, 2026
@vgrozdanic
vgrozdanic force-pushed the feat/llm-cache-pricing branch 4 times, most recently from 6c4cc1a to 0ce56d9 Compare October 13, 2026 05:17
@vgrozdanic
vgrozdanic marked this pull request as ready for review October 13, 2026 05:17
@vgrozdanic
vgrozdanic requested a review from a team as a code owner October 13, 2026 05:17
@vgrozdanic
vgrozdanic force-pushed the feat/llm-cache-pricing branch from 0ce56d9 to 022ecfe Compare October 13, 2026 07:16
@sentry

sentry Bot commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

Sentry Snapshot Testing

Name Added Removed Changed Renamed Unchanged Skipped Status
sentry-frontend
sentry-frontend
0 0 0 0 620 0 ✅ Unchanged

⚙️ sentry-frontend Snapshot Settings

@vgrozdanic
vgrozdanic merged commit 13ee87a into master Oct 13, 2026
66 checks passed
@vgrozdanic
vgrozdanic deleted the feat/llm-cache-pricing branch October 13, 2026 08:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Scope: Backend Automatically applied to PRs that change backend components

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants