Skip to content

fix(replays): Use handle_query_errors across replay endpoints - #125992

Merged
gggritso merged 4 commits into
masterfrom
fix/replays-segment-index-snuba-overload
Oct 7, 2026
Merged

gggritso merged 4 commits into
masterfrom
fix/replays-segment-index-snuba-overload

Conversation

@sentry-junior

@sentry-junior sentry-junior Bot commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

When the replays ClickHouse cluster hits its concurrent query limit (TOO_MANY_SIMULTANEOUS_QUERIES), replay endpoints return inconsistent statuses. Some let the error escape as a 500. The replay index and selector endpoints return a 400 through their own handled_snuba_exceptions decorator, which tells clients not to retry.

The main motivation to fix this is to improve consistency but also to improve alerting. We get Datadog alerts on elevated 500s but I don't want to be alerted due to Snuba timeouts!

This PR wraps the Snuba query calls in every replay endpoint with the shared handle_query_errors handler, which most Snuba-backed endpoints already use, and deletes the replay-specific handler.

Response Code Changes

Snuba error Before After
QueryTooManySimultaneous 400 (index/selector) or uncaught 500 429 (Throttled)
QueryMemoryLimitExceeded, QueryExecutionTimeMaximum 400 "Query limits exceeded" (index/selector) or 500 504 with the shared timeout message
QueryConnectionFailed 400 (index/selector) or 500 500, captured
Other 500 500 or the shared mapping

Exception Details

The replay count endpoint no longer returns str(e) for any ValueError. Invalid input gets a ParseError with a fixed message, and unexpected errors become a generic 500 so we don't leak any internal detail to Sentry users at runtime.

via George Gritsouk.

--

View Junior Session [Sentry]

Wrap the recording segment index endpoint in handle_query_errors so Snuba
errors like QueryTooManySimultaneous map to the shared API responses
(429 Throttled) instead of surfacing as uncaught 500s.

Co-Authored-By: George Gritsouk <george.gritsouk@sentry.io>
@github-actions github-actions Bot added the Scope: Backend Automatically applied to PRs that change backend components label Sep 30, 2026
Wrap Snuba queries in every replay endpoint with the shared
handle_query_errors handler and remove the replay-specific
handled_snuba_exceptions decorator. Snuba concurrency limits now return
429 everywhere instead of 400 or an uncaught 500, and memory/execution
limits return the shared 504 timeout response.

Co-Authored-By: George Gritsouk <george.gritsouk@sentry.io>
@sentry-junior sentry-junior Bot changed the title fix(replays): Return 429 for Snuba overload on recording segment index fix(replays): Use handle_query_errors across replay endpoints Sep 30, 2026
@gggritso
gggritso self-requested a review September 30, 2026 00:56

@sentry-junior sentry-junior Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

About the CodeQL "Information exposure through an exception" alert on organization_replay_count.py: this PR didn't introduce that code. return Response({"detail": str(e)}, status=400) is already on master (line 148). It only moved because it's now indented inside with handle_query_errors():. It returns the message from InvalidSearchQuery/ValueError, which reports invalid search input back to the caller, not a stack trace. Changing that response is out of scope here, so I'm leaving the code as is. A maintainer can dismiss the alert, or it can be handled in a separate PR.

@linear-code

linear-code Bot commented Sep 30, 2026 •

Copy link
Copy Markdown

VULN-3039

REPLAY-1009

@gggritso gggritso changed the title fix(replays): Use handle_query_errors across replay endpoints fix(replays): Use handle_query_errors across replay endpoints Sep 30, 2026
The replay-count endpoint returned str(e) for any ValueError raised while
building the query, so internal or library error text could reach the
client. Raise ParseError with fixed messages for user input validation in
replay_counts, and let unexpected ValueErrors surface as a generic 500.
Invalid search syntax still goes through handle_query_errors.

Co-Authored-By: George Gritsouk <george.gritsouk@sentry.io>
@gggritso

gggritso commented Oct 2, 2026

Copy link
Copy Markdown
Member

@cursor review

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit b348f74. Configure here.

@gggritso
gggritso marked this pull request as ready for review October 2, 2026 19:21
@gggritso
gggritso requested a review from a team October 2, 2026 19:21

@wmak wmak left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tyvm

@gggritso
gggritso merged commit ea5938b into master Oct 7, 2026
72 of 73 checks passed
@gggritso
gggritso deleted the fix/replays-segment-index-snuba-overload branch October 7, 2026 17:27

This branch was successfully deployed

1 active deployment
Preview — 12fbe2db Deployed Oct 7, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Scope: Backend Automatically applied to PRs that change backend components

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants