fix(rule-flood-guard): scope flood detection and rule disabling per tenant - #2495
Merged
yllada merged 2 commits intoAug 20, 2026
Merged
Conversation
…2_rule_flood_guard_multitenant
🛑 AI review — Sensitive area, extra care recommendedThis PR touches critical paths or introduces changes the model cannot judge with sufficient confidence. Review carefully before merging. ✅
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The problem
rule-flood-guardwatches for correlation rules that flood the alert list anddisables them. It was doing both halves of that job across tenant boundaries.
Counting. The ClickHouse aggregation grouped by
nameanddataSourceonly. The SDK driver renders those fields as a literal
GROUP BY, so everytenant's alerts landed in the same bucket:
Disabling. Both backend calls carried
X-Internal-Keyand noX-Tenant-Id. The middleware treats an internal caller without that header astenantless, and
engine_config.gomaps an empty tenant to the platform tenant.Since
RuleStore.SetEnabledwrites into a per-tenantDisabledRuleslist, thedisable was recorded against the platform tenant while the tenant that actually
flooded kept flooding.
search-by-filtershad the mirror image of the same problem: with no tenant itreturns every tenant's rules, so a rule name shared across tenants could get
another customer's rule disabled, and the
ruleActiveflag was read from thewrong tenant's view — skipping rules that were live and flooding for a customer
because the platform tenant already had them off.
What changed
aggregation.gotenantIdadded as the firstGROUP BYfield. NewbucketsFromGroupsflattens the nested tree by reading levels throughGroup.Fieldrather than by depth.backend.goX-Tenant-Idset on rule lookup, deactivation and notification.Notifyreaches the offending tenant, with a copy to the platform tenant unless they are the same.guard.goREADME.mdTwo smaller corrections came along with it. The notification now names the
offending tenant and reports the observed count instead of the threshold, and
it interpolates the configured
windowHours— the template said "in the last24h" unconditionally, which misreported the period whenever an operator changed
the window.