Skip to content

[FEATURE] Add a "per‑event" notification option (vs. the batched per‑cycle summary) for deadlocks & blocking #1141

Closed

Description

@gotqn

Which component(s) does this affect?

  • Full Dashboard
  • Lite
  • SQL collection scripts
  • Installer
  • Documentation

Problem Statement

Summary:

Deadlock and blocking alerts currently aggregate all recent events in the window into a single notification (one tray/email/webhook card with N detail blocks, and the detail list appears capped). There's no way to receive one notification per event, which teams need for per‑incident tracking (e.g., one Azure DevOps ticket per deadlock). Downstream automation can't split the batched payload reliably because the per‑event details are flattened into repeated name/value facts rather than a structured array.

Current behavior:

  • BuildDeadlockContextAsync / BuildBlockingContextAsync (Lite MainWindow.xaml.cs:1668,1739; Dashboard equivalents) build one AlertContext from GetRecentDeadlocksAsync/…Blocking(hoursBack: 1) and send a single alert (TrySendAlertEmailAsync).
  • The card shows Current Value = N but lists only a few events (apparent Take(3) cap), so high‑volume incidents are truncated.

Proposed Solution

Proposed:

  1. A notification mode setting (global and/or per‑server): Summary (current default) | Per‑event (one notification per deadlock/blocking event).
  2. In Per‑event mode, a configurable max‑per‑cycle cap (e.g. default 10) to prevent floods during a storm, with a final "+X more" summary if the cap is hit.
  3. Each per‑event message should carry a stable per‑event identifier/fingerprint (the involved objects — ties into the "stable dedup fingerprint" request) so downstream dedup/ticketing is per incident.
  4. Apply to deadlocks and blocking first; consider extending to long‑running query.

Keep: Summary as the default; Per‑event is opt‑in. Existing cooldown/gating still applies per event.

Use Case

I want to be able to create an issue about particular problem and if the message is grouping and not showing all errors, I can miss something.

Why it matters:

  • One ticket/notification per incident is impossible: the webhook delivers one payload with multiple events as repeated Victim SQL/Processes facts — not parseable into distinct events without fragile heuristics.
  • Truncation hides events beyond the cap.

Alternatives Considered

No response

Additional Context

No response

Activity

  1. added a commit that references this issue on Jun 17, 2026
  2. erikdarlingdata commented on Jun 17, 2026

    @erikdarlingdata
    Owner

    The grouping + per-incident fingerprint half of this is implemented and merged to dev (so it's in tonight's nightly); the per-event delivery mode itself is the next step.

    In the nightly now:

    Still to come (the headline ask here):

    • The Summary | Per-event notification mode toggle + the configurable per-cycle cap. The grouping above is the foundation it sits on, so this is a focused follow-on.

    So you can already test per-incident dedup/counter behavior against the grouped fingerprints in tonight's nightly with your Logic App; one-notification-per-event delivery will follow. Leaving this open to track that piece.

  3. added a commit that references this issue on Jun 18, 2026
  4. erikdarlingdata commented on Jun 18, 2026

    @erikdarlingdata
    Owner

    Update: the per-event delivery mode itself is now implemented and merged to dev too — so it'll be in tonight's nightly.

    So you can flip on Per-event mode in the nightly and have your Logic App open/track one ticket per incident, counting recurrences by the Dedup Key.

  5. erikdarlingdata commented on Jun 18, 2026

    @erikdarlingdata
    Owner

    Verification update: built tonight's dev locally and confirmed the per-event delivery mode works end-to-end —

    • Settings → Alerts now shows Deadlock/blocking delivery: Summary | Per-event with a max per cycle cap (default 10), defaulting to Summary. Confirmed rendering in the running app.
    • Full unit suite green (984 tests, incl. the new per-event split coverage); both apps build clean.

    So it's good to go in tonight's nightly to test against your Logic App. The per-server override remains the one follow-up.

  6. added 2 commits that reference this issue on Jun 18, 2026
  7. erikdarlingdata commented on Jun 25, 2026

    @erikdarlingdata
    Owner

    Per-event notification mode shipped on dev: AlertNotificationMode + PerEventNotification, Settings UI in both apps, default Summary, per-cycle cap. The remaining optional per-server override is tracked separately in #1236. Closing the core request; lands on main with 3.1.0.

  8. added 5 commits that reference this issue on Jun 25, 2026
  9. added a commit that references this issue on Jul 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions