Skip to content

[BUG] 3.5.0->3.6.0 Deadlocks #2677

Closed
Closed

Description

@ZedZipDev

Component

Lite

Performance Monitor Version

3.6.0

SQL Server Version

Azure SQL DB

Windows Version

W11

Describe the Bug

After 3.5.0->3.6.0 Update I opened the Blocking->Deadlocks page and see only 2 deadlocks records and in fact it is 1 deadlock
2026-08-21 Victim
2026-08-21

But there are more deadlocks , I retrieve them using query.
I have "Last 7 days" so it looks like PMLite did not detect any deadlocks after 21.08 and shows me only these 2 rows.
But it (3.5.0) worked 24hrs/7.
By the way we discussed it in other issue. It seems the PMLite does not detect deadlocks anymore.

Let me know what additional details can I provide.
Thanx.

Steps to Reproduce

Run PMLIte, go to Blocking->Deadlocks

Expected Behavior

Deadlocks detected

Actual Behavior

No deadlocks

Error Messages / Log Output


Screenshots

Image Image Image

Additional Context

No response

Activity

  1. erikdarlingdata commented on Aug 28, 2026

    @erikdarlingdata
    Owner

    Found it, and thank you for the persistence — this is a real bug, and it is not a 3.5.0→3.6.0 regression (the deadlock code barely changed between them). The timing was a coincidence.

    What actually happened. On Azure SQL DB our deadlock capture is a database-scoped Extended Events session. After an Azure failover or scale operation, that session can end up listed in sys.database_event_sessions but missing from sys.dm_xe_database_sessions — it exists, but the ring-buffer reader can't see it. That state is transient (it clears once the failover settles). Our recovery code recreated the session once and, if that didn't restore visibility on the spot, set an internal 'gave up' flag — and that flag was never cleared until the app restarted. So one transient Azure blip on ~Aug 21 stopped capture, and because PMLite runs 24/7 for you, it never got the restart that would have cleared it. Blocked-process capture rides the same code path, so it went quiet too.

    Immediate workaround: restart PMLite. That clears the flag, the session gets recreated cleanly, and deadlock (and blocked-process) capture should resume right away. If it does, that also confirms this diagnosis — I'd appreciate you letting me know either way.

    The fix (#2678): the permanent 'gave up' flag becomes a 30-minute cooldown that then re-attempts, and it clears the instant the session reads back as visible — so it self-heals without needing a restart. Ships in the next release. The reason it needed the flag at all (not thrashing DROP/CREATE every cycle, which would wipe captured-but-unread events) is preserved by the cooldown.

  2. added a commit that references this issue on Aug 28, 2026
  3. ZedZipDev commented on Aug 28, 2026

    @ZedZipDev
    Author

    Thank you @erikdarlingdata , I am sure it will be fixed as usually .
    Btw, one thing: PMLite 3.5.0 is active, I have clicked , the update was found, downloaded and then suggested me to update (PMLite 3.5.0 is still active), then it was restarted and my "Last 7 days" -> "Last 4 hrs". I have set "Last 7 days" again, exited and run the PMLite 3.6.0 again. It shows "Last 7 days".

  4. erikdarlingdata commented on Aug 28, 2026

    @erikdarlingdata
    Owner

    Confirmed in code and closing as fixed. Deadlock capture goes through the same EnsureDatabaseScopedXeSessionsAsync recreate path (RemoteCollectorService.Deadlocks.cs) that carried the permanent 'gave up' flag — the exact one #2678 turned into a self-healing 30-minute cooldown (the flag now clears the instant the session reads back as visible, and re-attempts after the cooldown instead of staying dark until an app restart). So the deadlock-detection outage this issue reports is resolved by #2678 (merged to dev, ships in the next release); blocked-process capture rode the same latch and is fixed by the same change.

    Until that release ships, a PMLite restart clears the flag and resumes capture immediately. Thanks again for the persistence — please reopen if you still see gaps after updating to the next release.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions