Skip to content

PgWaitSamplerLiveTests can count another test's lock wait: 31 samples from a 30-snapshot cycle under full-suite load #3939

Closed

Description

@erikdarlingdata

Summary

PgWaitSamplerLiveTests.OneCycleAgainstAStockTarget_LandsServiceSampledRowsTheReadDiscloses failed once in a full local Darling.Tests run:

Assert.InRange() Failure: Value not in range
Range:  (15 - 30)
Actual: 31
PgWaitSamplerLiveTests.cs(156)

It passed in two other full runs on the same build and 3 of 3 times when run alone. So it is a load flake, not a product regression. It was seen on the #3903 branch, which touches nothing in the wait sampler.

Likely cause

The assertion expects the test's own lock waiter to appear in at most SamplerSnapshotsPerCycle (30) snapshots. But PgWaitSamplingCollector.SamplerSnapshotSql reads pg_stat_activity cluster-wide: it has no datname filter, which is correct for the product. The class carries [Collection("live-postgres")], which serializes it against the other shared-store classes but not against the #1776 own-store classes. Those mint scratch databases on the same cluster and run migrations, TimescaleDB setup and bulk seeds in parallel.

A Lock: relation wait from any of them during the 30-second window adds a sample to the same (Lock, relation) row. Thirty-one samples from thirty snapshots is exactly one extra.

Possible fixes

  • Bound the assertion to the test's own waiter. For example, compare against a pid-scoped count, or assert the row is at least SnapshotsPerCycle / 2 and treat the upper bound as SnapshotsPerCycle plus the samples of backends that are not the waiter.
  • Or run the sampler test against a target the parallel own-store classes cannot reach.

Activity

  1. added
    in-progressActively being worked by a local session or its agents (PR open or in flight)
    on Sep 23, 2026
  2. erikdarlingdata commented on Sep 23, 2026

    @erikdarlingdata
    OwnerAuthor

    Closed by the watcher: delivered in PR #3990, merged to dev.

  3. removed
    in-progressActively being worked by a local session or its agents (PR open or in flight)
    on Sep 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions