Convert the Viewer read-path teardowns; ratchet 87 -> 19 (#1902, batch 2 of 3)

CHANGELOG.md

@@ -203,6 +203,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
   **The two gotchas [#1896] predicted both recurred, at scale.** Ten teardown helpers across six classes fetched `TestContext.Current.CancellationToken` internally, which under `LiveStoreCleanup` - deliberately `CancellationToken.None`, so a cancelled run still restores the store - would have re-signalled itself on exactly the path the helper exists to survive; they take a token now. Two classes already opened their own cleanup connection and so were half-right, but still threw from the `finally` and still used the test's own token; `LiveStoreCleanup` supplies both halves and the hand-rolled connection goes with it. One teardown deletes through the mute-rule STORE, which is the API under test and owns its own data source, so it takes `RunOwnedAsync` - it was never exposed to the closed-connection half and only needs the masking rule. **A one-number ratchet lands with the batch**: `LiveCleanupConversionRatchetTests` counts what remains and fails both if the count RISES (someone adds a new one behind the batches) and if it FALLS without the ceiling coming down (a batch leaving slack that new ones could be spent on). A named allowlist of eighty entries would be a merge-conflict generator that every batch edits by hand; one number cannot be partially wrong. When it reaches zero the assertion becomes the real invariant and [#1902] closes. Test-suite only - no shipped code changed.
 
+- **The whole Viewer read-path test family now reports its own failures too, and the ratchet drops to 19** ([#1902], batch two of three) - 68 teardowns across the 21 `Viewer*` live classes, taken as one tranche because they are one shape: plant rows for a viewer query, read them back, delete them in a `finally` that ran on the body's connection. **Every one of the 21 had the token gotcha** - 25 delete helpers reaching for `TestContext.Current.CancellationToken` internally, which under `LiveStoreCleanup` (deliberately `CancellationToken.None`, so a cancelled run still restores the store) re-signals itself on precisely the path the helper exists to survive. None of those files declares a local `ct` at all, so their body call sites now pass the ambient token explicitly rather than introducing a mixed idiom inside one method. What is left for batch three is the 19 Darling engine sites, including the one teardown that cleans up through an `NpgsqlDataSource` rather than a connection - deliberately isolated so it gets read properly instead of being buried in a sixty-eight-site diff.
+
+- **A compression test that asserts a job has NEVER RUN stopped losing to the job actually running** ([#1888]'s predicted follow-through, found by a batch-two run) - `StuckCompressionJobsSql_NeverRunJob_ReadsNullLastRunStartedAt` pins the [#1760] sentinel: `timescaledb_information.job_stats.last_run_started_at` reads `-infinity`, not NULL, for a job that has never executed, and the production query has to neutralise that or the stuck-Running arm fires on a job that never ran. The test created its compression policy UNPARKED, and `add_compression_policy` creates the job SCHEDULED with no `initial_start` so TimescaleDB launches it within a second or two ([#1788]) - one background launch destroys the sentinel outright, and unlike a chunk count no amount of re-reading recovers it. [#1889] fixed exactly this for three sibling tests by creating and parking in ONE transaction, so the scheduler (a separate backend) cannot see the job until the row already reads `scheduled = false`; this was the fourth site, and it now uses the same helper. [#1888] called it: raising CI's worker slots "will make background jobs launch reliably for the first time, so any other test that has been quietly relying on the scheduler being unable to run is going to start failing ... whether others exist is unknown, and finding out is the actual work here." **Proven both ways rather than by re-running until green**: with the old unparked creation and a deliberate four-second scheduler window the assertion fails as `Expected: True / Actual: False` - byte-identical to the intermittent full-suite failure - and with the parked creation and the same window it passes every time. No permanent guard is added, for cause: three other live tests in the same file create unparked policies BY DESIGN (an already-armed legacy policy is what the converge tests converge), so any source-level rule would either fire on those or be narrowed until it pinned nothing.
+
 ## [3.3.0] - 2026-07-29
 
 ### Important

Darling/Darling.Tests/LiveCleanupConversionRatchetTests.cs

@@ -42,10 +42,12 @@ public sealed class LiveCleanupConversionRatchetTests
     ///
     /// <para>126 when #1902 was filed; 87 after batch one (the config-writing and server-registering classes,
     /// which were the highest blast radius — a leaked <c>config_mute_rules</c> row mutes another test's alerts
-    /// and a leaked <c>collect.servers</c> row is a phantom server in every fleet read). Batches two and three
-    /// take the rest.</para>
+    /// and a leaked <c>collect.servers</c> row is a phantom server in every fleet read); 19 after batch two,
+    /// which took the whole <c>Viewer*</c> read-path family. What is left is the Darling engine classes,
+    /// including the one teardown that cleans up through an <c>NpgsqlDataSource</c> rather than a connection —
+    /// deliberately isolated into batch three rather than buried in a sixty-eight-site diff.</para>
     /// </summary>
-    private const int Ceiling = 87;
+    private const int Ceiling = 19;
 
     [Fact]
     public void UnconvertedLiveTeardowns_NeverExceedTheRatchet()

Darling/Darling.Tests/TimescaleSupportTests.cs

@@ -1482,7 +1482,15 @@ await ExecuteAsync(connection,
                this test is the catalog's behaviour rather than a second dialect of the same DDL. */
             await ExecuteAsync(connection, TimescaleSupport.CreateHypertableSql(table, "collection_time"), ct);
             await ExecuteAsync(connection, TimescaleSupport.EnableCompressionSql(table), ct);
-            await ExecuteAsync(connection, TimescaleSupport.AddCompressionPolicySql(table), ct);
+
+            /* PARKED, in one transaction — the same lever #1889 pulled for the three compression tests, and
+               this one needs it MORE than they do. Their assertions are about which chunks a run compressed;
+               this one asserts the job has NEVER RUN (last_run_started_at = '-infinity', the #1760 sentinel),
+               which a single background launch destroys outright and no amount of re-reading recovers. An
+               unparked add_compression_policy creates the job SCHEDULED and TimescaleDB launches it within a
+               second or two (#1788), so the window between creating it and parking it below was the whole
+               defect: observed failing as Expected: True / Actual: False on a full-suite run, twice. */
+            await AddCompressionPolicyParkedAsync(connection, table, ct);
 
             long jobId;
             using (var find = new NpgsqlCommand(

Darling/Darling.Tests/ViewerBlockingDepthTests.cs

@@ -359,11 +359,12 @@ public async Task BprRead_WidenedColumns_ReportXml_LongBlock_SourceBadge_Against
         using var connection = new NpgsqlConnection(connectionString);
         await connection.OpenAsync(TestContext.Current.CancellationToken);
         await PgMigrations.MigrateAsync(connection, TestContext.Current.CancellationToken);
-        await DeleteRowsAsync(connection, "blocked_process_reports", BprServerId);
-        await DeleteRowsAsync(connection, "dmv_blocking_snapshots", BprServerId);
+        await DeleteRowsAsync(connection, "blocked_process_reports", BprServerId, TestContext.Current.CancellationToken);
+        await DeleteRowsAsync(connection, "dmv_blocking_snapshots", BprServerId, TestContext.Current.CancellationToken);
 
         await using var viewer = new ViewerDataService(connectionString!);
 
+        var bodySucceeded = false;
         try
         {
             var collectionTime = TruncateToSeconds(DateTime.UtcNow.AddMinutes(-10));
@@ -397,11 +398,16 @@ await InsertBlockedProcessReportAsync(connection, BprServerId, collectionTime, c
             Assert.False(shortBlock.IsLongBlock);
             Assert.Equal("500 ms", shortBlock.WaitTimeFormatted);
             Assert.False(shortBlock.HasReportXml);
+
+            bodySucceeded = true;
         }
         finally
         {
-            await DeleteRowsAsync(connection, "blocked_process_reports", BprServerId);
-            await DeleteRowsAsync(connection, "dmv_blocking_snapshots", BprServerId);
+            await LiveStoreCleanup.RunAsync(connectionString!, bodySucceeded, async (cleanup, cleanupCt) =>
+            {
+                await DeleteRowsAsync(cleanup, "blocked_process_reports", BprServerId, cleanupCt);
+                await DeleteRowsAsync(cleanup, "dmv_blocking_snapshots", BprServerId, cleanupCt);
+            });
         }
     }
 
@@ -415,11 +421,12 @@ public async Task BprRead_DmvFallback_SurfacesWhenNoXe_AgainstDevPostgres()
         using var connection = new NpgsqlConnection(connectionString);
         await connection.OpenAsync(TestContext.Current.CancellationToken);
         await PgMigrations.MigrateAsync(connection, TestContext.Current.CancellationToken);
-        await DeleteRowsAsync(connection, "blocked_process_reports", DmvFallbackServerId);
-        await DeleteRowsAsync(connection, "dmv_blocking_snapshots", DmvFallbackServerId);
+        await DeleteRowsAsync(connection, "blocked_process_reports", DmvFallbackServerId, TestContext.Current.CancellationToken);
+        await DeleteRowsAsync(connection, "dmv_blocking_snapshots", DmvFallbackServerId, TestContext.Current.CancellationToken);
 
         await using var viewer = new ViewerDataService(connectionString!);
 
+        var bodySucceeded = false;
         try
         {
             var collectionTime = TruncateToSeconds(DateTime.UtcNow.AddMinutes(-10));
@@ -434,11 +441,16 @@ await InsertDmvBlockingSnapshotAsync(connection, DmvFallbackServerId, collection
             Assert.Equal(155, only.BlockedSpid);
             Assert.Equal(160, only.BlockingSpid);
             Assert.False(only.HasReportXml);   // the DMV read never selects the report XML column
+
+            bodySucceeded = true;
         }
         finally
         {
-            await DeleteRowsAsync(connection, "blocked_process_reports", DmvFallbackServerId);
-            await DeleteRowsAsync(connection, "dmv_blocking_snapshots", DmvFallbackServerId);
+            await LiveStoreCleanup.RunAsync(connectionString!, bodySucceeded, async (cleanup, cleanupCt) =>
+            {
+                await DeleteRowsAsync(cleanup, "blocked_process_reports", DmvFallbackServerId, cleanupCt);
+                await DeleteRowsAsync(cleanup, "dmv_blocking_snapshots", DmvFallbackServerId, cleanupCt);
+            });
         }
     }
 
@@ -452,10 +464,11 @@ public async Task DeadlockRead_CarriesGraphXml_ParsesToProcessRows_AgainstDevPos
         using var connection = new NpgsqlConnection(connectionString);
         await connection.OpenAsync(TestContext.Current.CancellationToken);
         await PgMigrations.MigrateAsync(connection, TestContext.Current.CancellationToken);
-        await DeleteRowsAsync(connection, "deadlocks", DeadlockServerId);
+        await DeleteRowsAsync(connection, "deadlocks", DeadlockServerId, TestContext.Current.CancellationToken);
 
         await using var viewer = new ViewerDataService(connectionString!);
 
+        var bodySucceeded = false;
         try
         {
             var collectionTime = TruncateToSeconds(DateTime.UtcNow.AddMinutes(-10));
@@ -481,10 +494,13 @@ public async Task DeadlockRead_CarriesGraphXml_ParsesToProcessRows_AgainstDevPos
             Assert.Equal(2, details.Count);
             Assert.True(details.Single(d => d.Spid == 55).IsVictim);
             Assert.False(details.Single(d => d.Spid == 60).IsVictim);
+
+            bodySucceeded = true;
         }
         finally
         {
-            await DeleteRowsAsync(connection, "deadlocks", DeadlockServerId);
+            await LiveStoreCleanup.RunAsync(connectionString!, bodySucceeded, async (cleanup, cleanupCt) =>
+                await DeleteRowsAsync(cleanup, "deadlocks", DeadlockServerId, cleanupCt));
         }
     }
 
@@ -498,11 +514,12 @@ public async Task BprRead_CarriesBestEffortPlans_InRow_GatesPerRow_AgainstDevPos
         using var connection = new NpgsqlConnection(connectionString);
         await connection.OpenAsync(TestContext.Current.CancellationToken);
         await PgMigrations.MigrateAsync(connection, TestContext.Current.CancellationToken);
-        await DeleteRowsAsync(connection, "blocked_process_reports", BprPlanServerId);
-        await DeleteRowsAsync(connection, "dmv_blocking_snapshots", BprPlanServerId);
+        await DeleteRowsAsync(connection, "blocked_process_reports", BprPlanServerId, TestContext.Current.CancellationToken);
+        await DeleteRowsAsync(connection, "dmv_blocking_snapshots", BprPlanServerId, TestContext.Current.CancellationToken);
 
         await using var viewer = new ViewerDataService(connectionString!);
 
+        var bodySucceeded = false;
         try
         {
             var t = TruncateToSeconds(DateTime.UtcNow.AddMinutes(-10));
@@ -532,11 +549,16 @@ await InsertBlockedProcessReportAsync(connection, BprPlanServerId, t, t.AddSecon
             var neither = rows.Single(r => r.WaitTimeMs == 3000);
             Assert.False(neither.HasBlockedQueryPlan);
             Assert.False(neither.HasBlockingQueryPlan);
+
+            bodySucceeded = true;
         }
         finally
         {
-            await DeleteRowsAsync(connection, "blocked_process_reports", BprPlanServerId);
-            await DeleteRowsAsync(connection, "dmv_blocking_snapshots", BprPlanServerId);
+            await LiveStoreCleanup.RunAsync(connectionString!, bodySucceeded, async (cleanup, cleanupCt) =>
+            {
+                await DeleteRowsAsync(cleanup, "blocked_process_reports", BprPlanServerId, cleanupCt);
+                await DeleteRowsAsync(cleanup, "dmv_blocking_snapshots", BprPlanServerId, cleanupCt);
+            });
         }
     }
 
@@ -550,10 +572,11 @@ public async Task DeadlockRead_CarriesVictimPlan_ThreadedOntoEveryProcessRow_Aga
         using var connection = new NpgsqlConnection(connectionString);
         await connection.OpenAsync(TestContext.Current.CancellationToken);
         await PgMigrations.MigrateAsync(connection, TestContext.Current.CancellationToken);
-        await DeleteRowsAsync(connection, "deadlocks", DeadlockPlanServerId);
+        await DeleteRowsAsync(connection, "deadlocks", DeadlockPlanServerId, TestContext.Current.CancellationToken);
 
         await using var viewer = new ViewerDataService(connectionString!);
 
+        var bodySucceeded = false;
         try
         {
             var t = TruncateToSeconds(DateTime.UtcNow.AddMinutes(-10));
@@ -580,10 +603,13 @@ any of them (gated on presence, not IsVictim). */
             Assert.Equal(2, details.Count);
             Assert.All(details, d => Assert.True(d.HasVictimQueryPlan));
             Assert.All(details, d => Assert.Equal("<ShowPlanXML>victim</ShowPlanXML>", d.VictimQueryPlanXml));
+
+            bodySucceeded = true;
         }
         finally
         {
-            await DeleteRowsAsync(connection, "deadlocks", DeadlockPlanServerId);
+            await LiveStoreCleanup.RunAsync(connectionString!, bodySucceeded, async (cleanup, cleanupCt) =>
+                await DeleteRowsAsync(cleanup, "deadlocks", DeadlockPlanServerId, cleanupCt));
         }
     }
 
@@ -597,11 +623,12 @@ public async Task BlockingSlicer_HourlyBuckets_XePreferredDmvExcluded_AgainstDev
         using var connection = new NpgsqlConnection(connectionString);
         await connection.OpenAsync(TestContext.Current.CancellationToken);
         await PgMigrations.MigrateAsync(connection, TestContext.Current.CancellationToken);
-        await DeleteRowsAsync(connection, "blocked_process_reports", BlockingSlicerServerId);
-        await DeleteRowsAsync(connection, "dmv_blocking_snapshots", BlockingSlicerServerId);
+        await DeleteRowsAsync(connection, "blocked_process_reports", BlockingSlicerServerId, TestContext.Current.CancellationToken);
+        await DeleteRowsAsync(connection, "dmv_blocking_snapshots", BlockingSlicerServerId, TestContext.Current.CancellationToken);
 
         await using var viewer = new ViewerDataService(connectionString!);
 
+        var bodySucceeded = false;
         try
         {
             var hourA = TruncateToHour(DateTime.UtcNow.AddHours(-5));
@@ -618,11 +645,16 @@ public async Task BlockingSlicer_HourlyBuckets_XePreferredDmvExcluded_AgainstDev
             Assert.Equal(2, buckets[0].SessionCount);   // hour A = 2 XE events (DMV excluded)
             Assert.Equal(1, buckets[1].SessionCount);   // hour B = 1 XE event
             Assert.True(buckets[0].BucketTime < buckets[1].BucketTime);
+
+            bodySucceeded = true;
         }
         finally
         {
-            await DeleteRowsAsync(connection, "blocked_process_reports", BlockingSlicerServerId);
-            await DeleteRowsAsync(connection, "dmv_blocking_snapshots", BlockingSlicerServerId);
+            await LiveStoreCleanup.RunAsync(connectionString!, bodySucceeded, async (cleanup, cleanupCt) =>
+            {
+                await DeleteRowsAsync(cleanup, "blocked_process_reports", BlockingSlicerServerId, cleanupCt);
+                await DeleteRowsAsync(cleanup, "dmv_blocking_snapshots", BlockingSlicerServerId, cleanupCt);
+            });
         }
     }
 
@@ -636,10 +668,11 @@ public async Task DeadlockSlicer_HourlyBuckets_AgainstDevPostgres()
         using var connection = new NpgsqlConnection(connectionString);
         await connection.OpenAsync(TestContext.Current.CancellationToken);
         await PgMigrations.MigrateAsync(connection, TestContext.Current.CancellationToken);
-        await DeleteRowsAsync(connection, "deadlocks", DeadlockSlicerServerId);
+        await DeleteRowsAsync(connection, "deadlocks", DeadlockSlicerServerId, TestContext.Current.CancellationToken);
 
         await using var viewer = new ViewerDataService(connectionString!);
 
+        var bodySucceeded = false;
         try
         {
             var hourA = TruncateToHour(DateTime.UtcNow.AddHours(-5));
@@ -653,10 +686,13 @@ public async Task DeadlockSlicer_HourlyBuckets_AgainstDevPostgres()
             Assert.Equal(2, buckets.Count);
             Assert.Equal(2, buckets[0].SessionCount);
             Assert.Equal(1, buckets[1].SessionCount);
+
+            bodySucceeded = true;
         }
         finally
         {
-            await DeleteRowsAsync(connection, "deadlocks", DeadlockSlicerServerId);
+            await LiveStoreCleanup.RunAsync(connectionString!, bodySucceeded, async (cleanup, cleanupCt) =>
+                await DeleteRowsAsync(cleanup, "deadlocks", DeadlockSlicerServerId, cleanupCt));
         }
     }
 
@@ -771,9 +807,9 @@ private static DateTime TruncateToSeconds(DateTime value) =>
     private static DateTime TruncateToHour(DateTime value) =>
         DateTime.SpecifyKind(new DateTime(value.Ticks - (value.Ticks % TimeSpan.TicksPerHour)), DateTimeKind.Unspecified);
 
-    private static async Task DeleteRowsAsync(NpgsqlConnection connection, string table, int serverId)
+    private static async Task DeleteRowsAsync(NpgsqlConnection connection, string table, int serverId, System.Threading.CancellationToken ct)
     {
         using var cleanup = new NpgsqlCommand($"DELETE FROM {table} WHERE server_id = {serverId};", connection);
-        await cleanup.ExecuteNonQueryAsync(TestContext.Current.CancellationToken);
+        await cleanup.ExecuteNonQueryAsync(ct);
     }
 }