Skip to content

Convert the Viewer read-path teardowns; ratchet 87 -> 19 (#1902, batch 2 of 3) - #1916

Merged
erikdarlingdata merged 2 commits into
devfrom
fix/1902-finally-batch2
Jul 31, 2026
Merged

erikdarlingdata merged 2 commits into
devfrom
fix/1902-finally-batch2

Conversation

@erikdarlingdata

Copy link
Copy Markdown
Owner

Batch two of three for #1902. Does not close it — 19 sites remain and the ratchet tracks them.

The tranche

The whole Viewer* read-path family: 68 teardowns across 21 classes. Taken as one tranche because they are one shape — plant rows for a viewer query, read them back, delete them in a finally that ran on the body's connection, so a test that failed for a real reason reported its teardown's error instead.

Class Sites Class Sites
ViewerQueriesTests 10 ViewerPlanHostTests 3
ViewerBlockingDepthTests 7 ViewerConfigChangesTests 2
ViewerFileIoBlockingTests 7 ViewerLatchSpinlockTests 2
ViewerDailyHealthTests 6 ViewerOverviewLanesTests 2
ViewerMemoryTests 5 ViewerWaitStatsTests 2
ViewerConfigurationTests 4 ViewerBlockingStatsTests 1
ViewerCpuTempDbTests 4 ViewerCpuSchedulerTests 1
ViewerQueriesRestTests 4 ViewerDeadlockSeverityStatsTests 1
ViewerPerfmonRunningJobsTests 3 ViewerFleetRollupTests 1
ViewerPlanCacheTests 1
ViewerRecommendationsTests 1
ViewerSessionStatsTests 1

61 sites were the single-call shape, 7 delete from two tables. Both go through LiveStoreCleanup.RunAsync taking the cleanup connection and its token.

New Ceiling: 19 (was 87).

Every one of the 21 had the token gotcha

25 delete helpers reached for TestContext.Current.CancellationToken internally. Under LiveStoreCleanup — which passes CancellationToken.None deliberately, so a cancelled run still restores the shared store — such a helper re-signals itself on precisely the path it exists to survive, and skips the delete. All 25 take a token now.

None of those 21 files declares a local ct at all; they use the ambient token inline throughout. So their body call sites pass it explicitly rather than introducing a mixed idiom inside one method. Cleanup call sites take cleanupCt.

A flake this batch's runs surfaced, fixed here

StuckCompressionJobsSql_NeverRunJob_ReadsNullLastRunStartedAt failed on the second full-suite run, in TimescaleSupportTests — a file this batch does not otherwise touch (git diff --name-only matched it zero times before the fix).

It pins the #1760 sentinel: job_stats.last_run_started_at reads -infinity, not NULL, for a job that has never executed, and the production query must neutralise that or the stuck-Running arm fires on a job that never ran. The test created its policy unparked, and add_compression_policy creates the job SCHEDULED with no initial_start, so TimescaleDB launches it within a second or two (#1788). One background launch destroys the sentinel outright — unlike a chunk count, no amount of re-reading recovers it.

#1889 fixed exactly this for three sibling tests by creating and parking in ONE transaction, so the scheduler (a separate backend) cannot see the job until the row already reads scheduled = false. This was the fourth site, and it now uses the same helper — a one-line swap to a helper already in the file, written for this race.

#1888 predicted it when it raised CI's worker slots: "any other test that has been quietly relying on the scheduler being unable to run is going to start failing … whether others exist is unknown, and finding out is the actual work here."

Proven both ways rather than by re-running until green:

Shape 4-second scheduler window Result
Old, unparked yes fails Expected: True / Actual: False — byte-identical to the intermittent full-suite failure
New, parked yes, identical passes 3/3

So the fix closes the window rather than narrowing it. It is a separate commit (3f56c20e) from the mechanical batch.

No permanent guard is added, and that is deliberate. Three other live tests in the same file create unparked policies BY DESIGN — an already-armed legacy policy is exactly what the converge tests converge — so any source-level rule would either fire on those or be narrowed until it pinned nothing. A brittle guard here would be worse than none.

Test plan

PostgreSQL 18.4 + TimescaleDB 2.28.1 from the repo's pg-runtime.zip, throwaway cluster on port 5967 with max_worker_processes = 24 (mirroring the product's sizing, per #1888 — which is why the sentinel flake was reproducible here at all), CI's connection-string shape.

Deviations and filings

Nothing filed. The one defect discovered was fixed here rather than deferred — it is a one-line mirror of a pattern already in the same file, and #1888's own text names finding these as the work. The remaining 19 sites are #1902's declared scope for batch three.

The skip count moved 9 → 10 against dev; that is a gated test dev added, not this branch.

Test-suite only — no shipped code changed.

erikdarlingdata and others added 2 commits July 31, 2026 01:17
… site (#1888 follow-through)

StuckCompressionJobsSql_NeverRunJob pins the #1760 sentinel: job_stats reads
last_run_started_at = '-infinity', not NULL, for a job that has never executed.
It created its policy UNPARKED, and add_compression_policy creates the job
SCHEDULED with no initial_start, so TimescaleDB launches it within a second or
two (#1788). One background launch destroys the sentinel outright - unlike a
chunk count, no amount of re-reading recovers it.

#1889 fixed exactly this for three sibling tests by creating and parking in ONE
transaction, so the scheduler (a separate backend) cannot see the job until the
row already reads scheduled = false. This was the fourth site and now uses the
same helper.

#1888 predicted it when it raised CI's worker slots: "any other test that has
been quietly relying on the scheduler being unable to run is going to start
failing ... whether others exist is unknown, and finding out is the actual work
here." It surfaced on a batch-two full-suite run, in a file this batch does not
otherwise touch.

Proven both ways rather than by re-running until green: with the old unparked
creation plus a deliberate four-second scheduler window the assertion fails as
Expected: True / Actual: False, byte-identical to the intermittent full-suite
failure; with the parked creation and the same window it passes 3/3.

No permanent guard, for cause: three other live tests in this file create
unparked policies BY DESIGN - an already-armed legacy policy is what the
converge tests converge - so a source-level rule would either fire on those or
be narrowed until it pinned nothing.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…h 2 of 3)

68 teardowns across the 21 Viewer* live classes, taken as one tranche because
they are one shape: plant rows for a viewer query, read them back, delete them
in a finally that ran on the body's connection - so a test that failed for a
real reason reported its teardown's error instead.

Every one of the 21 had the token gotcha: 25 delete helpers reaching for
TestContext.Current.CancellationToken internally, which under LiveStoreCleanup
(deliberately CancellationToken.None, so a cancelled run still restores the
store) re-signals itself on precisely the path the helper exists to survive.
None of those files declares a local ct at all, so their body call sites pass
the ambient token explicitly rather than introducing a mixed idiom inside one
method.

61 sites were the single-call shape and 7 delete from two tables; both go
through LiveStoreCleanup.RunAsync taking the cleanup connection and its token.

Left for batch three: the 19 Darling engine sites, including the one teardown
that cleans up through an NpgsqlDataSource rather than a connection -
deliberately isolated so it gets read properly rather than buried here.

Verified: three full gated-live runs on fresh PostgreSQL 18.4 + TimescaleDB
2.28.1 databases, 3999 passed / 0 failed / 10 skipped, exit 0, no residue; Lite
1849/0; both suites -t:Rebuild, 0 warnings.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@claude

claude Bot commented Jul 31, 2026

Copy link
Copy Markdown

Review

Reviewed the full diff (23 files, 68 teardown conversions across 21 Viewer* classes + the TimescaleSupportTests flake fix). This is test-infrastructure-only — no shipped SQL/product code changed — so the "no missing-index DMV folklore" and T-SQL style-guide concerns don't really apply here; focused on correctness of the mechanical conversion, the ratchet, and the flake fix instead.

Verification performed (not just eyeballing the diff):

Lite/Darling parity: No parity gap. Lite.Tests has no Viewer*Tests naming family at all — the Viewer* live-Postgres teardown classes are Darling-specific, so there's no Lite counterpart that also needed this conversion.

Correctness/security: Nothing found. Each DeleteRowsAsync-style helper's signature change (dropping the internal TestContext.Current.CancellationToken reach-in for a passed-in ct) is applied uniformly, and every call site correctly threads TestContext.Current.CancellationToken for setup-phase deletes (still on the test's own connection/token) vs. cleanupCt for the finally-phase deletes (via the fresh LiveStoreCleanup connection with CancellationToken.None underneath). No SQL string interpolation changes — the existing server_id = {id} interpolations are all int-typed args, not user input, so no injection surface introduced or changed.

No findings to raise. This is a clean, well-verified mechanical batch.

@erikdarlingdata
erikdarlingdata merged commit 4b3720b into dev Jul 31, 2026
4 checks passed
@erikdarlingdata
erikdarlingdata deleted the fix/1902-finally-batch2 branch July 31, 2026 05:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant