Skip to content

Darling.Tests: settle the compression self-heal healthy-job assertion (gated-live flake) - #1604

Merged
erikdarlingdata merged 3 commits into
devfrom
feature/harden-compression-selfheal-flake
Jul 21, 2026
Merged

erikdarlingdata merged 3 commits into
devfrom
feature/harden-compression-selfheal-flake

Conversation

@erikdarlingdata

Copy link
Copy Markdown
Owner

Why

The CompressionJobSelfHeal gated-live test (*_AgainstDevPostgres, real TimescaleDB) asserts a freshly-applied compression policy job is not flagged as stuck. But a just-added TimescaleDB job can momentarily read next_start = '-infinity' in job_stats before its background scheduler assigns the first real next run — and the stuck-job detector is correct to flag -infinity — so asserting immediately after ApplyCompressionPolicy raced that window and intermittently false-failed on a slow CI runner.

Fix

Deterministically settle the job into the healthy state the assertion is actually about: arm it with a real future next_start (via the same alter_job the self-heal uses), then wait for the catalog to reflect a non-(-infinity) next_start before the "not flagged" assertion. Bounded to 30s, failing loudly if it never settles — a job that stays -infinity genuinely IS stuck and must not silently pass.

Test-only. The detector (ReadStuckCompressionJobsAsync) and the self-heal path are unchanged. Related to the #1585/#1586/#1588 compression self-heal work.

Verification

Builds clean against current dev (0 errors). The authoritative check is CI's darling-pg job, which runs this gated-live test against real TimescaleDB (locally it skips without DARLING_TEST_PG).

🤖 Generated with Claude Code

erikdarlingdata and others added 3 commits July 21, 2026 08:10
…t's healthy-job assertion

The CompressionJobSelfHeal gated-live test asserts a freshly-applied compression
policy job is NOT flagged as stuck. But a just-added TimescaleDB job can
momentarily read next_start = '-infinity' in job_stats before its background
scheduler assigns the first real next run - and the stuck-job detector is CORRECT
to flag -infinity - so asserting immediately after ApplyCompressionPolicy raced
that window and intermittently false-failed on a slow CI runner.

Fix: arm the job with a real future next_start (via the same alter_job the
self-heal uses), then wait for the catalog to reflect a non-(-infinity)
next_start before the "not flagged" assertion. Bounded to 30s, failing loudly if
it never settles (a job that stays -infinity IS genuinely stuck and must not
silently pass). Test-only; the detector and self-heal are unchanged.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@erikdarlingdata
erikdarlingdata merged commit e7ad6e4 into dev Jul 21, 2026
3 checks passed
@erikdarlingdata
erikdarlingdata deleted the feature/harden-compression-selfheal-flake branch July 23, 2026 03:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant