Skip to content

CI: run the self-hosted log collector's two end-to-end tests on pull requests - #4504

Merged
erikdarlingdata merged 4 commits into
devfrom
ci/pr-log-target-e2es
Sep 28, 2026
Merged

erikdarlingdata merged 4 commits into
devfrom
ci/pr-log-target-e2es

Conversation

@erikdarlingdata

@erikdarlingdata erikdarlingdata commented Sep 27, 2026 •

Copy link
Copy Markdown
Owner

Why

Darling/Darling.Tests/PgLogEventsPipelineTests.cs has two live end-to-end tests gated on
DARLING_TEST_PG_LOG_TARGET and DARLING_TEST_PG_UTC_LOG_TARGET. Neither variable was set in any
workflow, so both tests skipped in every CI run since they were written.

What changes

build.yml's darling-pg job gets a fifth throwaway PostgreSQL cluster (port 5545), alongside the
existing store (5541) and three log-format clusters (5542/5543/5544):

  • logging_collector = on, log_destination = 'stderr', log_timezone = 'UTC'
  • log_line_prefix = '%m %u@%d [%p] ' (the %u@%d shape the second test's reader.GetName(1) == "log_timezone" assertion needs)
  • log_connections = on, log_disconnections = on, log_lock_waits = on
  • deadlock_timeout = '100ms' (inherited from Start-LogFormatCluster's shared defaults)

One cluster serves both tests: both need logging_collector on and UTC timezone, and the first
test's extra preconditions (log_connections, log_lock_waits) don't conflict with the second's.

A workload step immediately after the cluster starts puts one real event of each family the first
test asserts on into that cluster's own log, via psql.exe from the job's own pg-runtime bin:

  • an ordinary connect (SELECT 1) — produces the connection/disconnection LOG pair;
  • SELECT 1/0 — a real ERROR the server writes itself;
  • a lock wait: a background job holds LOCK TABLE inside a transaction and sleeps 3s, while a
    second psql requests the same table and blocks past deadlock_timeout, producing the
    log_lock_waits "still waiting for" / "acquired" pair;
  • a connect to a nonexistent database — a FATAL, distinct from the ERROR above.

Every workload call checks its own exit code and throws on an unmet precondition (a psql failure
that should have succeeded, or a success where a failure was expected), so a silently-quiet log
fails the step instead of failing the test with no explanation.

DARLING_TEST_PG_LOG_TARGET and DARLING_TEST_PG_UTC_LOG_TARGET are set on "Run Darling PG tests"
(the connection-string shape of the existing 5542-5544 vars), both comment blocks are updated to
name the new cluster and both tests, and the "Stop PostgreSQL" / failure-artifact steps are
extended to cover the new data directory and log file.

nightly.yml starts no log-format clusters today (only the 5541 store), so there is nothing to
mirror there; left unchanged.

Test plan

  • python3 -c "import yaml,sys; [yaml.safe_load(open(f)) for f in sys.argv[1:]]" .github/workflows/build.yml .github/workflows/nightly.yml — clean.
  • actionlint .github/workflows/build.yml — clean.
  • dotnet build Darling/Darling.Tests/Darling.Tests.csproj -c Release -p:EnableWindowsTargeting=true — 0 Warnings, 0 Errors.
  • Darling.Tests.dll -class Darling.Tests.PgLogEventsPipelineTests against a two-container docker rig with the 5545 settings: Total: 61, Failed: 0 (pre-mutation) / Total: 61, Failed: 13 (mutated, including both new E2Es).
  • Proven in PR CI:
    • This PR (0e99b5f3a): the 5545 workload step passes, and all three Darling PG tests shards pass (5,674 / 4,962 / 5,103 run, 0 failed). The shard that hosts PgLogEventsLivePostgresTests now skips 0; before this change it skipped exactly these two tests. So both run and pass.
    • A throwaway draft (Not for merge: CI proof (log-target mutation) #4505, now closed) with LOG dropped from PgLogEntryAssembler's label alternation: the workload step passes, and TheSelfHostedCollector_ReadsTheTargetsOwnLog_EndToEnd goes RED, along with ThePipeline_StoresReadsFiltersAndDedupes_AgainstDevPostgres and the parser pins.
    • AgainstAUtcTarget_… stays green under that mutation, because its line is an ERROR, not a LOG.
    • A second throwaway draft (Not for merge: CI proof (UTC log-target mutation) #4507, now closed) with DateTimeStyles.AdjustToUniversal dropped from PgLogEntryAssembler's timestamp parse, so parsed times come back as local rather than UTC: AgainstAUtcTarget_TheLogQueriesReadTheSettingWithTheTail_AndStoreWhatTheServerWrote and TheSelfHostedCollector_ReadsTheTargetsOwnLog_EndToEnd both go RED on their OccurredAtUtc.Kind == Utc assertion.

CHANGELOG

None: CI-only change.

…requests

Adds a fifth throwaway log-format cluster (port 5545, logging_collector on,
log_connections/log_disconnections/log_lock_waits on, a %u@%d-bearing
prefix, UTC timezone) to build.yml's darling-pg job, plus a deterministic
workload step that produces one real error, connection, and lock-wait
event before the tests run. Sets DARLING_TEST_PG_LOG_TARGET and
DARLING_TEST_PG_UTC_LOG_TARGET on the test step so both previously-skipped
live end-to-end tests in PgLogEventsPipelineTests run on every pull
request. nightly.yml starts no log-format clusters today, so it is left
unchanged.
psql only accepts libpq keywords or -h/-p/-U/-d flags, not an
Npgsql-style connection string; the step's calls used the latter
and failed with invalid connection option 'Host'. The tests'
own env vars (Npgsql form, read by Npgsql) are unchanged.
Outside a transaction block, PostgreSQL rejects LOCK TABLE immediately, so the waiter session never actually waits for the lock. Wrap it in BEGIN/COMMIT to match the holder session. Also add -v ON_ERROR_STOP=1 to the psql calls that are meant to succeed, so a SQL error in them is no longer silently swallowed.
@erikdarlingdata
erikdarlingdata marked this pull request as ready for review September 28, 2026 00:06
@erikdarlingdata
erikdarlingdata merged commit 1899737 into dev Sep 28, 2026
19 of 20 checks passed
@erikdarlingdata
erikdarlingdata deleted the ci/pr-log-target-e2es branch September 28, 2026 00:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant