Repository navigation
CI: run slow classes that have not failed in five weeks off the PR path: 101 Darling classes (35% of shard time) and 35 Lite classes #5946
Description
Claude posting for Erik Darling.
The Darling PostgreSQL shards decide how long a PR run takes, and about a third of their time goes to slow test classes that have not failed in five weeks. This issue proposes to run those classes in the nightly and on dev pushes, and on a PR only when the change reaches them. That is the mechanism #5510 already uses for 27 classes. The candidates are 101 Darling classes (35% of the Darling shard time) and 35 Lite classes (27% of the Lite test time).
Source: GitHub Actions data for build.yml from 2026-10-09 to 2026-10-16, CI failure logs from 2026-09-07 to 2026-10-07 and from 2026-10-09 to 2026-10-16, .github/scripts/darling-shard-weights.json at dev 67835f03d, and one full Lite run on Windows from 2026-10-07.
The long pole (measured)
- A Darling PostgreSQL shard takes a median of 11.3 minutes: 1.6 minutes of setup and 9.3 of tests. There are 6 shards per run.
- A Darling shard is the last test job to finish in 270 of 405 PR runs that ran tests.
- PR runs take a median of 22.6 minutes of wall time.
The rule for a candidate
A class is a candidate when all of these are true:
- It takes 5 seconds or more: weighted seconds from
darling-shard-weights.jsonfor Darling, and seconds in one full Windows run for Lite. - It failed no CI job in either failure window above. October 8 is not covered.
- It does not carry
[Trait("Cost", "Slow")]already. - It is not a Guard class.
| Suite | Candidates | Their seconds | Share of the suite |
|---|---|---|---|
| Darling.Tests | 101 | 1,447 of 4,139 weighted | 35% |
| Lite.Tests | 35 | 329 of 1,222 | 27% |
Expected effect (inferred, not measured)
The Darling weights model each class's share of a shard's wall time. So taking 35% of them out of PR runs would cut the Darling shards' test time by about a third. That is about 19 of the 55 runner-minutes of Darling test time per PR run, and about 3 minutes of wall time. The Lite saving is smaller, about 4 runner-minutes. Measure both after the change.
Proposal
- Add
[Trait("Cost", "Slow")]to the candidates below, so the existing Pull requests skip 27 slow, never-failed test classes unless the change reaches them (#5459) #5510 rule skips them on a PR unless the change reaches them. - Re-run the rule every month, from the latest weights and failure history. Take the trait off any class that failed in the window.
- The nightly and dev pushes keep running everything.
Risk
- A regression in one of these classes fails on the dev push after the merge, not in the PR. Dev pushes run everything today, so that is one merge later.
- Somebody must read nightly and dev push failures every day.
- "Has not failed" is not "has no value". These classes stay. They only move off the PR path.
Darling.Tests candidates (101)
| Class | Seconds |
|---|---|
ComposeQueryStoreStampExactnessLiveTests |
66.0 |
PgIoStatsHourlyLiveTests |
65.3 |
ServerEditPasswordRuleViewerTests |
64.1 |
ServerIdentityLockLiveTests |
48.5 |
ViewerFleetAzureMasterScopeLiveTests |
43.3 |
StoreQueryHistoryToolLiveTests |
42.9 |
ViewerBlockedProcessReportsCoverageLiveTests |
35.1 |
ServerEditLiveTests |
33.9 |
PerfmonRegroupTimingLiveTests |
33.2 |
QueryStoreTopDailyBuilderLiveTests |
32.8 |
DarlingModuleMapGroupedSourceLiveTests |
31.9 |
DarlingModuleMapRecentRefreshLiveTests |
29.2 |
StoreServerPasswordRulesLiveTests |
26.3 |
ExternalAlertGrantLiveTests |
26.3 |
ViewerSystemEventsDataStartLiveTests |
22.7 |
QueryStoreIntervalConvergenceLiveTests |
22.7 |
ViewerShortWindowDataStartTests |
22.2 |
ViewerClutterAndMemoryPressureDataStartLiveTests |
22.0 |
ForcePlanFailuresMemoLiveTests |
20.8 |
TrendStaleStatsLiveTests |
19.9 |
DiagnosticsBundleStoreHistoryLiveTests |
19.8 |
ServerEditViewerRoleLiveTests |
19.8 |
QueryStoreIntervalPurgeRowCappedTests |
19.0 |
ProcedureStatsTotalsWriterLiveTests |
18.3 |
BlockingDatabaseFilterLiveTests |
17.8 |
QueryStoreIntervalFloorWarmUpTests |
17.0 |
QueryStoreWatermarkReaderLiveTests |
16.0 |
QueryStoreIntervalPartitionsLiveTests |
16.0 |
QueryStoreTopDailyReadLiveTests |
15.6 |
SecondaryReplicaScopeLiveTests |
13.8 |
PgDeadlockRemaskAlertPagingLiveTests |
13.8 |
ViewerEventDataStartLiveTests |
13.1 |
ViewerJobHistoryDataStartLiveTests |
12.8 |
FinOpsStorageGrowthViewLiveTests |
12.8 |
ScratchPostgresExitDrainTests |
12.4 |
AlertNotebookEndpointTests |
12.0 |
DatabaseWatermarkCacheRunnerLiveTests |
11.9 |
StoreStatementHistoryLiveTests |
11.8 |
FinOpsUtilizationViewLiveTests |
11.7 |
WebServerPageRangeTests |
11.2 |
McpCollectionCaveatGrantLiveTests |
11.2 |
WebhookPostTimeoutTests |
10.2 |
QueryStoreWatermarksWriterLiveTests |
10.2 |
TuningHourlyCreateLiveTests |
9.9 |
FinOpsRecommendationsToolParityLiveTests |
9.8 |
QueryStoreBackfillDroppedDatabaseTests |
9.6 |
PasswordKeyTablesLiveTests |
9.5 |
WebRangeKeptHistoryBehaviourTests |
9.4 |
DarlingMcpServerTrendToolsLiveTests |
9.0 |
ExternalAlertStoreLiveTests |
8.9 |
BlockingListPlanFlagsLivePostgresTests |
8.8 |
QueryStoreBackfillCandidateCacheLiveTests |
8.5 |
BoundedDdlBusyStreakLiveTests |
8.4 |
PlanViewerBehaviourTests |
8.4 |
CollectionHealthRunTimeFieldsLiveTests |
8.3 |
SlowReadsLiveTests |
8.1 |
PgDeadlocksResumeLiveTests |
8.0 |
ViewerEventCapDataStartLiveTests |
7.9 |
QueryStoreIntervalRetentionLiveTests |
7.9 |
MaterializationHoleProbeLiveTests |
7.9 |
WebBlockingSaveXmlBehaviourTests |
7.9 |
QueryStatsHourLedgerTests |
7.9 |
GridToolsBehaviourTests |
7.8 |
DarlingMcpHealthParserToolsWindowNoticeLiveTests |
7.8 |
FinOpsOptimizationViewLiveTests |
7.7 |
ComposeHourlyRawEdgesModuleMapLiveTests |
7.7 |
FinOpsRecommendationsSecondaryReplicaLiveTests |
7.6 |
ReadAdmissionWebTests |
7.6 |
ViewerQueryStoreRegressionsDataStartLiveTests |
7.4 |
ProvisioningBatchLockedTransactionLiveTests |
7.3 |
DiagnosticsBundleSeedFailureLiveTests |
7.3 |
QueryStoreBackfillSkipFailingDatabaseTests |
7.2 |
ViewerPlanCorrectionsDataStartLiveTests |
7.0 |
QueryStoreWideFleetEligibilityLiveTests |
6.9 |
WebPerfmonPerSecondBehaviourTests |
6.8 |
BoundedCompressionDdlTests |
6.7 |
QueryStoreIntervalPartitionRungTests |
6.7 |
PlanRegressionDailyTriggerLiveTests |
6.5 |
QueryStoreIntervalWideGapCacheLiveTests |
6.4 |
WebChartWidthBehaviourTests |
6.4 |
ComposeQueryStoreRankedSingleScanLiveTests |
6.3 |
ViewerConfigChangesDataStartTests |
6.1 |
RetiredBaselineSweepRecoveryLiveTests |
6.1 |
QueryStoreBackfillCutChunkLiveTests |
6.0 |
FinOpsPostgresTargetTests |
6.0 |
CollectorRunTimeViewerLiveTests |
5.9 |
FinOpsRecommendationsReadsLiveTests |
5.9 |
ProcedureStatsIdleHoursLiveTests |
5.9 |
FleetCollectionHealthRecoveryLiveTests |
5.8 |
DroppedDatabaseScopeLiveTests |
5.6 |
FinOpsApplicationConnectionsViewLiveTests |
5.4 |
RetentionCollectionYieldLiveTests |
5.4 |
AnomalyGateNullWindowMonteCarloTests |
5.3 |
AlertHistoryDismissTests |
5.3 |
CollectorRunTimeCliVerbLiveTests |
5.3 |
SensitiveStatementsTests |
5.3 |
FinOpsIdleCoverageLiveTests |
5.2 |
ProcedureStatsFloorAndHistoryLiveTests |
5.2 |
McpArgumentTypeTests |
5.1 |
QueryStoreIntervalMaintenancePassLiveTests |
5.1 |
ProcedureTrendTotalsStitchLiveTests |
5.1 |
Lite.Tests candidates (35)
| Class | Seconds |
|---|---|
ConfigChangeAttributionPipelineTests |
24.6 |
ArchiveQuerySnapshotsDailyCompactionTests |
21.9 |
WebhookCooldownSeedTests |
20.4 |
FactScorerTests |
14.6 |
ArchiveCompactionLowDiskTests |
14.0 |
DuckDbSentinelResetRaceTests |
13.5 |
QueryWindowTruncationTests |
12.6 |
QueryStoreDedupReadTests |
12.3 |
DeadlockAndBlockedProcessEnsureLogLevelLiteTests |
10.2 |
AnalysisCoverageTests |
9.8 |
SystemEventsReaderTests |
9.4 |
ArchiveCompactionSwapTests |
9.1 |
CollectionHealthReadBehaviourTests |
9.0 |
ArchiveCompactionSwapLockTests |
9.0 |
LocalDatabaseReopenTests |
8.7 |
LongQueryTraceReadOnlyIntentLiteTests |
8.5 |
ServerTimeHelperCollectionTests |
8.2 |
AnalysisAsOfAnchorTests |
8.0 |
QueryStoreSliceRepairTests |
7.4 |
DatabaseStateExpectedStoreTests |
7.3 |
DataStartBannerBlockingEventsTests |
7.3 |
QueryStoreTextAfterRankingTests |
7.2 |
OverviewBlockingLaneDataStartLiveTests |
7.1 |
DismissedArchiveSidecarTests |
6.7 |
LatestValueLookbackTests |
6.2 |
QueryStoreBackfillSkipFailingDatabaseTests |
6.0 |
LiteMcpServerClockTests |
6.0 |
EventTimeAnalysisReadsTests |
6.0 |
IndexObjectStatsTests |
5.8 |
ObjectLockingBudgetTests |
5.6 |
ReplayedIndexRebuildTests |
5.4 |
FactCollectorMiseryTests |
5.3 |
StoredEventCopiesDeadlockTests |
5.2 |
AnalysisServiceTests |
5.2 |
McpServerLocalOffsetTests |
5.1 |
Activity
erikdarlingdata commentedon Oct 16, 2026
Claude posting for Erik Darling. Adversarial review before any lane, at Erik's ask (an Opus reviewer re-derived the numbers from the run history and the repo).
Verdict: floats with changes. Held for tomorrow, Darling half only.
Change: skip a slow class only when the test map says the PR does not reach it (today the skip applies only to FULL selections,
run-darling-pg-shard.ps1:85-112). Drop the Lite half.Revert the Cost=Slow map-coverage gate: it skipped no class on the la…
erikdarlingdata commentedon Oct 16, 2026
Claude posting for Erik Darling.
Closing as measured: the gate would skip nothing.
The lane on #5967 built the gate the review asked for (a Cost=Slow Darling class is skipped on a pull request only when it has not failed in 35 days and the test map says the change does not reach it) and ran it against the last five merged pull requests (#5958, #5954, #5952, #5939, #5937) with the newest nightly map of 2,545 Darling classes.
The gate was reverted out of #5967, which now carries only #5955 (the test projects build in parallel). The Lite half was dropped at review. The shard-time lever on the pull request path is the selection and the build order, not this tag.