Lite: refuse a newer data file, re-apply newer columns on every start, and run each due collector once after a sleep (#4727, #4728)

[erikdarlingdata]

Fixes #4727
Fixes #4728

Why

Three Lite faults, all in how the collection loop and the data file's schema version are handled.

1. An older Lite pointed at the shared data root (after a rollback, or from an older portable copy) opened a data file stamped with a newer schema version. Versions 60 to 65 added columns to 13 tables, so every batch for those tables then failed, and the only sign was a line in the log.
2. The schema steps for versions 60 to 65 each caught a failed ALTER TABLE, logged a warning and let the version be stamped anyway. The step never ran again, so one transient fault during the one start that ran it left that table's batches failing on every later start. A file stamped without the column had the same problem.
3. After the computer sleeps and resumes, the collection loop's delay returned long after the slot it waited for, and the loop handed the collectors that stale slot. Every collector due then ran, and ran again at the next slot a minute later. That wasted queries and stored one extra sample per collector.

What changes

A data file stamped newer than the app is refused (#4727 item 1)

• DuckDbInitializer.InitializeCoreAsync reads the schema stamp before it creates or alters anything (a missing schema_version table still reads as version 0, so a fresh file takes the same create path as before).
• A stamp above CurrentSchemaVersion throws the new SchemaVersionTooNewException. Its message names the file and both versions, says the file was not changed, and says what to do.
• The exception goes out through the existing startup error path: MainWindow_Loaded already catches an initialization failure and shows ex.Message under "Initialization Error". The collectors are built after that call, so nothing collects and nothing writes.
• The check sits above OpenDatabaseAsync, so it can never be mistaken for the DuckDB storage-format mismatch that triggers the Parquet rebuild.

The columns versions 60 to 65 added are re-applied on every start (#4727 item 2)

• The 15 (table, column, type) entries the v60 to v65 steps add are now one static list, DuckDbInitializer.AddedColumns, each tagged with its version.
• Each step calls one shared method, AddMissingColumnsAsync, over its own version's entries. The same method runs once over the whole list on every start of an existing file (existing version above 0), after the table and index statements. A fresh file still gets the columns from the table statements.
• The method reads which tables and columns exist in one query and runs ADD COLUMN IF NOT EXISTS only for a column that is missing. A start where nothing is missing runs no ALTER. A table that does not exist yet (a file older than the table) is skipped quietly, because the table statements that follow create it with the column; before, that case logged a warning from the step.
• A column add that still fails logs one Error naming the table and column, and the start continues, so the next start retries it. The stamp is not held back: the every-start pass heals a stamped file.
• On the Dependency Error: on the bundled DuckDB (DuckDB.NET 1.5.5), ADD COLUMN IF NOT EXISTS goes through, as a real add and as a no-op, on a table that has an index and a v_ view over it. DROP COLUMN is the statement that fails while an index exists (a view does not block it). So the shared method needs no index drop, and the v60 to v65 steps never had one. The new test runs the heal with the indexes and views standing, as a real file has them, so a DuckDB upgrade that starts refusing the add would fail it.
• The source pins for the v60 to v64 steps (DeltaFamilyIntervalColumnTests x2, PerfmonCounterTypeTests, QsCaptureModeRungTests, TimeHonestyRungTests) asserted the literal ALTER text or the per-step tuples inside each step. They now assert the entries in AddedColumns and the step's call to AddMissingColumnsAsync. They check the same columns and types, in the new place.

After a sleep and resume, each due collector runs once (#4728)

• The wait moved into internal static CollectionBackgroundService.WaitForNextCycleAsync(cycleStart, interval, utcNow, delay, ct), which ExecuteAsync calls. The clock and the delay are passed in, so a test drives the production path.
• After the delay, if the time is a whole interval or more past the slot it waited for, the cycle starts at the latest grid slot at or before now: CollectorCadence.NextDue(nextCycle, now, interval) - interval, the same arithmetic as Darling's ServedSlot. A wait that returns on time hands over the same slot as before.

Test plan

• SchemaColumnHealingTests (3 tests) pass. The main test initializes a file, drops the indexes DuckDB needs in order to drop the columns, drops all 15 columns, puts the indexes back, checks the v_ views are standing and the stamp is current, re-initializes, and asserts all 15 columns are back with their types and visible through their v_ views. It failed before the change (first failing assertion: wait_stats.sample_interval_seconds is back).
• A failing add logs exactly one Error naming wait_stats.broken_column, and the next entry is still added. A table that does not exist yet logs no warning and no error.
• CollectionCycleResumeTests (2 tests) pass. A 1-minute loop with a 5-minute and a 15-minute collector, a 47-minute sleep, and ScheduleManager mark-and-due: after the resume the 5-minute collector runs once at the resume slot and next 5 minutes later, the 15-minute collector runs once, and the 1-minute collector runs at the resume slot and the next one. With the old shape in the method it failed: at 10:52 the collectors were handed the slot from 10:05, before the sleep. A second test pins that the grid is unchanged when there is no sleep.
• NewerSchemaFileGuardTests (2 tests) still pass; with the guard neutralised the refusal test fails.
• AgedDatabaseMigrationTests, PerfmonCounterTypeTests, DeltaFamilyIntervalColumnTests, QsCaptureModeRungTests, TimeHonestyRungTests, EntraCredentialSelectionOrderingTests, AnalysisBudgetTests, FindingsRetentionHorizonPinTests and SharedBaselineCacheTests pass.
• Lite.Tests builds with 0 warnings.
• Full Lite.Tests run after merging dev: 5604 tests, 1 failed. QueryWindowTruncationTests.SetWindowTruncatedBanner_Truncated_ShowsSinceEffectiveStart expected Showing since 2026-01-15 08:30:00 and got 04:30:00 (a 4-hour shift, so a time-zone setting left over from an earlier test in the run). The class is not touched by this PR, and it passes alone (11 of 11).
• The two Darling tests that read Lite's initializer as text (TimeHonestyRungTests.TheLiteTwinIsSchemaV63_AndCarriesBothColumns and QsCaptureModeRouteKnobToastRungTests.LiteTwinsOnlyTheCaptureModes_AtSchemaV64) now pin the versioned AddedColumns entries, for example (63, "cpu_utilization_stats", "sample_time_utc", "TIMESTAMP"), and pin that the v63 and v64 steps each call AddMissingColumnsAsync(connection, AddedColumnsForVersion(N)) for their own version, with each step cut at the next if (fromVersion < so a later step's call cannot satisfy it. RED: seeding each defect in DuckDbInitializer.cs in turn failed the matching test and left the other class passing. Dropping a v63 entry failed the v63 test, dropping a v64 entry failed the v64 test, and pointing the v63 step at version 62 failed the v63 test (also when a later step called version 63). With the file restored, TimeHonestyRungTests (8 tests), QsCaptureModeRouteKnobToastRungTests (6) and BuiltinAlertPersistenceRungTests (11) pass, and the full Darling.Tests run after merging dev has 17186 tests, 0 failed, 0 errors, 1123 skipped (the live tests that need a test server, which was not configured) and 1 not run.
• Not run: a live start of Lite against a real upgraded data folder.

Things to check: the every-start pass runs only when the file's stamp was above 0 before the start (a file with no schema_version reads as fresh and gets its columns from CREATE TABLE IF NOT EXISTS); CreateArchiveViewsCoreAsync is not touched.

CHANGELOG

SECTION: Fixed
ENTRY:

• An older Lite refuses to open a data file from a newer version ([Lite: an older app stops collecting for 13 tables on a newer data file, and a failed schema step is stamped as done #4727]) - An older Lite pointed at a data folder that a newer Lite had upgraded kept running, and every batch for the 13 tables that gained columns in schema versions 60 to 65 failed with only a log line to show for it. Lite now stops at startup with a message that names the file and both schema versions, and does not write to the file.
• Lite re-applies its newer columns on every start ([Lite: an older app stops collecting for 13 tables on a newer data file, and a failed schema step is stamped as done #4727]) - A data file where one of the columns added in schema versions 60 to 65 failed to apply, or was stamped without it, kept failing that table's collection on every start. Lite now adds any of those columns that are missing on each start, so the file gets it back. A column that still cannot be added is logged as an error naming the table and column, and the next start tries again.
• After the computer sleeps and resumes, Lite runs each due collector once ([Lite: after sleep and resume, collectors due before the sleep run twice a minute apart #4728]) - After a sleep, each collector that was due ran twice about a minute apart, which wasted queries and stored an extra sample per collector. Lite now starts the cycle at the current collection slot after a resume, so each due collector runs once.
  REF:
  [Lite: an older app stops collecting for 13 tables on a newer data file, and a failed schema step is stamped as done #4727]: Lite: an older app stops collecting for 13 tables on a newer data file, and a failed schema step is stamped as done #4727
  [Lite: after sleep and resume, collectors due before the sleep run twice a minute apart #4728]: Lite: after sleep and resume, collectors due before the sleep run twice a minute apart #4728