Repair the Query Store split slices stored before #1907, and disclose what cannot be reached (#1912)

CHANGELOG.md

@@ -9,6 +9,21 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Important
 
+- **Lite repairs its stored Query Store history once, on the first launch after upgrading - expect a few extra seconds and a slightly larger archive** ([#1912]) - builds before [#1907] stored Query Store's flushed and still-in-memory slice of one interval as two separate rows. They are ADDITIVE, so every read that keeps one row per interval reported a FRACTION of that interval's work - on the live evidence, 8 executions where 94 was true. [#1907] fixed the collection side and made the read deterministic; it could not touch what was already stored. The first launch of this version collapses those rows into the single row the collector would write today, in the hot store **and in the parquet archive**, and records that it has done so - it runs once, not on every start. On a real archive the Query Store files totalled ~46 MB and the largest (533,109 rows) rewrote in under a second, so "a few extra seconds" is the honest expectation rather than a warning.
+
+  **Your archive may get slightly BIGGER even though rows were removed, and that is not a fault.** Parquet size is dominated by how well each column compresses within a row group, not by row count, so re-encoding a file can cost more than the rows it dropped - measured at 28.0 MB becoming 31.6 MB while 22,760 rows were removed. The repair keeps the archive's own ZSTD compression; it is the re-layout, not a codec change, and the data is both correct and fewer rows than before.
+
+  **Nothing is at risk if it is interrupted.** Each archive file is rewritten beside the original and swapped in only after its row count AND its total execution count both check out - the second is the invariant the collapse's own arithmetic guarantees, and it is what would catch a bad aggregate that still produced the right number of rows. A file that fails verification is left exactly as it was, logged, and retried on the next launch; the other files still get repaired, so one bad file cannot block the rest. No backup copies are kept, because verify-before-promote is the safety. Until a file is repaired it keeps being read normally, with [#1907]'s deterministic tie-break resolving it, so a half-finished repair is a delay and never a corruption. Lite always starts, even if the repair cannot run at all.
+
+- **OPERATOR ACTION, Darling only: run `--collapse-legacy-slices` PROMPTLY after upgrading, and the sooner the better** ([#1912]) - the same repair Lite does for itself, on the store where Darling keeps the same rows. It is deliberately a verb rather than an automatic step, for the reason `--backfill-rollups` is ([#1759]): it rewrites stored rows and re-materializes rollups, so it runs when a person decides to, and `--dry-run` reports exactly what it would change without changing anything.
+
+  ```
+  PerformanceMonitor.Darling.Service.exe --collapse-legacy-slices --dry-run
+  PerformanceMonitor.Darling.Service.exe --collapse-legacy-slices
+  ```
+
+  **Timing is the whole point, and it is unusual enough to be worth stating plainly: this verb gets LESS useful every day you wait.** It can only repair Query Store rows that are still in the raw tier, which holds a few days. Run it right after upgrading and it fixes the recent history you are actually looking at, and the rollups built from it. Run it a week later and there is nothing left in raw from the affected period, so it will correctly report nothing to do while the numbers stay wrong. **What it does not reach is permanent** - the daily rollups are kept indefinitely and cannot be rebuilt from raw that retention has already dropped, so Query Store execution counts and count-weighted totals for the pre-upgrade period stay UNDERSTATED for the life of that store. There is no ordering of these operations that reaches further back; this is a disclosure, not a step you can take later. Safe to re-run at any time - a second pass finds nothing, because the signature it matches cannot occur in rows collected since [#1907]. Applies to every Darling store including **DARLING01**.
+
 - **Lite's data now lives OUTSIDE the install directory - and on any version older than this one, upgrading with `Setup.exe` destroys it** ([#1832]) - Lite kept everything in `%LOCALAPPDATA%\PerformanceMonitorLite`, which is also the installer's own install root. Re-running `Setup.exe` over an existing install renames that folder aside and deletes it, so an installer-based upgrade took the DuckDB store, the Parquet archive, the logs, and `settings.json` with it, in one move, before any code from the new build ran. **In-app updates were always safe** - Help > About replaces the `current\` subfolder in place and never touches the data sitting beside it - which is precisely why the loss looked random instead of reproducible. Two things always survived and still do: the monitored-server list (`%ProgramData%\PerformanceMonitorLite\config\servers.json`, machine-wide) and every password and webhook URL in Windows Credential Manager.
 
   **The one-time caveat: reaching THIS version via `Setup.exe` still loses your data.** The installer deletes the old directory before the new build starts, so no fix shipping inside the new build can prevent it - the code that would migrate your data has not run yet when the data is deleted. Upgrade **in place** instead and nothing is lost: **Help > About** downloads and applies the update, or extract the portable ZIP over your existing copy. From this version forward `Setup.exe` is safe, because the data is no longer where the installer writes.
@@ -51,6 +66,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Fixed
 
+- **Darling gains `--collapse-legacy-slices`, which repairs the Query Store rows stored before the split-slice fix and re-materializes the rollups they fed** ([#1912]) - the companion to Lite's automatic repair above, for the store where the same rows live. Same collapse, same count-weighted math, same idempotence: the pre-fix signature is two or more rows sharing the entire dedup key AND `collection_time`, which no row collected since [#1907] can match, so the verb is safe to re-run and a second pass reports nothing to do. It surveys first and `--dry-run` reports real numbers while changing nothing.
+
+  **It is bounded by raw retention, and that bound is the whole story.** Only rows still in the raw tier can be collapsed, so the repair reaches the last few days - which at upgrade time is exactly the recent history an operator is looking at, and is why **running it promptly after upgrading is what makes it worth anything**. Query Store numbers older than the raw window keep their understated counts PERMANENTLY: the daily rollups are kept indefinitely and cannot be rebuilt from raw that retention has already dropped. That is disclosed rather than worked around, beside the [#1849] boundary it shares.
+
+  **The refresh is clamped to the rows actually collapsed, and that is a safety property rather than an optimization.** Measured on PostgreSQL 18.4 + TimescaleDB 2.28.1: a refresh whose range lies entirely within DROPPED raw chunks **destroys** the materialization there - with `force` and without - while a range spanning dropped and live chunks preserves it, which is why it took a three-way single-variable probe to see at all. Aiming at a nominal "pre-fix period" instead of at the collapsed rows' own span would therefore have blanked the 21-day hourly and the indefinitely-kept daily below raw's floor, which is the one thing [#1759]/[#1793] forbid. Rows that were collapsed are inside raw's extent by definition, so deriving the window from them cannot reach under it. The behavior is pinned by a live test, so a future TimescaleDB that makes it safe fails loudly rather than quietly widening what a mistake would cost. The shipped `--backfill-rollups` was checked for the same exposure and is safe by construction - it converges down to raw's oldest row and never refreshes below it.
+
 - **The retention chunk-geometry guard now reads the chunks on disk, not just the interval the catalog is configured with** ([#1925]) - [#1915] shipped the geometry half of the coverage invariant by reading each relation's CONFIGURED chunk interval. That is the whole story only for a store that never changed one. `set_chunk_time_interval`, and a change to the raw chunk interval, apply to chunks cut AFTER them - existing chunks keep the width they were cut at - so a relation that has lived through a change carries a MIX, and the configured interval describes only the newest of them.
 
   **Widening reads safe; narrowing was a false green.** Measured directly on PostgreSQL 18.4 / TimescaleDB 2.28.1: a table created at 1 day and widened to 7 reports `7 days` as its interval with six 1-day chunks still underneath it. Read that way, widening is conservative - the configured number is the wider, worse one. NARROWING is the opposite: the configuration returns to a small value while wide chunks sit beneath it, and a guard reading configuration alone passes a store whose real rounding it never saw. That is not hypothetical arithmetic; the test now builds exactly that store - wide chunks, then a narrowed configuration - and the pre-[#1925] reading passes it, which is what the mutation run demonstrates.