Lite connection alerts + generic webhook channel (#1506)

CHANGELOG.md

@@ -9,6 +9,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Added
 
+- **Lite: connection-lost / connection-restored alerts, and a generic webhook channel for both apps** ([#1506]) — Lite was the only one of the three apps that could not tell you your server had gone down unless you happened to be watching the tray: the Dashboard emails on the connection edge and Darling's `DarlingSelfAlertEvaluator` emails *and* webhooks it, but Lite's edge fired a tray balloon and nothing else. Both edges in `MainWindow.CheckConnectionsAndNotify` now also route through Lite's alert path — email, webhook, and a `config_alert_log` row — as **"Server Unreachable"** / **"Server Restored"**, the metric names the shared `AlertSeverity` map and `AlertMetricClassifier` already understood (CRITICAL-red, and the restore classified as a green resolution), so no new naming was invented. The tray balloon is unchanged; this is purely additive. Delivery stays **edge-triggered** — the transition logic moved out of the inline WPF condition into a pure, unit-tested `ConnectionEdgeDetector`, so an eight-hour outage polled every minute produces ONE "Server Unreachable", not 480 — and it respects mute rules and per-server silencing exactly as Lite's other alerts do. Gated on the existing `App.NotifyConnectionChanges` preference (default on, already in Settings), which is also the name Darling uses; no second preference was added.
+- Alongside it, a third webhook channel in the shared `WebhookAlertService` — **generic HTTP POST** — beside Teams and Slack, for **Lite and Darling**. It POSTs an operator-authored JSON body with `{{metric}}`, `{{server}}`, `{{value}}`, `{{threshold}}`, `{{severity}}`, `{{context}}` and `{{timestamp}}` placeholders and an arbitrary JSON headers object, which covers PagerDuty / Opsgenie / n8n and — the motivating use case — a GitHub `repository_dispatch` that re-runs a workflow when an alert fires. **This is deliberately not a script/exe runner:** arbitrary command execution in a signed binary is an EDR-flagged attack path, and a webhook reaches the same automation with no process-execution surface. Every substituted placeholder is **JSON-escaped** (via `JsonSerializer`, which — unlike `JsonEncodedText.Encode` — tolerates the lone surrogates SQL Server names can carry instead of throwing), so a server name carrying a quote or backslash — `HOST\INSTANCE`, or something hostile — cannot break out of the template and corrupt or inject into the posted JSON; the substituted body is validated as JSON before it goes on the wire, malformed headers JSON degrades to a clear logged error instead of throwing into the alert loop, and a `User-Agent` is defaulted (GitHub's API 403s requests without one). Settings live beside Teams/Slack in both apps' Settings windows with a "Send Test" button and a `repository_dispatch` example; the URL and the headers JSON are treated as **secrets** — Windows Credential Manager in Lite, and in Darling's Postgres control plane (schema **V26**) `generic_url` + `generic_headers` are column-`REVOKE`d from the read-only viewer role like the existing webhook URLs, since the headers carry the `Authorization` bearer token. The deprecated Dashboard satisfies the shared settings interface with the channel off and gains no new UI.
+
 - **Lite viewer: System Events tab (Tier-1 parity BUILD — drains the coverage pin to ZERO)** ([#1509]) — the Lite port of the Darling viewer's **System Events** tab, the LAST unbuilt-UI pair on the parity board. Lite already collected `system_health_events` (raw `system_health` XE) and `default_trace_events` (built-in Default Trace) but had **no viewer surface** (the coverage pin from #1499 allow-listed both); this adds one, so `CollectorViewerCoverageTests.KnownStoreOnlyOrUnbuiltTables` is now **empty** — every collector table has a Lite reader and the ratchet is fully drained. New top-level tab (index 18, the new LAST tab, after Collection Health — nothing renumbered, so switch `case 18` still equals its 0-based XAML index) with 11 inner sub-tabs in Darling's order. The two lead sub-tabs — **Corruption Events** + **Contention Events** — are the SYSTEM-component counter CHARTS (bad pages / dump requests / access violations / write-AV; non-yielding tasks / latch warnings / sick spinlocks by type / SQL-vs-system CPU), all eight fed by ONE `sp_server_diagnostics` SYSTEM read with **no** significance filter (every snapshot plotted over time), in `ServerTab.SystemHealthCharts.cs` using Lite's chart idioms (shared `ChartStyle`/`ChartHoverHelper`, `SeriesColors`/`ChartPalette`, `UtcOffsetMinutes` display conversion, Y-floor-at-0). The remaining nine are parse-on-read GRIDS — Scheduler Issues, Severe Errors, Memory Conditions, Memory Broker, Memory Node OOM, Significant Waits, CPU Tasks, I/O Issues, Default Trace — with **every Darling grid column** reproduced (headers, bindings, formats). The reader (`LocalDataService.SystemEvents.cs`, mirroring Darling's `ViewerDataService.SystemEvents` reads adapted to DuckDB) fetches the raw `event_xml` for each category over the window from the `v_system_health_events` archive view, shreds it with the Common `SystemHealthParser`, and keeps the significant rows by calling the shared `PerformanceMonitor.Common.SystemHealthSignificance` **directly** (zero duplication — Lite and Darling share the one sp_HealthParser predicate source hoisted in #1507). The shred runs off the UI thread (the loaders wrap the reads in `Task.Run`, mirroring the deadlock-graph parse). Severe Errors resolves each event's `database_id` to a name from the latest `v_database_size_stats` snapshot (DuckDB `QUALIFY ROW_NUMBER() OVER (PARTITION BY database_id ORDER BY collection_time DESC)`, the equivalent of Darling's Postgres `DISTINCT ON`) and honors the #1319 database filter client-side on the resolved name; Default Trace honors it in SQL on `database_name`. The **Default Trace lane** reads `v_default_trace_events`, gates via the shared `DefaultTraceEventSignificance`, and — because the trace `StartTime` is the monitored server's LOCAL wall clock (not UTC like the XE `@timestamp`) — windows on the server-local bounds (`GetTimeRangeServerLocal`) and de-skews each row's `event_time` to naive-UTC (subtracting the connected server's `UtcOffsetMinutes`) so it shares the system_health rows' UTC frame and renders/sorts consistently through `ServerTimeHelper.FormatServerTime`. Wired into the TabControl + `RefreshVisibleTabAsync` switch + `MainTabControl_SelectionChanged` sub-tab-reload path + `ServerTab.xaml.cs` init/dispose, with a `DataGridFilterManager` on each of the nine grids. Real-DuckDB round-trip reader tests (`SystemEventsReaderTests`) plant fixture rows into the store and assert each category's significant set, the column projection, the DB-name resolution + #1319 filter, the event_time window bounding, and the Default Trace de-skew.
 - **Lite viewer: CPU Scheduler, Plan Cache & Session Stats tabs (Tier-1 parity BUILD, batch of 3)** ([#1505]) — the Lite ports of three more Darling viewer per-server tabs whose data Lite already collected but never surfaced (the coverage pin from #1499 allow-listed `cpu_scheduler_stats` / `plan_cache_stats` / `session_summary_stats` as unbuilt UI). Placed together in the diagnostics tail right after Latches & Spinlocks — **CPU Scheduler** (index 14), **Plan Cache** (15), **Session Stats** (16), shifting Collection Health 14→17 with the `RefreshVisibleTabAsync` switch re-aligned so every top-level tab's 0-based index still equals its case. All three are **point-in-time snapshot** collectors (one aggregate row per collection, not cumulative-counter deltas), so the readers (`LocalDataService.CpuScheduler.cs` / `.PlanCache.cs` / `.SessionStats.cs`, mirroring Darling's `ViewerDataService` reads adapted to DuckDB against the `v_cpu_scheduler_stats` / `v_plan_cache_stats` / `v_session_summary_stats` archive views) plot the counts directly — no `LAG`/delta math. **CPU Scheduler**: a runnable / blocked / queued task-count trend over the window plus a latest-snapshot metric/value grid (schedulers, workers, NUMA, OS-memory pressure) whose warning rows highlight from the collector's own CASE-computed pressure flags, with the rolled-up pressure level + recommendation headline rows (`ClassifyCpuPressure`, install/47 `report.cpu_scheduler_pressure` parity). **Plan Cache**: a single-use vs multi-use plan-cache size (MB) trend (the single-use bloat signal, on the shared `SinglePagePlans`/`MultiPagePlans` palette keys), a summary strip with the TRUE total plans (uncapped SUM, independent of the top-30 grid), oldest-plan age, and a derived single-use bloat badge + forced-parameterization hint (`ClassifyPlanCacheBloat`, `report.plan_cache_bloat` parity), plus a filterable latest-snapshot composition grid per (cacheobjtype, objtype). **Session Stats**: a server-wide session-status trend (Total / Running / Sleeping / Background / Dormant / Idle >30m / Waiting for Memory, each on its fixed `Session*` palette key, a series only drawn when non-zero in the window so all-zero states stay out of the legend) plus a summary strip for the non-chartable attribution (top application / top host by connections + distinct databases). All three tabs mirror Darling's rendering with Lite's chart idioms (shared `ChartStyle`/`ChartHoverHelper`, `SeriesColors`/`ChartPalette`, Y-floor-at-0, `UtcOffsetMinutes` display conversion, `SetLimitsX` window-pinning), wired into the TabControl + switch + `ServerTab.xaml.cs` init/dispose, with the Plan Cache composition grid on a `DataGridFilterManager`. Removes the three tables from the coverage-pin allow-list (`CollectorViewerCoverageTests.KnownStoreOnlyOrUnbuiltTables`, now **2** — only `system_health_events` / `default_trace_events` remain), so the ratchet now REQUIRES their readers.
 - **Lite viewer: Latches & Spinlocks tab (Tier-1 parity BUILD)** ([#1504]) — the Lite port of the Darling viewer's consolidated **Latches & Spinlocks** per-server tab. Lite already collected `latch_stats` (`sys.dm_os_latch_stats`) and `spinlock_stats` (`sys.dm_os_spinlock_stats`) but had **no viewer surface** (the coverage pin from #1499 allow-listed them as unbuilt UI); this adds one, closing a parity-board Tier-1 gap. New top-level tab (index 13, between Daily Summary and Collection Health, mirroring Darling's diagnostics-tail placement): the latch **wait-time trend** (ms/sec) and spinlock **collisions trend** (collisions/sec) for the TOP 5 contenders stack vertically, each above a collapsed Expander holding its per-class latest-snapshot grid. The reader (`LocalDataService.LatchSpinlock.cs`) mirrors the Darling Postgres reads adapted to DuckDB against the `v_latch_stats` / `v_spinlock_stats` archive views — per-second rates computed in SQL from the per-contender `LAG` interval, the same `date_trunc`/`extract(epoch)`/`LAG` idiom the Wait Stats trend uses (the delta tables carry no stored `sample_interval_seconds`). The tab (`ServerTab.LatchSpinlock.cs`) mirrors Darling's rendering with Lite's chart idioms (shared `ChartStyle`/`ChartHoverHelper`, cycling `SeriesColors`, Y-floor-at-0, `UtcOffsetMinutes` display conversion, zero-line-when-empty), wired into the TabControl + `RefreshVisibleTabAsync` switch + `DataGridFilterManager`s for both grids. Removes `latch_stats` / `spinlock_stats` from the coverage-pin allow-list (`CollectorViewerCoverageTests.KnownStoreOnlyOrUnbuiltTables`, now 5), so the ratchet REQUIRES the reader.

Darling/Darling.Tests/DarlingAnalysisPipelineTests.cs

@@ -516,6 +516,11 @@ private sealed class TestAlertSettings : IAlertSettings
         public bool SlackWebhookEnabled => false;
         public string SlackWebhookUrl => "";
         public string SlackProxyAddress => "";
+        public bool GenericWebhookEnabled => false;
+        public string GenericWebhookUrl => "";
+        public string GenericWebhookHeadersJson => "";
+        public string GenericWebhookBodyTemplate => "";
+        public string GenericWebhookProxyAddress => "";
         public double AnalysisNotifySeverity { get; init; } = 1.5;
         public int AnalysisNotifyCooldownMinutes { get; init; } = 360;
     }

Darling/Darling.Tests/DarlingManagedRolesTests.cs

@@ -129,7 +129,13 @@ public void ViewerRestrictedConfigTables_SecretAndNonSecretColumns_AreDisjointAn
         {
             ["config_monitored_servers"] = new[] { "encrypted_password" },
             ["config_command"] = new[] { "args_json" },
-            ["config_notification"] = new[] { "smtp_encrypted_password", "smtp_username", "teams_url", "slack_url" },
+            /* generic_url is a bearer secret like the sibling webhook URLs, and generic_headers holds the
+               Authorization token itself (#1506 / V26) — both must stay column-REVOKEd from the viewer role. */
+            ["config_notification"] = new[]
+            {
+                "smtp_encrypted_password", "smtp_username", "teams_url", "slack_url",
+                "generic_url", "generic_headers",
+            },
         };
 
         /* Exactly the three known secret-bearing tables, with the expected secret columns. */

Darling/Darling.Tests/DarlingObservabilityTests.cs

@@ -32,9 +32,9 @@ public sealed class DarlingObservabilityTests
     private const int TestServerId = -424242;
 
     [Fact]
-    public void MigrationScripts_TwentyFiveVersions_V23CollectionLogHypertable_V24JobHistory_V25AgentStatus()
+    public void MigrationScripts_TwentySixVersions_V24JobHistory_V25AgentStatus_V26GenericWebhook()
     {
-        Assert.Equal(25, PgMigrations.Scripts.Count);
+        Assert.Equal(26, PgMigrations.Scripts.Count);
         Assert.Equal(1, PgMigrations.Scripts[0].Version);
         Assert.Equal(2, PgMigrations.Scripts[1].Version);
         Assert.Equal(3, PgMigrations.Scripts[2].Version);
@@ -60,7 +60,16 @@ public void MigrationScripts_TwentyFiveVersions_V23CollectionLogHypertable_V24Jo
         Assert.Equal(23, PgMigrations.Scripts[22].Version);
         Assert.Equal(24, PgMigrations.Scripts[23].Version);
         Assert.Equal(25, PgMigrations.Scripts[24].Version);
-        Assert.Equal(25, StorageVersion.SchemaVersion);
+        Assert.Equal(26, PgMigrations.Scripts[25].Version);
+        Assert.Equal(26, StorageVersion.SchemaVersion);
+
+        /* V26 (#1506) adds the generic webhook channel's four columns to the V17 control-plane table.
+           Schema-qualified config.* and IF NOT EXISTS, per the file's additive-ALTER idiom. */
+        var v26 = PgMigrations.Scripts[25].Sql;
+        Assert.Contains("ALTER TABLE config.config_notification ADD COLUMN IF NOT EXISTS generic_url", v26, StringComparison.Ordinal);
+        Assert.Contains("ALTER TABLE config.config_notification ADD COLUMN IF NOT EXISTS generic_headers", v26, StringComparison.Ordinal);
+        Assert.Contains("ALTER TABLE config.config_notification ADD COLUMN IF NOT EXISTS generic_body_template", v26, StringComparison.Ordinal);
+        Assert.Contains("ALTER TABLE config.config_notification ADD COLUMN IF NOT EXISTS generic_proxy", v26, StringComparison.Ordinal);
 
         /* V5 completes the v_* twin of Lite's DuckDB view layer -- the copy-parity tail tabs
            (Running Jobs, Configuration, Daily Summary, Collection Health) read these five, so

Darling/Darling.Tests/ViewerControlPlaneStage3bTests.cs

@@ -109,6 +109,7 @@ public sealed class ViewerNotificationSqlTests
     {
         "smtp_host", "smtp_port", "smtp_use_ssl", "smtp_username", "smtp_encrypted_password", "smtp_from_address",
         "smtp_recipients", "email_cooldown_minutes", "teams_url", "teams_proxy", "slack_url", "slack_proxy",
+        "generic_url", "generic_headers", "generic_body_template", "generic_proxy",
     };
 
     [Fact]
@@ -122,7 +123,7 @@ public void UpsertSql_WritesEveryColumn_AsBoundParameters_SingleRowOnConflictUpd
             Assert.Contains(column, sql, StringComparison.Ordinal);
         }
 
-        for (var i = 1; i <= 12; i++)
+        for (var i = 1; i <= 16; i++)
         {
             Assert.Contains("$" + i.ToString(System.Globalization.CultureInfo.InvariantCulture), sql, StringComparison.Ordinal);
         }