Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 41 additions & 0 deletions Darling/Darling.Tests/DarlingMcpStoreMetricsToolsTests.cs
Original file line number Diff line number Diff line change
Expand Up @@ -111,6 +111,47 @@ public void Reads_ArePostgresDialect_NoTsqlIsms_NoBareNow(string sqlName)
Assert.DoesNotContain("now()", sql, StringComparison.Ordinal);
}

/// <summary>
/// The daily series is a LAST-SNAPSHOT selection, so the description has to say so (#3119). An
/// operator asking a maximum question of it — this job's longest run that day, whether it entered its
/// warning band — gets a confident wrong answer, because the day's peak is dropped rather than
/// smoothed, and nothing else on the surface says so.
///
/// <para><b>Asserted TOGETHER with the shipped SQL rather than on its own.</b> The caveat is true only
/// while the read is a <c>DISTINCT ON</c> with a newest-first tiebreak; a read that became
/// max-preserving would make the sentence wrong in the other direction. Requiring both means either
/// half moving lands here and the pairing gets re-decided, instead of the sentence outliving the query
/// it describes.</para>
///
/// <para><b>One ordered match rather than three substrings, and deliberately strict.</b> A description
/// that said "last snapshot" about some other field and "maximum" about a third would satisfy three
/// independent <c>Contains</c> calls while telling a reader nothing about the daily series. Strictness
/// here fails toward re-deciding the sentence: a rewording that still carries the claim goes red and
/// gets re-approved, where a looser match would let a rewording that DROPPED it pass.</para>
/// </summary>
[Fact]
public void TheDescription_SaysADailyPointIsALastSnapshotAndNotAMaximum()
{
var description = ToolMethods().Single().GetCustomAttribute<DescriptionAttribute>()?.Description;
Assert.NotNull(description);

/* The selection the caveat is about. Both halves: DISTINCT ON alone would keep an arbitrary row,
and it is the newest-first tiebreak that makes the kept row the day's LAST. */
var sql = DarlingStoreMetricsReader.StoreMetricsDailySql;
Assert.Contains(
"DISTINCT ON (object_kind, object_name, date_trunc('day', metric_time))",
sql,
StringComparison.Ordinal);
Assert.Contains("metric_time DESC", sql, StringComparison.Ordinal);

/* The claim, in one match: the daily point, what it IS, and what it is not. */
Assert.Matches(@"daily point is that day's LAST snapshot[^.]*maximum", description!);

/* And the route that answers what this series cannot, so the caveat leaves a reader somewhere to
go rather than only telling them to distrust the number in front of them. */
Assert.Contains("timescaledb_information.job_history", description!, StringComparison.Ordinal);
}

[Fact]
public void GetStoreMetrics_IsInTheServerInstructions()
{
Expand Down
80 changes: 77 additions & 3 deletions Darling/Darling.Tests/RefreshCeilingProvenancePinTests.cs
Original file line number Diff line number Diff line change
Expand Up @@ -36,8 +36,9 @@ namespace Darling.Tests;
/// <see cref="TimescaleSupport.RefreshPhaseSlotSeconds"/>,
/// <see cref="TimescaleSupport.RefreshPhaseStepMinutes"/>,
/// <see cref="TimescaleSupport.RefreshSlotWarningSeconds"/>) or from the POPULATION THE COMMENT ITSELF
/// PUBLISHES. A pin that restated the summary would go stale by precisely the mechanism it exists to
/// stop.</para>
/// PUBLISHES — or, for the live envelope, from a QUOTED READING the same sentence states, with the
/// figures drawn against that reading derived rather than restated. A pin that restated the summary
/// would go stale by precisely the mechanism it exists to stop.</para>
///
/// <para><b>One pin is written to expire, deliberately.</b> The comment's reason for taking the maximum
/// rather than a percentile is partly that at this sample size the 95th percentile IS the maximum by nearest
Expand All @@ -64,7 +65,12 @@ namespace Darling.Tests;
/// mode that matters: an unfiltered reading migrating into the population a constant may be set from. The
/// pre-narrowing band and the 3.8x ratio drawn against it are likewise evidence — that band is #3012's
/// measurement and no constant here spells it — so they carry no pin. The excluded run's 1.45x ratio to the
/// population maximum IS pinned, because both of its terms are stated here.</para>
/// population maximum IS pinned, because both of its terms are stated here. The live envelope's measured
/// maximum is evidence on the same footing, and its day count with it; what IS pinned is every figure the
/// prose draws AGAINST that maximum — the slot margin, the share of the slot and the distance past the
/// watch line are all arithmetic over it and the grid's own constants, plus the band the shipped
/// classifier actually returns for it, so the paragraph cannot state a verdict the code does not
/// produce.</para>
///
/// <para><b>The guard is itself guarded, three ways.</b> Every extraction asserts its pattern MATCHED before
/// a value is compared. <see cref="EveryNumericPin_ReportsAnInjectedDrift"/> bumps each captured number ONE
Expand Down Expand Up @@ -212,6 +218,17 @@ TheInadmissibleReadings_CannotBeFoldedIntoTheStatusVerifiedSeries and the as-at
@"At ([0-9]+) s against a ([0-9]+)-second slot the margin is ([0-9]+) seconds",
true);

/* THE LIVE ENVELOPE (#3119). The measured maximum itself is a quoted reading and derives from
nothing here — but it is captured anyway, because every figure stated against it is derived
from it and from the grid's constants, so a bump to the reading has to disagree with all of
them rather than pass as a second opinion. Every group is a pair, since each figure is stated
to one decimal and the verification works in tenths. */
yield return (
"the measured live envelope",
CeilingDeclaration,
@"this job's maximum was ([0-9]+)\.([0-9]+) s\. That leaves ([0-9]+)\.([0-9]+) s of the slot, ([0-9]+)\.([0-9]+)% of it, and sits ([0-9]+)\.([0-9]+) s PAST",
true);

yield return (
"the exclude-whole occupancy sentence",
CompressionMinutesDeclaration,
Expand Down Expand Up @@ -658,6 +675,63 @@ takes the maximum partly because at this sample size the 95th percentile is the
Require(margin[1] == Slot, $"stated slot {margin[1]} s against {Slot} s");
Require(margin[2] == Slot - Ceiling, $"stated margin {margin[2]} s against {Slot - Ceiling} s derived");

/* THE LIVE ENVELOPE (#3119), which is the margin sentence's counterpart: that one states the
clearance the published population carries and this one states what a measured day leaves. In
TENTHS throughout, because each figure is stated to one decimal and integer arithmetic keeps a
rounding mode from being able to move an assertion. */
var live = Read("the measured live envelope");
var measured10 = live[0] * 10 + live[1];
var statedMargin10 = live[2] * 10 + live[3];
var statedShareTenths = live[4] * 10 + live[5];
var statedPastWatch10 = live[6] * 10 + live[7];

Require(measured10 > Ceiling * 10,
$"the measured maximum {measured10 / 10}.{measured10 % 10} s is at or below the {Ceiling} s "
+ "recorded ceiling, so a paragraph stating it as an envelope NARROWER than the population's "
+ "clearance is describing something that is not the case");

/* THE VERDICT, from the SHIPPED classifier and ABOVE the arithmetic, because Verify is fail-fast:
a maximum restated low enough to change its band trips the margin equality below on the way
past, and a verdict clause underneath that one could not go red on its own. It is also the only
clause here that reads the code rather than the constants - the two inequalities say where the
reading sits, and only the classifier says what the shipped code does with it. */
Require(
TimescaleSupport.ClassifyRefreshSlotHeadroom(measured10 / 10.0)
== TimescaleSupport.RefreshSlotHeadroom.ApproachingSlot,
$"ClassifyRefreshSlotHeadroom bands {measured10 / 10}.{measured10 % 10} s as "
+ $"{TimescaleSupport.ClassifyRefreshSlotHeadroom(measured10 / 10.0)} rather than "
+ "ApproachingSlot, so the live envelope paragraph states a verdict the shipped classifier "
+ $"does not produce. Against a {Slot} s slot the warning band opens at {WatchLine} s");

Require(statedMargin10 == Slot * 10 - measured10,
$"stated slot margin {statedMargin10 / 10}.{statedMargin10 % 10} s against "
+ $"{(Slot * 10 - measured10) / 10}.{(Slot * 10 - measured10) % 10} s derived from the "
+ $"{Slot} s slot less the measured maximum");

/* The distance PAST the watch line, which also carries the strict-past claim: a maximum at or
below the line makes this difference zero or negative, and no figure the pattern can hold
equals that. */
Require(statedPastWatch10 == measured10 - WatchLine * 10,
$"stated distance past the watch line {statedPastWatch10 / 10}.{statedPastWatch10 % 10} s "
+ $"against {(measured10 - WatchLine * 10) / 10}.{(measured10 - WatchLine * 10) % 10} s "
+ $"derived from the {WatchLine} s line");

Require(statedShareTenths == statedMargin10 * 1000 / (Slot * 10),
$"stated {statedShareTenths / 10}.{statedShareTenths % 10}% of the slot against "
+ $"{statedMargin10 * 1000 / (Slot * 10) / 10}.{statedMargin10 * 1000 / (Slot * 10) % 10}% "
+ $"derived from {statedMargin10 / 10}.{statedMargin10 % 10} s over the {Slot} s slot, "
+ "truncated to one decimal as the prose states it");

/* AND ITS CLOSING SCOPE, the same rule the snapshot enumeration below carries. An absolute day
has ENDED and stays true; "recently", "the latest day" or a bare "currently" is read against a
clock a doc comment does not have, so it rots while looking scoped. */
Require(
Regex.IsMatch(ceilingProse,
@"Over <c>[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]</c> - one closed day"),
"the live envelope's population is no longer CLOSED by an absolute day, so those figures read "
+ "as a claim about current load and are false as soon as the load moves. Restore a window "
+ "that has ended, or drop the figures and keep the mechanism, which is the timeless half");

/* The EXCLUDED run. Its clock arithmetic has to reproduce its own stated duration, it has to end one
second after the stated boundary, and its ratio to the population maximum has to be the ratio the
prose draws — which is what keeps that paragraph about this population rather than free-floating. */
Expand Down
5 changes: 5 additions & 0 deletions Darling/Darling.Tests/TimescaleSupportTests.cs
Original file line number Diff line number Diff line change
Expand Up @@ -2656,6 +2656,11 @@ public void TheRefreshSlotLogLine_IsLeveledByBand_AndSaysNothingWithoutAReading(
approaching);
Assert.StartsWith("Warning:", approaching.Joined, StringComparison.Ordinal);

/* #3119: the Warning line names the route that can answer a maximum question. The hourly
self-metrics series serves a last-snapshot-per-day point, so it cannot stand alone as the
per-run history an operator following this line goes looking for. */
Assert.Contains("timescaledb_information.job_history", approaching.Joined, StringComparison.Ordinal);

var exceeded = new CapturingTestLogger();
TimescaleSupport.LogHeaviestRefreshSlotHeadroom(
new HeaviestRefreshSlotReading(
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -245,7 +245,7 @@ public static string Build(DarlingPeerDirectory.Snapshot peers)

### Store self-metrics

`get_store_metrics` reads the MONITORING STORE's own growth series — not a monitored SQL Server's. The service records an hourly self-metrics snapshot into the store: per-hypertable total size, pre/post-compression bytes and chunk count; the query-text / query-plan payload dimension tables' total size (the store's dominant payloads) and row counts; and the whole store's size with the enabled-server count. The tool returns the latest snapshot per object plus a daily series, including the whole-store daily growth in bytes and the derived per-server ingest rate (daily growth ÷ enabled servers) — the number to multiply when onboarding N servers. Use it for capacity forecasting: what is driving store growth and how fast. 400 days of history; no `server_name` parameter, because the store is the subject.
`get_store_metrics` reads the MONITORING STORE's own growth series — not a monitored SQL Server's. The service records an hourly self-metrics snapshot into the store: per-hypertable total size, pre/post-compression bytes and chunk count; the query-text / query-plan payload dimension tables' total size (the store's dominant payloads) and row counts; and the whole store's size with the enabled-server count. The tool returns the latest snapshot per object plus a daily series, including the whole-store daily growth in bytes and the derived per-server ingest rate (daily growth ÷ enabled servers) — the number to multiply when onboarding N servers. Each daily point is that day's LAST snapshot rather than its maximum, so a maximum question needs the per-run route the tool's own description names. Use it for capacity forecasting: what is driving store growth and how fast. 400 days of history; no `server_name` parameter, because the store is the subject.

| Tool | Purpose | Key Parameters |
|------|---------|----------------|
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ public sealed class DarlingMcpStoreMetricsTools
public const int MaxDaysBack = StoreSelfMetrics.RetentionDays;

[McpServerTool(Name = "get_store_metrics"), Description(
"Gets the monitoring store's OWN size and growth metrics — not a monitored SQL Server's. The service records an hourly self-metrics snapshot: per-hypertable total size, pre/post-compression bytes and chunk count; the query-text and query-plan payload dimension tables' total size (the store's dominant payloads) and row counts; the whole store's size with the enabled-server count; and one row per TimescaleDB background job (CAGG refresh, compression, retention) with its last run duration, schedule interval, duration-vs-cadence percent, and run/failure totals — the jobs whose runtimes scale with fleet size. Returns the latest snapshot per object plus a daily series over the window, with the whole-store daily growth in bytes and the derived per-server ingest rate (daily growth / enabled servers). Also reports, read LIVE from the catalog rather than from the recorded series, every retention policy the rollup-coverage gate is holding PAUSED, with the tier's actual data span and how many times its configured drop_after horizon it is really holding — a held policy records zero failures and a normal-looking last run, so it is invisible in the stored job telemetry and is a common cause of unexplained store growth. Use for capacity forecasting: what is driving store growth, how fast, what adding N servers would multiply, which background job is closest to outgrowing its own cadence, and whether retention is actually running.")]
"Gets the monitoring store's OWN size and growth metrics — not a monitored SQL Server's. The service records an hourly self-metrics snapshot: per-hypertable total size, pre/post-compression bytes and chunk count; the query-text and query-plan payload dimension tables' total size (the store's dominant payloads) and row counts; the whole store's size with the enabled-server count; and one row per TimescaleDB background job (CAGG refresh, compression, retention) with its last run duration, schedule interval, duration-vs-cadence percent, and run/failure totals — the jobs whose runtimes scale with fleet size. Returns the latest snapshot per object plus a daily series over the window, with the whole-store daily growth in bytes and the derived per-server ingest rate (daily growth / enabled servers). Each daily point is that day's LAST snapshot, never its maximum or its mean — the settled figure a growth question wants, but it means a MAXIMUM question (what was this job's longest run that day, did it enter its warning band) cannot be answered from this series: the day's peak is DROPPED rather than smoothed, so a day whose worst run crossed a threshold reads as a day that never approached it. The route that carries one row per run is TimescaleDB's own job history (timescaledb_information.job_history). The hourly snapshot behind the series has the same limit one grain down: it samples last_run_duration once an hour at a fixed offset, so a run longer than that offset is never recorded at all. Also reports, read LIVE from the catalog rather than from the recorded series, every retention policy the rollup-coverage gate is holding PAUSED, with the tier's actual data span and how many times its configured drop_after horizon it is really holding — a held policy records zero failures and a normal-looking last run, so it is invisible in the stored job telemetry and is a common cause of unexplained store growth. Use for capacity forecasting: what is driving store growth, how fast, what adding N servers would multiply, which background job is closest to outgrowing its own cadence, and whether retention is actually running.")]
public static async Task<string> GetStoreMetrics(
NpgsqlDataSource postgres,
[Description("Days of daily-series history. Default 30; max 400 (the series' own retention).")] int days_back = 30)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,9 @@ namespace PerformanceMonitor.Darling.Service.Mcp;
/// snapshot per object (one row per hypertable / dimension / the store itself, at each object's newest
/// metric_time), and the daily series — the LAST sample of each object per day, which is what a
/// growth/forecast question wants (the hourly grain exists so a single missed run costs nothing, not
/// because anyone forecasts by the hour). Plus the one derivable number the issue called out: the
/// because anyone forecasts by the hour) and what a MAXIMUM question cannot use: the day's largest
/// reading is dropped rather than smoothed, so a peak that crossed a threshold is invisible at this
/// grain (#3119). Plus the one derivable number the issue called out: the
/// per-server daily ingest rate (whole-store daily growth divided by the enabled-server count), computed in
/// <see cref="ComputeDailyGrowth"/> — pure, so it is unit-tested without a store.
/// </summary>
Expand Down Expand Up @@ -48,7 +50,10 @@ FROM collect.store_metrics

/// <summary>The daily series — the LAST sample of each object per day (DISTINCT ON over the day
/// bucket, newest first within it), so each day contributes one settled point per object rather than
/// 24 near-duplicates. $1 window start (naive UTC).</summary>
/// 24 near-duplicates. A last snapshot and not a per-day maximum: the tiebreak takes the newest row
/// in the bucket, so the day's largest value is discarded unless it happens to be the last one, and
/// <c>get_store_metrics</c>' description says so where a caller reads it (#3119). $1 window start
/// (naive UTC).</summary>
public const string StoreMetricsDailySql = @"
SELECT DISTINCT ON (object_kind, object_name, date_trunc('day', metric_time))
object_kind,
Expand Down
Loading
Loading