Store Job Over Cadence is mis-tuned and mis-worded for the hourly refresh: its remedy changes the materialised window and its default sits one point from firing #3060

#2136's Store Job Over Cadence alert judges a background job's last_run_duration against a percentage of its schedule_interval. Two things about it are wrong for the heaviest hourly continuous-aggregate refresh, and both are operator-facing.

1. Its remedy text is actively wrong for this job

The alert tells the operator to extend the job's schedule_interval. For the hourly refresh family, HourlyRefreshScheduleInterval is also the aggregate's end_offset — so extending it does not just relax the deadline, it changes what window the refresh materialises. Following the instruction silently alters collection behaviour to quiet an alert.

2. Its default sits one percentage point from firing on a healthy run

Verified arithmetically against origin/dev:

So at its recorded worst this job runs at 24.0% of cadence against a 25% default. One point of margin, on a figure that has been observed at 66% of its slot this evening and rises with retail load.

And #2136's clamp is justified on a number that does not describe this job. Its reasoning cites "the production worst runs ~7% of cadence" — 252 s. The job in question has a recorded ceiling 3.4x that. The clamp was chosen against a population this job is not in.

3. The coincidence that makes it look fine is fragile in both directions

The 25% default landing exactly on one slot is an accident of two independent decisions — nobody tied StoreJobOverCadence's knob to RefreshPhaseStepMinutes. The knob is an operator setting clamped [5,100] with no reference to the slot constant, so:

• at 50 it fires at 1,800 s — two slots, a full slot past the point where Pin the compression policies to a phase grid clear of the refresh slots (#3035) #3041's compression grid is invalid;
• at the clamp's top 100 it fires at 3,600 s — four slots past;
• and if RefreshPhaseStepMinutes were changed to 10, the slot becomes 600 s while a 25% knob still fires at 900 s — past invalidation rather than before it.

In every one of those cases the alert fires after the condition it would need to warn about.

What this is not

Not a request for a second alert. PR #3055 (#3044) deliberately took the quiet form for exactly this reason: the alert seam is occupied, and a second signal at the slot width would double-fire on the same job, reading and hour. This issue is about the occupant being mis-tuned and mis-worded, not about adding another.

Fix, and the part worth guarding

The remedy text is the user-visible half and the cheaper one: it should not tell an operator to extend an interval that doubles as an end_offset, or should name the exception.

The coupling is the durable half. Either tie the knob's validation to RefreshPhaseSlotSeconds so a setting that fires past invalidation is rejected, or assert the relationship in a test so the two constants cannot drift apart silently. Right now nothing connects them, and the thing that makes the default look correct is a coincidence.

Also worth correcting while there: the DarlingAlertSettings comment justifying the clamp on "~7% of cadence" is stale against this job's measured behaviour.

Provenance

Found by the #3044 lane while establishing that a new alert was not the right shape — it checked what already occupies the seam rather than assuming it was empty. The arithmetic above is recomputed here from RefreshPhaseSlots, RefreshPhaseStepMinutes and HeaviestHourlyRefreshObservedCeilingSeconds on origin/dev.