[BUG] Azure collector failures are swallowed into status=SUCCESS, and get_collection_health then reports HEALTHY with a finding that says the opposite of what happened #3754

Component

Darling

Performance Monitor Version

3.8.0-nightly.20260919.453

Describe the Bug

On Azure SQL DB, two collectors fail on every monitored database, every sweep, and both record status = SUCCESS, rows_collected = 0. get_collection_health then reports them HEALTHY with errors: 0 and tells the reader, in prose, that the collector read successfully and the source was genuinely empty.

The failures are real and visible in the service log:

[WARN] [DarlingWorker] [<server>] [xedb1] Failed to reconcile the long-query completion XE session: The action 'sqlserver.server_principal_name' is not available for Azure SQL Database.
[WARN] [DarlingWorker] [<server>] [xedb2] Failed to reconcile the long-query completion XE session: The action 'sqlserver.server_principal_name' is not available for Azure SQL Database.
[WARN] [DarlingWorker] Failed to collect database_scoped_config from [xedb1] on '<server>': Reference to database and/or server name in 'xedb1.sys.sp_executesql' is not supported in this version of SQL Server.
[WARN] [DarlingWorker] Failed to collect database_scoped_config from [xedb2] on '<server>': Reference to database and/or server name in 'xedb2.sys.sp_executesql' is not supported in this version of SQL Server.

But the store says:

long_query_completions | status=SUCCESS | rows=0 | err=-
database_scoped_config | status=SUCCESS | rows=0 | err=-

And get_collection_health says, verbatim, for both:

"status":"HEALTHY" ... "errors":0 ... "output_finding":"Stored 0 rows across 1 runs with no current denial (denied_since_last_success is false), so this collector read and found nothing rather than being unable to read. For one that stores a row only when an event occurs - a deadlock, a blocked-process report, a blocking chain, a held xmin - zero is the correct resting state on a well-behaved target and needs no action. No run recorded a note, which is what that reading rests on."

Every clause of that is wrong here:

• "read and found nothing rather than being unable to read" — both were unable to read. One never got its XE session created; the other's query was rejected on all three fan-out items.
• "needs no action" — both need action; neither will ever collect on Azure without a code change.
• "stores a row only when an event occurs" — database_scoped_config is not an event-triggered collector at all; it is a config snapshot that should return a row per setting per database. Zero from it is never a resting state.
• "No run recorded a note, which is what that reading rests on" — the reading rests on an absence that is itself the bug: the failures were logged to the service log and never recorded on the run.

The fanout block on the same rows even shows the work happening (items: 3, slowest: xedb1, 509 ms for long_query_completions), so cost was attributed to runs that accomplished nothing.

Why this matters more than the two underlying Azure bugs. Those are ordinary platform gaps and are filed separately (#3753 for the action; the sp_executesql one is adjacent). This is the surface an operator — or an agent reading MCP — consults to decide whether monitoring is trustworthy, and on a server where two collectors are completely broken it returns HEALTHY and a sentence asserting the opposite. A silent zero that reads as a confident all-clear is the failure mode the health surface exists to prevent.

Expected Behavior

A collector whose per-item work threw should not land as SUCCESS. Minimally, the swallowed per-item exception is recorded on the run (the notes / error_message channel blocked_process_report already uses for events_read=0 ...), so that:

• collection_log shows a non-SUCCESS status, or at least a note naming the failure;
• get_collection_health stops asserting "read and found nothing rather than being unable to read" for a run that recorded a fault;
• the output_finding prose's event-collector framing is not applied to config collectors.

Additional Context

Found while opting the fleet into every XE session and adding a fresh Azure SQL DB target to exercise the database-scoped path. On-prem targets are unaffected — the equivalent collectors there genuinely do read.

Secondary observation, not claimed as a bug: the same payload reports "target_has_user_databases": false for this Azure server, which has two user databases (xedb1, xedb2) that other collectors fan out across in the same response. Worth a look while in here.

Repro: register an Azure SQL DB logical server with 2+ databases, enable long_query_completions, wait one sweep, then compare the service log against get_collection_health.