Skip to content

[QUESTION] Add new server to monitoring #2252

Description

@000al000

Which component is your question about?

Darling

Performance Monitor Version

3.4.0 (server) / 3.3.0 (viewer)

Is your question about how it works, or the results?

How the tool works

What's your question?

Configuration:
Darling headless (on windows server)
Viewer on remote PC (downloaded as version 3.4, but in fact 3.3)

One server was added and start to darling.json and working fine
Try to add second server to darling.json, --test-connection seen 2 servers, result ok

Try restarting service and connect with viewer - but second server not seeing in viewer

How tell server to "reread" servers config from json?

Additional Context

No response

Activity

  1. 000al000 commented on Aug 14, 2026

    @000al000
    Author

    If i run PerformanceMonitor.Darling.Service.exe i cant see in console output second server

    config
    "servers": [
    {
    "name": "plvs-itebd",
    "host": "plvs-itebd",
    "auth": "integrated",
    "trustServerCertificate": true,
    "monthlyCostUsd": 0,
    "alertDeliveryModeOverride": null,
    "excludedDatabases": []
    },
    {
    "name": "dcvs-bd01",
    "host": "dcvs-bd01",
    "auth": "sql",
    "username": "PerfomanceMonitor",
    "encryptedPassword": "hided_from_all",
    "monthlyCostUsd": 0,
    "trustServerCertificate": true,
    "alertDeliveryModeOverride": null,
    "excludedDatabases": []
    }
    ],

  2. 000al000 commented on Aug 14, 2026

    @000al000
    Author

    PS C:\PerformanceMonitorDarling> .\PerformanceMonitor.Darling.Service.exe --test-connection
    Validating connectivity to 2 server(s)...
    [PASS] plvs-itebd: SQL major version 13, Enterprise, msdb access: yes
    [PASS] dcvs-bd01: SQL major version 14, Standard, msdb access: yes
    All servers reachable.

  3. 000al000 commented on Aug 14, 2026

    @000al000
    Author

    Error, if i try to add from remote viewer

    warn: PerformanceMonitor.Darling.Service.DarlingWorker[0]
    [dcvs-bd01] Connect failed, retrying in 60s: Key not valid for use in specified state

  4. erikdarlingdata commented on Aug 14, 2026

    @erikdarlingdata
    Owner

    Your config is fine and the tool is behaving as built — but the behaviour is wrong, and you've found two separate faults. Nothing you can put in darling.json will fix the first one.

    Why the second server never appears

    darling.json seeds the server list exactly once, and after that the store is authoritative. From StoreConfigProvider:

    if (await CountAsync(connection, "config_monitored_servers", cancellationToken) == 0)
    {
        await SeedMonitoredServersAsync(connection, config, now, cancellationToken);
    }

    Once that table holds a single row, every later start skips the seed and the service takes its server set from the store instead (config_monitored_servers WHERE is_enabled = TRUE). So adding dcvs-bd01 to the file after your first successful start is a no-op, and restarting the service cannot help — there is no "re-read the JSON" path, by design.

    That also explains why your evidence looks contradictory:

    • --test-connection reads darling.json directly, which is why it validated 2 servers and both passed. It never consults the store.
    • The service and Viewer read the store, which has 1 server.

    Both outputs are correct; they are answering different questions. That is our bug, not your misreading — the service should tell you it is ignoring servers present in the file, and today it says nothing.

    To add the server now: use the Viewer's Add Server dialog, or the MCP add_servers tool. Either writes to config_monitored_servers, which is the path the running service actually reads.

    Your second error is a different fault

    [dcvs-bd01] Connect failed, retrying in 60s: Key not valid for use in specified state
    

    That is a DPAPI failure, not SQL Server refusing you. A stored credential was encrypted in one Windows context and is being decrypted in another — the classic shape is a credential saved by the Viewer on your remote PC (encrypted for that user/machine) which the service account on the server then cannot decrypt. It is consistent with your report that the failure appears specifically when adding from the remote Viewer.

    If dcvs-bd01 uses integrated auth like your first server, the cleanest workaround while I fix this is to add it from a Viewer running on the Darling host itself, or via add_servers, so no credential crosses machines.

    What I'm filing

    1. The service silently ignores servers added to darling.json after the first seed. At minimum it should log, on every start, that the file lists N servers the store does not have and name them — this exact question has now cost you a config edit, a service restart and a support round trip.
    2. The remote-Viewer credential path produces an undiagnosable DPAPI error. "Key not valid for use in specified state" tells an operator nothing; it should name what could not be decrypted and why a remote Viewer's credential cannot be used by the service.

    Also worth knowing since you mentioned it: your Viewer reporting 3.3.0 while the download said 3.4 is a known packaging problem from that release, unrelated to this.

  5. 000al000 commented on Aug 14, 2026

    @000al000
    Author

    I've got "situation" :))

    Service working on Win2012 and can't run Viewer on this Win (gemini say it too old for viewer), so cant add server from viewer

    So i only have web view option + remote viewer

    Is there any way to add new servers?

  6. erikdarlingdata commented on Aug 14, 2026

    @erikdarlingdata
    Owner

    That is a real corner, and you've found a genuine gap rather than a missing menu item. Short answer: the web viewer cannot add servers, and there is no CLI verb either. Here is what does work, and what I'm filing.

    Why the web viewer can't

    It is deliberately a read-only surface. add_servers and remove_server are in ExcludedToolNames — the list of tools kept off /api/read/* precisely because they WRITE the monitored-server registry. So there is nothing to enable; the capability isn't there.

    And I checked the full CLI verb list before answering: --test-connection, --validate-config, --enable-mcp, --enable-web, --encrypt-password, --export-viewer-config, --print-viewer-connection, --backfill-rollups, --recompress-plan-dim, --vacuum-full, --configure-network, --configure-firewall, --config, --version, --help. No add-server verb. On a headless host with no local GUI, that leaves you with no supported path — which is the bug, not your situation.

    What works today, in the order I'd try it

    1. Your remote Viewer — and it may already be fine. Your first server uses "auth": "integrated". Integrated auth stores no password, so there is no DPAPI blob to encrypt or decrypt. If dcvs-bd01 is also integrated, the credential path that produced Key not valid for use in specified state shouldn't be involved at all. Worth one more attempt: add it as integrated, and if that error returns, tell me the exact dialog fields you used — that would mean something is writing an encrypted blob even when there is no password, which is a sharper bug than the one I filed.

    2. Enable the MCP surface and call add_servers. --enable-mcp exists for exactly this shape of host. It is the right path for a headless box: add_servers validates each entry, probes the connection in-process, and — importantly — computes the server identity correctly (see below).

    3. Direct SQL, but I'd rather you didn't, and here is the specific reason. server_id is not free-form: it is a deterministic hash of the storage name (host + database + read-only intent). Insert a row with any other value and the collectors will write history under the identity they compute while the registry points at a different one — the exact class of split-history problem tracked in #2158. So a hand-written INSERT is not a shortcut, it is a trap unless you can reproduce that hash.

    One useful thing if you do end up going near the store: you do not need to restart the service. It polls config_service.config_version every sweep and hot-swaps its held config when the value changes, so bumping that column is the supported way to make it re-read. That is also why restarting never helped you — the restart wasn't the missing ingredient, the row was.

    Filing

    A headless host has no supported way to add a server. Windows Server 2012 can't run the Viewer, the web surface excludes the write tools by design, and there is no CLI verb — so the only routes are a GUI on another machine or an MCP client. An --add-server verb (reusing exactly what add_servers does, identity computation and connection probe included) would close it. Filing that now, on top of #2254 (the silent darling.json ignore) and #2255 (the DPAPI diagnostic).

    Thank you for pushing on this — the first report exposed the seed behaviour, and this one exposes that we have no answer at all for a headless install.

  7. erikdarlingdata commented on Aug 14, 2026

    @erikdarlingdata
    Owner

    Both gaps you found are now fixed on dev, so I'm closing this and leaving the one remaining fault tracked separately.

    Your Windows Server 2012 corner — #2256, merged as #2259. There is now a CLI verb, so the box that cannot run the Viewer can register servers itself:

    type servers.json | PerformanceMonitor.Darling.Service.exe --add-server
    

    JSON array on stdin, same shape the add_servers MCP tool takes, and it goes through that same code path — validation, dedupe, the connection probe, password encryption and the server_id identity are shared rather than reimplemented. Stdin rather than an argument because a password in argv shows up in the process list and in shell history. Run it with empty stdin and it prints the JSON shape plus two copy-paste pipelines. It reports one line per server with the probe result, and no restart is needed — the registry write bumps a version beacon the service polls every sweep. Exit 0 requires that something landed and nothing failed, so you can use it as a deployment gate; re-running the same file is idempotent.

    The silent part — #2254, merged as #2257. The service now names, at startup, any server in darling.json that is not in the store. That is what should have told you the second server was being ignored instead of leaving you to infer it. It compares on server_id rather than on name, so a same-name-different-host entry is still reported as absent.

    Still open: #2255 — Key not valid for use in specified state when adding from a remote Viewer. That is a real bug and a different one: the Viewer DPAPI-encrypts the password on the machine you are sitting at, and the service cannot decrypt a LocalMachine-scoped blob produced on a different host. Tracked there rather than here.

    Both fixes are on dev and ride the next release. Thanks for pushing on this — the "no supported way to add a server at all" case was genuinely not covered, and it took a real deployment to surface it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    questionFurther information is requested

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions