You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
resource_digest children_rss misses host-service's own process #6700
The resource_digest telemetry (#6697) reports child-process memory by summing terminal-session subtrees from the resource-metrics collector. The host-service daemon itself (and pty-daemon) belong to no workspace session, so a host-service memory leak is invisible to the one metric that reads as "everything outside Electron". host-service is our top crasher (7.7k SIGABRT events in the last 14 days on Sentry), so this is the process we most want a memory series for.
apps/desktop/src/main/lib/resource-digest/resource-digest.ts — SAMPLE_KEYS and collectDefaultSample
apps/desktop/src/main/lib/host-service-coordinator.ts — knows the spawned host-service child PIDs
apps/desktop/src/main/lib/resource-metrics/process-tree.ts — existing single-ps-snapshot helpers to read RSS for a PID set
Approach
Add a host_service_rss_bytes sample key: ask the coordinator for its live child PIDs and sum their RSS (macOS phys_footprint where available, matching the collector). Consider a matching host_service_rss_high alert threshold.
Gotchas
Sum across orgs (the coordinator can run one host-service per org).
A dead/restarting host-service should contribute 0, not fail the sample.
This is a follow-up task from #6697 rather than a user-visible bug: resource_digest's child-process memory sample only ever walks subtrees rooted at terminal session PIDs, so the host-service daemon (and the pty-daemon it supervises) are invisible to the one number that reads as "everything outside Electron" — which is unfortunate given host-service is the top SIGABRT source. I confirmed the gap against the collector code, and the approach in the issue is sound, but there are several traps in the current coordinator/process-tree APIs that will bite a naive implementation. Note up front: apps/desktop/src/main/lib/resource-digest/ does not exist at main (b104fd0) — no file in the repo mentions resource_digest, SAMPLE_KEYS, collectDefaultSample, or children_rss, so #6697 hasn't landed here yet and this work has to be based on that branch.
The gap is real
The only PID roots the collector knows about come from terminal sessions:
apps/desktop/src/main/lib/resource-metrics/session-sources.ts:30 (v1, from the runtime registry's listSessions) and :70 (v2, from host-service's /terminal/resource-sessions endpoint).
apps/desktop/src/main/lib/resource-metrics/index.ts:275-278 expands exactly those roots via getSubtreePids, and :326 sums each session subtree.
Nothing ever adds the host-service PID as a root. collectV2WorkspaceSessionMap talks to host-service over HTTP (session-sources.ts:91) but only asks for its sessions; the serving process's own memory never enters the snapshot.
Implementation traps in the existing code
The coordinator exposes no PID accessor.instances is private (host-service-coordinator.ts:171) and the public surface is getConnection/getConnections/getProcessStatus/getActiveOrganizationIds (:396-426) — port, secret, machineId, status, org ids, no PIDs. You'll need something like getChildPids(): { organizationId: string; pid: number; owned: boolean }[].
A starting instance has pid: 0, and PID 0 is a machine-wide footgun. The instance is registered with pid: 0 at host-service-coordinator.ts:699-715 and only assigned the real PID at :780. If a 0 reaches getSubtreePids, childrenOf.get(0) is not empty — on both macOS and Linux PID 1 reports ppid 0 (process-tree.ts:60-67 builds the map straight from ps -eo ppid=), so the walk descends into the entire process table and the sample reports the whole machine's RSS as host-service. Guard with pid > 1, mirroring isSignalablePid in host-service-manifest.ts:126.
Adopted instances aren't ours.tryAdopt registers entries with owned: false (host-service-coordinator.ts:663-675) pointing at a host-service another live app instance spawned. If you include them, two running desktop instances both report the same RSS under the same key; if you exclude them, an adopting instance reports 0 for an org that is genuinely running. Either is defensible — pick one deliberately and tag the sample (owned/adopted) so the series is interpretable.
Own PID only, not the subtree. In v2, terminal PTYs live under the pty-daemon that host-service spawns (packages/host-service/src/daemon/DaemonSupervisor.ts:1193), so a subtree sum from the host-service PID double-counts everything already in children_rss. Sum the host-service process alone.
The pty-daemon is not reliably a descendant. It's spawned detached: !isDev (DaemonSupervisor.ts:1194) and is explicitly designed to outlive host-service restarts (packages/host-service/src/daemon/manifest.ts:1-3, plus the adopt path around DaemonSupervisor.ts:916), so after any host-service crash the surviving daemon is reparented to PID 1 and drops out of the tree — i.e. it would vanish from the metric exactly during the crash loop you're trying to measure. Read its PID from readPtyDaemonManifest/listPtyDaemonManifests (manifest.ts:95, :139) plus isProcessAlive (:164) instead, and emit it as a separatepty_daemon_rss_bytes key — folding it into host_service_rss_bytes makes a leak un-attributable. (v1's daemon is spawned by Electron main instead: apps/desktop/src/main/lib/terminal-host/client.ts:1260-1266.)
phys_footprint parity is opt-in per PID.enrichWithPhysFootprint (process-tree.ts:132-149) only rewrites the PIDs you hand it and no-ops off macOS. If the host-service PIDs aren't in that list, the new key is RSS while children_rss is phys_footprint, and the two aren't comparable on the platform most of your users are on.
Cost.captureProcessSnapshot shells out to ps -eo with a 5s timeout (process-tree.ts:156). Reuse the digest's existing snapshot if it has one rather than adding a second ps per sample.
"Dead host-service contributes 0"
That mostly falls out for free — snapshot.byPid.get(pid) returns undefined after exit and contributes nothing — but be aware handleChildExitdeletes the instance (host-service-coordinator.ts:957) before the respawn timer fires, so during a crash loop there is no entry at all. Emit an explicit 0 rather than omitting the key, and consider pairing it with host_service_uptime_ms (the coordinator already tracks spawnedAt, :703): without it, a respawn resets RSS to baseline and a leak reads as a fix.
Reproduction / verification
Manual, on top of the #6697 branch: start the app, jq .pid ~/.superset/host/<org>/manifest.json, then ps -o pid,ppid,rss -p <pid> and compare against the emitted sample; leave it running and confirm the series tracks Activity Monitor's "Memory" column for that process rather than RSS on macOS. Then kill -9 the PID and confirm the sample reads 0 through the respawn window instead of throwing or dropping the key.
For a bun:test, keep the arithmetic in a pure helper — sumHostServiceRss(snapshot: ProcessSnapshot, pids: number[]) — and drive it with a hand-built ProcessSnapshot (the pattern in resource-metrics/process-tree.test.ts), covering: PID 0/starting (must be 0, not the whole tree), a PID absent from the snapshot (dead → 0), two orgs (summed), and an adopted entry (whatever you decide). If any test reaches listPtyDaemonManifests, note assertIsolatedDaemonNamespaceInTests (manifest.ts:52) throws under NODE_ENV=test unless SUPERSET_HOME_DIR points at a temp dir.
Suggested shape
getChildPids() on the coordinator (filtering status === "running" and pid > 1, exposing owned) → digest collects those plus daemon-manifest PIDs → both PID sets are appended to the list passed to enrichWithPhysFootprint → emit host_service_rss_bytes (own PID only, summed across orgs, 0 when nothing is running) and pty_daemon_rss_bytes separately. I'd hold off on host_service_rss_high until there's a week of the raw series to pick a threshold from — a static byte threshold on a process whose baseline varies with workspace count will mostly alert on large-but-healthy installs.
Activity
github-actions commentedon Aug 20, 2026
Summary
This is a follow-up task from #6697 rather than a user-visible bug:
resource_digest's child-process memory sample only ever walks subtrees rooted at terminal session PIDs, so the host-service daemon (and the pty-daemon it supervises) are invisible to the one number that reads as "everything outside Electron" — which is unfortunate given host-service is the top SIGABRT source. I confirmed the gap against the collector code, and the approach in the issue is sound, but there are several traps in the current coordinator/process-tree APIs that will bite a naive implementation. Note up front:apps/desktop/src/main/lib/resource-digest/does not exist atmain(b104fd0) — no file in the repo mentionsresource_digest,SAMPLE_KEYS,collectDefaultSample, orchildren_rss, so #6697 hasn't landed here yet and this work has to be based on that branch.The gap is real
The only PID roots the collector knows about come from terminal sessions:
apps/desktop/src/main/lib/resource-metrics/session-sources.ts:30(v1, from the runtime registry'slistSessions) and:70(v2, from host-service's/terminal/resource-sessionsendpoint).apps/desktop/src/main/lib/resource-metrics/index.ts:275-278expands exactly those roots viagetSubtreePids, and:326sums each session subtree.Nothing ever adds the host-service PID as a root.
collectV2WorkspaceSessionMaptalks to host-service over HTTP (session-sources.ts:91) but only asks for its sessions; the serving process's own memory never enters the snapshot.Implementation traps in the existing code
The coordinator exposes no PID accessor.
instancesis private (host-service-coordinator.ts:171) and the public surface isgetConnection/getConnections/getProcessStatus/getActiveOrganizationIds(:396-426) — port, secret, machineId, status, org ids, no PIDs. You'll need something likegetChildPids(): { organizationId: string; pid: number; owned: boolean }[].A
startinginstance haspid: 0, and PID 0 is a machine-wide footgun. The instance is registered withpid: 0athost-service-coordinator.ts:699-715and only assigned the real PID at:780. If a 0 reachesgetSubtreePids,childrenOf.get(0)is not empty — on both macOS and Linux PID 1 reportsppid0 (process-tree.ts:60-67builds the map straight fromps -eo ppid=), so the walk descends into the entire process table and the sample reports the whole machine's RSS as host-service. Guard withpid > 1, mirroringisSignalablePidinhost-service-manifest.ts:126.Adopted instances aren't ours.
tryAdoptregisters entries withowned: false(host-service-coordinator.ts:663-675) pointing at a host-service another live app instance spawned. If you include them, two running desktop instances both report the same RSS under the same key; if you exclude them, an adopting instance reports 0 for an org that is genuinely running. Either is defensible — pick one deliberately and tag the sample (owned/adopted) so the series is interpretable.Own PID only, not the subtree. In v2, terminal PTYs live under the pty-daemon that host-service spawns (
packages/host-service/src/daemon/DaemonSupervisor.ts:1193), so a subtree sum from the host-service PID double-counts everything already inchildren_rss. Sum the host-service process alone.The pty-daemon is not reliably a descendant. It's spawned
detached: !isDev(DaemonSupervisor.ts:1194) and is explicitly designed to outlive host-service restarts (packages/host-service/src/daemon/manifest.ts:1-3, plus the adopt path aroundDaemonSupervisor.ts:916), so after any host-service crash the surviving daemon is reparented to PID 1 and drops out of the tree — i.e. it would vanish from the metric exactly during the crash loop you're trying to measure. Read its PID fromreadPtyDaemonManifest/listPtyDaemonManifests(manifest.ts:95,:139) plusisProcessAlive(:164) instead, and emit it as a separatepty_daemon_rss_byteskey — folding it intohost_service_rss_bytesmakes a leak un-attributable. (v1's daemon is spawned by Electron main instead:apps/desktop/src/main/lib/terminal-host/client.ts:1260-1266.)phys_footprint parity is opt-in per PID.
enrichWithPhysFootprint(process-tree.ts:132-149) only rewrites the PIDs you hand it and no-ops off macOS. If the host-service PIDs aren't in that list, the new key is RSS whilechildren_rssis phys_footprint, and the two aren't comparable on the platform most of your users are on.Cost.
captureProcessSnapshotshells out tops -eowith a 5s timeout (process-tree.ts:156). Reuse the digest's existing snapshot if it has one rather than adding a secondpsper sample."Dead host-service contributes 0"
That mostly falls out for free —
snapshot.byPid.get(pid)returnsundefinedafter exit and contributes nothing — but be awarehandleChildExitdeletes the instance (host-service-coordinator.ts:957) before the respawn timer fires, so during a crash loop there is no entry at all. Emit an explicit0rather than omitting the key, and consider pairing it withhost_service_uptime_ms(the coordinator already tracksspawnedAt,:703): without it, a respawn resets RSS to baseline and a leak reads as a fix.Reproduction / verification
Manual, on top of the #6697 branch: start the app,
jq .pid ~/.superset/host/<org>/manifest.json, thenps -o pid,ppid,rss -p <pid>and compare against the emitted sample; leave it running and confirm the series tracks Activity Monitor's "Memory" column for that process rather than RSS on macOS. Thenkill -9the PID and confirm the sample reads 0 through the respawn window instead of throwing or dropping the key.For a
bun:test, keep the arithmetic in a pure helper —sumHostServiceRss(snapshot: ProcessSnapshot, pids: number[])— and drive it with a hand-builtProcessSnapshot(the pattern inresource-metrics/process-tree.test.ts), covering: PID 0/starting(must be 0, not the whole tree), a PID absent from the snapshot (dead → 0), two orgs (summed), and an adopted entry (whatever you decide). If any test reacheslistPtyDaemonManifests, noteassertIsolatedDaemonNamespaceInTests(manifest.ts:52) throws underNODE_ENV=testunlessSUPERSET_HOME_DIRpoints at a temp dir.Suggested shape
getChildPids()on the coordinator (filteringstatus === "running"andpid > 1, exposingowned) → digest collects those plus daemon-manifest PIDs → both PID sets are appended to the list passed toenrichWithPhysFootprint→ emithost_service_rss_bytes(own PID only, summed across orgs,0when nothing is running) andpty_daemon_rss_bytesseparately. I'd hold off onhost_service_rss_highuntil there's a week of the raw series to pick a threshold from — a static byte threshold on a process whose baseline varies with workspace count will mostly alert on large-but-healthy installs.