Skip to content

Wait for the process group in NodeChildProcessSpawner release and kill - #8018

Merged
tim-smart merged 6 commits into
mainfrom
eff-1226/childprocess-process-group-cleanup-tests
Sep 4, 2026
Merged

tim-smart merged 6 commits into
mainfrom
eff-1226/childprocess-process-group-cleanup-tests

Conversation

@tim-smart

@tim-smart tim-smart commented Sep 4, 2026 •

Copy link
Copy Markdown
Contributor

Summary

NodeChildProcessSpawner signalled the process group on scoped release and kill but only waited for the leader's exit. When the leader dies before its descendants (for example a sh -c wrapper that does not exec, as in #7821), release returned early, descendants kept running and forceKillAfter never escalated.

Release and kill now wait for the leader's exit and then for the rest of the process group to terminate, based on process existence (kill(-pgid, 0)) rather than on stdio. This is a replacement for the approach in #7875, which waited on Node's close event and could hang release when a pipe was held or left unread.

Closes #7821
Closes EFF-1226

Semantics

  • exitCode and isRunning stay tied to the leader's exit.
  • Without forceKillAfter: after the leader exits, the remaining group members get up to 1 second to exit. No SIGKILL is ever sent.
  • With forceKillAfter: the group wait is unbounded until the timeout fires, then the group receives SIGKILL followed by a final 1 second bounded wait (a non-reaping PID 1 can leave zombies in the group).
  • Neither path waits on Node's close, so a descendant holding an inherited pipe or unread backpressured stdout cannot hang release.
  • A leader that already exited 0 still leaves its group alone. A non-zero exit still signals the group once without waiting, as before.
  • Windows keeps the exit wait only; taskkill /T /F already terminates the tree.
  • Both the group poll and the forceKillAfter deadline use native time, so cleanup and escalation are not governed by the Effect clock (TestClock tests). The leader's exit event wakes the check immediately, so an ordinary release does not pay the 10ms poll interval.
  • Group membership is checked with kill(-pgid, 0), which counts zombies. Under a non-reaping PID 1 every release pays the full wait; documented in the changeset and module docs.

Tests

The first commit adds test/fixtures/process-group.ts and five tests. Three failed on main and pass with the fix:

  • scope release waits for descendants that outlive the leader
  • scope release force kills descendants that ignore the kill signal
  • kill force kills descendants that ignore the kill signal

One more fails when escalation runs on the Effect clock and passes with the native deadline:

  • forceKillAfter escalation does not depend on the Effect clock (it.effect, so under TestClock)

Two guard the #7875 failure modes and pass before and after:

  • scope release returns when a descendant holds the inherited pipe without forceKillAfter
  • scope release returns when stdout is unread and backpressured

The fixture is plain Node: a leader with the default SIGTERM disposition and a same-group descendant with inherited stdio, so it does not depend on /bin/sh exec behaviour. Assertions use marker and heartbeat files, not the stdout stream.

Test plan

  • pnpm lint
  • pnpm check in packages/platform/node-shared and packages/effect
  • pnpm jsdocs --check (no diagnostics for the touched file)
  • pnpm test --run packages/platform/node-shared/test/NodeChildProcessSpawner.test.ts: 75 passed
  • pnpm test --run packages/effect/test/unstable/process/ChildProcess.test.ts: 30 passed
  • pnpm test --run packages/tools/api-diff/test/Worktrees.test.ts: 1 passed

@changeset-bot

changeset-bot Bot commented Sep 4, 2026 •

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: c2230a3

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 30 packages
Name Type
@effect/platform-node-shared Patch
effect Patch
@effect/opentelemetry Patch
@effect/vitest Patch
@effect/ai-anthropic Patch
@effect/ai-openai-compat Patch
@effect/ai-openai Patch
@effect/ai-openrouter Patch
@effect/atom-react Patch
@effect/atom-solid Patch
@effect/atom-vue Patch
@effect/platform-browser Patch
@effect/platform-bun Patch
@effect/platform-deno Patch
@effect/platform-node Patch
@effect/sql-clickhouse Patch
@effect/sql-d1 Patch
@effect/sql-libsql Patch
@effect/sql-mssql Patch
@effect/sql-mysql2 Patch
@effect/sql-pg Patch
@effect/sql-pglite Patch
@effect/sql-sqlite-bun Patch
@effect/sql-sqlite-do Patch
@effect/sql-sqlite-node Patch
@effect/sql-sqlite-react-native Patch
@effect/sql-sqlite-wasm Patch
@effect/docgen Patch
@effect/doctest Patch
@effect/openapi-generator Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@ghost ghost added the 4.0 label Sep 4, 2026
@tim-smart tim-smart changed the title Add failing regression tests for NodeChildProcessSpawner process group cleanup Wait for the process group in NodeChildProcessSpawner release and kill Sep 4, 2026
@ghost ghost added the bug Something isn't working label Sep 4, 2026
@github-actions

github-actions Bot commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Bundle Size Analysis

Generated from PR build output; treat the content below as untrusted.

File Name Current Size Previous Size Difference
arbitrary-combinators.ts 34.05 KB 34.05 KB 0.00 KB (0.00%)
basic.ts 7.15 KB 7.15 KB 0.00 KB (0.00%)
batching.ts 10.38 KB 10.38 KB 0.00 KB (0.00%)
brand.ts 6.63 KB 6.63 KB 0.00 KB (0.00%)
cache.ts 11.03 KB 11.03 KB 0.00 KB (0.00%)
config.ts 21.75 KB 21.75 KB 0.00 KB (0.00%)
differ.ts 20.55 KB 20.55 KB 0.00 KB (0.00%)
http-client.ts 22.20 KB 22.20 KB 0.00 KB (0.00%)
http-router.ts 32.91 KB 32.91 KB 0.00 KB (0.00%)
logger.ts 11.12 KB 11.12 KB 0.00 KB (0.00%)
metric.ts 9.28 KB 9.28 KB 0.00 KB (0.00%)
optic.ts 6.87 KB 6.87 KB 0.00 KB (0.00%)
pubsub.ts 15.49 KB 15.49 KB 0.00 KB (0.00%)
queue.ts 12.09 KB 12.09 KB 0.00 KB (0.00%)
schedule.ts 11.20 KB 11.20 KB 0.00 KB (0.00%)
schema-binary.ts 39.70 KB 39.70 KB 0.00 KB (0.00%)
schema-class.ts 20.32 KB 20.32 KB 0.00 KB (0.00%)
schema-fromJsonSchemaDocument.ts 30.48 KB 30.48 KB 0.00 KB (0.00%)
schema-representation-roundtrip.ts 26.41 KB 26.41 KB 0.00 KB (0.00%)
schema-string-transformation.ts 13.89 KB 13.89 KB 0.00 KB (0.00%)
schema-string.ts 11.38 KB 11.38 KB 0.00 KB (0.00%)
schema-template-literal.ts 15.76 KB 15.76 KB 0.00 KB (0.00%)
schema-toArbitrary.ts 33.59 KB 33.59 KB 0.00 KB (0.00%)
schema-toCodeDocument.ts 24.76 KB 24.76 KB 0.00 KB (0.00%)
schema-toCodecJson.ts 19.51 KB 19.51 KB 0.00 KB (0.00%)
schema-toEquivalence.ts 19.65 KB 19.65 KB 0.00 KB (0.00%)
schema-toFormatter.ts 19.77 KB 19.77 KB 0.00 KB (0.00%)
schema-toJsonSchemaDocument.ts 23.84 KB 23.84 KB 0.00 KB (0.00%)
schema-toRepresentation.ts 19.79 KB 19.79 KB 0.00 KB (0.00%)
schema.ts 19.53 KB 19.53 KB 0.00 KB (0.00%)
stm.ts 13.03 KB 13.03 KB 0.00 KB (0.00%)
stream.ts 10.06 KB 10.06 KB 0.00 KB (0.00%)

@tim-smart
tim-smart merged commit ba2fd82 into main Sep 4, 2026
13 checks passed
@tim-smart
tim-smart deleted the eff-1226/childprocess-process-group-cleanup-tests branch September 4, 2026 09:11
TFSebben pushed a commit to TFSebben/supabase_cli that referenced this pull request Oct 9, 2026
Bumps `effect`, `@effect/platform-bun`, `@effect/platform-node`,
`@effect/platform-node-shared`, `@effect/sql-pg`, and `@effect/vitest`
from `4.0.0-rc.112` to the stable `4.0.2` release. `@effect/tsgo` is
unchanged.

The goal is zero change to CLI behaviour. Every edit is either required
by a removed or renamed API, or restores a behaviour whose upstream
default moved.

## Commit layout

The first three commits are the bump plus two scripted rewrites and can
be skipped by reviewers:

- `effect/unstable/*` import paths moved to `effect/*` (removed upstream
in rc.118,
[effect#8354](Effect-TS/effect#8354)).
- CLI and Config constructors renamed to PascalCase
([effect#7453](Effect-TS/effect#7453),
[effect#8121](Effect-TS/effect#8121)).

The remaining commits are hand migrations, one topic each.

## Hand migrations

- **node-postgres pool bridge** — `@effect/sql-pg` 4.0 is a native
client and dropped `PgClient.fromPool`
([effect#7426](Effect-TS/effect#7426)). The
remote database session keeps its own node-postgres pool for the zero
idle timeout, the per-connection role step-down hook, and raw `COPY`
connections, so `db-connection.pool-client.ts` is a small
`SqlClient.make` adapter over that pool with the same statement
execution, cancellation, and error classification the old bridge had.
- **Stack RPC strictness** — parse-option annotations no longer affect
decoding ([effect#8131](Effect-TS/effect#8131))
and the RPC transport passes no parse options, so the initialization
command and `runCommand` payload schemas now reject unknown keys through
a `Record` filter ahead of the struct decode, with the same "Expected no
excess property" wording. Encoding drops `undefined` entries first, so
optional fields passed as explicit `undefined` (for example
`pgProve.workingDir`) still cross the RPC JSON codec as they did with
the plain struct.
- **One statement per query** — the native `@effect/sql-pg` client sends
everything through the extended protocol, which rejects multi-command
strings. The realtime schema bootstrap runs as two statements, and the
generated `ALTER ROLE` / `ALTER DATABASE` batches come back from
Postgres as an array and run one at a time inside the same transaction.
- **Socket addresses, file sizes, encoding, sockets** — server addresses
are `NetAddress.SocketAddress` values, `File.seek` and `File.Info.size`
use `bigint` and `ByteSize`, `Encoding` moved to
`effect/encoding/Base64Url`, and the whole-stack WebSocket test helper
uses the reader and writer pair.
- **`Effect.partition`** returns `[passes, fails]`; the `stack destroy`
handler destructures in that order.
- **Management API contracts** — `packages/api` contracts are
regenerated; the 4.0.2 schema codegen emits code-point length checks and
Unicode-flagged patterns.
- **Config package** — `SchemaAST.Union.mode` moved under `options`,
`ToJsonSchemaOptions.additionalProperties` became `onExcessProperty`,
and regex patterns need the Unicode flag to be exported into JSON Schema
([effect#8482](Effect-TS/effect#8482)). Peer
dependency ranges are untouched and already satisfy 4.0.2.

## Preserved defaults

- The stack's `PgClient.layer` calls pin `idleTimeout: "10 seconds"`;
the default since 4.0.1 is 60 seconds
([effect#8679](Effect-TS/effect#8679)).
- The published JSON Schema files (`@supabase/config` `schema.json` and
`project-schema.json`, mirrored at `apps/docs/public/cli/*.schema.json`)
are byte-identical to the develop build. `toCliConfigJsonSchema` passes
`onExcessProperty: "error"` so structs keep `additionalProperties:
false` ([effect#8147](Effect-TS/effect#8147));
the shared document assembler leaves pattern-keyed records open, as the
generator has no per-schema setting for that; and bucket tables are
modelled so they render as the published object-or-array union.

## Behaviour changes reviewed and accepted

These upstream changes have no configuration knob. Each was checked
against the repo's call sites and judged not user-visible:

- Child-process `kill` and scoped release now wait for the process
group, bounded to one second without `forceKillAfter`
([effect#8018](Effect-TS/effect#8018)). The
stack host, bundled Postgres clients, and containers already set
`forceKillAfter`; other spawn sites can only see up to one extra second
of teardown when a descendant lingers.
- `Effect.cached` treats interruption as abandonment
([effect#8719](Effect-TS/effect#8719)). Every
cached effect in the repo is awaited by a single fiber or forked into an
owning scope, so the shared-interruption semantics do not apply.
- Empty bucket tables under `storage.analytics.buckets` and
`storage.vector.buckets` are modelled as an object-or-array union
instead of `Schema.Struct({})`. Runtime decoding now rejects a primitive
bucket value (`foo = "x"`), which the old schema accepted and ignored;
objects and arrays decode as before.
- HTTP client spans are named after the bare method (`GET` instead of
`http.client GET`), following the OpenTelemetry convention. Nothing in
the repo keys on the old name.
- Generated Management API string length checks count code points
instead of UTF-16 code units, which only differs for characters outside
the Basic Multilingual Plane.

## Patches

- Rebased: the `@effect/platform-node-shared` stdin EPIPE listener, now
against 4.0.2. 4.0.2 still removes the stdin sink's error listener
before the final `end()` flush, so a child that exits before reading its
input raises an uncaught EPIPE on Linux;
[effect#7712](Effect-TS/effect#7712) and
[effect#7714](Effect-TS/effect#7714) do not
cover that path.
- Rewritten: the `@effect/vitest` `runTest` patch. Upstream's finalizer
ordering fix
([effect#8154](Effect-TS/effect#8154)) covers
the abort hook, so the patch now only layers the two behaviours the
stack's `effect-timeout` tests assert on top of it: rejecting with the
pretty errors and reporting a finalizer failure after a timeout.

## Test-only changes

- Stack integration and CLI `db reset` e2e fixtures that seeded data
through `PgClient.layer` with semicolon-joined statements now issue one
statement per call; fixtures that go through `psql` are unchanged.
- The `db reset` e2e fixture reads `count(*)::int`, since the native
client decodes `int8` as `bigint`.

## Follow-ups

- Migrate the remote database session to the native `@effect/sql-pg`
client and delete the pool adapter. That changes result decoding (int8,
timestamps, bytea) across the SQL consumers and needs its own review.

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Julien Goux <hi@jgoux.dev>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

4.0 bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

platform-node-shared: scoped release and kill wait on the leader's exit, abandoning descendants when the leader is a non-exec'ing shell (Linux sh -c)

1 participant