Define a graceful drain policy for in-flight turns during shutdown #30

Activity

1. lidge-jun commented on Jun 25, 2026

   

   lidge-jun

   on Jun 25, 2026

   Owner

   More actions

   Thanks for the thorough write-up. I agree this is worth addressing — here's where things stand on dev and my take on a bounded approach.

   Current shutdown behavior

   The shutdown path (src/cli.ts:203-214) is immediate:

   const shutdown = () => {
     server.stop(true);   // Bun's immediate close
     removePid();
     if (!process.env.OCX_SERVICE) restoreNativeCodex();
     process.exit(0);
   };
   process.on("SIGINT", shutdown);
   process.on("SIGTERM", shutdown);

   There's no tracking of in-flight turns at the server level. Individual request paths do handle client-side disconnects well — bridge.ts has cancel() → clientCancelled → upstream abort, and ws-bridge.ts cancels the reader on WebSocket close — but those are reactive (client goes away), not proactive (server initiating shutdown while streams are live).

   What I think makes sense

   Your "possible approach" is close to what I'd want. A few specifics:

   1. Track active turns — a Set<AbortController> (or similar) at the server level, incremented on /v1/responses entry and decremented on stream close/abort. Lightweight, no per-request overhead beyond set add/delete.

   2. Shutdown sequence:

   • SIGINT/SIGTERM → set a "draining" flag → reject new /v1/responses with 503 + Retry-After: 5
   • Wait up to shutdownTimeoutMs (default ~5s, configurable in config.json) for the active set to empty
   • After deadline: abort remaining upstream controllers via the tracked set, log which turns were interrupted
   • Then proceed with current cleanup (pid, restore, exit)

   3. ocx stop --force — skip the drain, same as today's behavior. Service manager SIGKILL after the configured timeout also acts as a hard kill.

   4. Diagnostic signal — when a turn is aborted by shutdown, emit a response.failed SSE event with last_error: { type: "server_shutdown", message: "..." } before closing the stream, so Codex sees a clean terminal event instead of a dropped connection.

   What I'd skip

   • No persistent queue, no turn replay, no checkpoint/resume. This is a proxy, not a message broker.
   • No separate drain behavior for WebSocket vs HTTP/SSE — both use the same timeout.
   • No ocx stop vs Ctrl+C distinction — both should drain. Only --force or SIGKILL should skip.

   Sizing

   This is maybe 80-120 lines of change across server.ts (active set + reject-when-draining) and cli.ts (timed wait loop). The bridge already emits response.failed so we'd reuse that path.

   If you want to prepare a PR for this, go for it — happy to review. The scope is small enough that it shouldn't conflict with other work. Main thing to watch: Bun.serve().stop() behavior with closeActiveConnections: true vs false and how it interacts with a drain loop.
2. lidge-jun

   added a commit that references this issue on Jun 25, 2026

   fix: graceful shutdown drain + passthrough tee relay (Bun#32111)

   ...

   1fb4fdb
3. lidge-jun commented on Jun 25, 2026

   

   lidge-jun

   on Jun 25, 2026

   Owner

   More actions

   Implemented in 1fb4fdb on dev:

   • Active turn tracking: Set<AbortController> tracks all in-flight HTTP streams and WebSocket pumps at stream-lifetime granularity (not handleResponses return)
   • Graceful drain: SIGINT/SIGTERM//api/stop → set draining flag → reject new /v1/responses with 503 + Retry-After: 5 → wait up to shutdownTimeoutMs (default 5s, configurable in config.json) → abort remaining turns → clean exit
   • trackStreamLifetime() wraps Response bodies to register/unregister turns on stream close/cancel/error
   • WebSocket turns register at pump start, unregister in finally
   • New shutdownTimeoutMs config field (optional, default 5000)
   • Tests: 4 new tests in tests/shutdown-drain.test.ts

   Full suite: 360/360 pass, tsc clean.
4. lidge-jun

   closed this as completedon Jun 25, 2026
5. agentHits

   added a commit that references this issue on Sep 17, 2026

   fix: graceful shutdown drain + passthrough tee relay (Bun#32111)

   ...

   0f79104
6. lidge-jun

   added a commit that references this issue on Sep 28, 2026

   fix(devin): resolve models through catalog family metadata (carry #6091…

   ...

   dce4469