fix(daemon): keep a displaced daemon from deleting its replacement's socket

[georgeh0]

Completes the fix for #288 (follow-up to #291). Rebased on main (#299, #302, #303).

Why #291 had no effect

multiprocessing.connection.SocketListener registers util.Finalize(self, os.unlink, args=(address,), exitpriority=0) at bind time for AF_UNIX, and close() runs it unconditionally. Shutdown step 1 calls listener.close(), so a displaced daemon deleted the replacement's socket before the inode check in step 4 ran. Step 4 never found a file to guard. The test added in #291 compared touch()ed files and never ran daemon code, so it passed either way. #299 later removed that test too.

Fix

The daemon owns its listening socket on POSIX (_UnixListener in daemon.py). Windows keeps multiprocessing.connection.Listener (named pipes) and is unchanged. _UnixListener binds the path and records the socket file's identity. On close(), it unlinks the path only if it still has that identity, and it does so before closing the socket, because an open socket keeps its inode in use. The class sits under if sys.platform != "win32":, which keeps mypy happy on Windows, where typeshed has no socket.AF_UNIX.

I chose this over disarming the stdlib finalizer (listener._listener._unlink.cancel()) for three reasons:

• Private attributes. Disarming relies on two private attributes, neither in typeshed, and both can change between Python versions. A defensive getattr would silently bring the bug back if they were renamed.
• A second bug on Linux. On Linux, close() does not wake a thread blocked in accept(); only shutdown() does. macOS wakes it on close(). I checked both: macOS locally, Linux in python:3.13-slim. On main, a displaced daemon's wake-up connection goes to the replacement, so its own accept thread stays blocked. Shutdown then waits out the 5 s accept_thread.join(timeout=5) and the thread leaks. _UnixListener.close() calls shutdown() first. Doing that through the stdlib object would need a third private attribute (_socket).
• Typed, stable API only. It uses socket plus Connection(fd), the same wrapping SocketListener.accept() does. It matches the stdlib's blocking mode and backlog=1, so nothing else changes.

Identity is (st_dev, st_ino, st_ctime_ns), not just (dev, ino). On ext4/overlayfs, a freed inode number goes straight to the next file. In a python:3.13-slim container, a socket file recreated at the same path after the old socket was closed got the same inode number 20 out of 20 times. While the old socket was open, it never did. So once the original socket is gone, (dev, ino) alone can match a replacement's file. The ctime only collides if both files are created within one timestamp tick.

The guarded unlink now happens in listener.close() (shutdown step 1). That is when the stdlib finalizer actually removed the file before. Step 4 now only removes the PID file.

Single implementation. socket_identity() and unlink_socket_if_owned() in _daemon_paths.py are used at bind time, at daemon shutdown, and by the client.

Client _cleanup_stale_files also unlinked the socket path unconditionally (same class of bug). stop_daemon() now records the socket's identity before stopping and only removes that file. A stale socket left by a dead daemon is still removed.

Not fixed: two microsecond race windows (documented in comments)

• Bind-to-stat gap. A replacement binding between bind() and the stat would be recorded as ours. bind() can't report the file's identity.
• Stat-to-unlink gap. POSIX has no "unlink if inode matches", so a replacement binding between the stat and the unlink is still lost.

I considered two ways to close them:

1. Bind to a temp name, then os.replace it into place. This closes only the startup gap. The temp path must fit the sun_path limit (104 or 108 bytes) next to a path that was already length-checked, and a crash between bind and rename leaves stray socket files.
2. An flock-ed lock file shared by every daemon's and the client's bind/unlink. This closes both gaps, but adds a persistent lock-file artifact and a new coordination protocol.

Both windows need two daemons starting or stopping at the same instant, and the result heals itself: the client starts a new daemon when the socket is missing. Option 2 is the one to take if we want them closed.

Note: the shutdown wake-up connection

Step 1 still sends the Client(sock_path) wake-up on every platform, so the Windows path gets exercised everywhere. If a replacement owns the path, the wake-up reaches the replacement instead: one extra connection that counts as activity there. This is harmless and noted in the comment. On POSIX the wake-up is no longer needed for correctness, because close() wakes accept() itself.

Tests

• test_displaced_daemon_keeps_replacement_socket (tests/test_daemon_idle.py).
  • It starts a real in-process daemon and replaces its socket path with a new bound AF_UNIX socket. After the daemon idle-exits, it asserts the path still exists with the replacement's identity and that the PID file was removed.
  • It fails on main on both macOS and Linux. On Linux, main also takes about 6 s because of the accept-thread join.
• test_stop_daemon_removes_only_the_stopped_daemons_socket[False|True] (tests/test_client.py).
  • The stopped daemon's socket is closed before a replacement binds, so the inode number can be reused.
  • True fails without the client change. On Linux it also fails with a (dev, ino)-only identity.
  • False checks that a stale socket is still removed.
• The existing _assert_cleaned_up checks still pass.

Verification

• uv run pytest tests/: 350 passed after the rebase onto main.
• Linux (python:3.13-slim): the daemon idle-exit tests and client stop tests pass, and the new tests fail against the old code.
• uv run mypy --platform {win32,darwin,linux} src/ tests/: clean.
• uv run ruff check . and uv run ruff format --check .: clean.

🤖 Generated with Claude Code