feat: single-instance daemon for shared hyperd across MCP clients

[StefanSteiner]

Summary

This PR turns hyperdb-mcp into a robust shared-engine architecture. Three commits land together:

1. Single-instance daemon — one hyperd per user, shared across all MCP clients (Claude Code, Cursor, VS Code Copilot, etc.). Reduces memory overhead and lets multiple AI clients access the same persistent databases simultaneously.
2. Documentation — README "Operating Modes" section, DEVELOPMENT.md daemon internals, ROADMAP.md cleanup.
3. Crash recovery — daemon detects dead hyperd and restarts it automatically, with rate limiting to fail loudly when something is fundamentally broken.

Architecture

┌──────────────┐  ┌──────────────┐  ┌──────────────┐
│ MCP Client 1 │  │ MCP Client 2 │  │ MCP Client 3 │
└──────┬───────┘  └──────┬───────┘  └──────┬───────┘
       └─────────────────┼─────────────────┘
                         │ libpq (TCP 127.0.0.1)
              ┌──────────┴──────────┐
              │   hyperdb daemon    │
              │  (single instance)  │
              └──────────┬──────────┘
              ┌──────────┴──────────┐
              │       hyperd        │
              └─────────────────────┘

Key design decisions:

• TCP port binding as cross-platform single-instance lock (works identically on Windows/macOS/Linux)
• Discovery file at ~/.hyperdb/daemon.json (overridable via HYPERDB_STATE_DIR)
• Health protocol (PING/HEARTBEAT/STOP/STATUS/REPORT_HYPERD_ERROR) for liveness checks, idle tracking, and crash signaling
• Idle timeout (30 min default) with debounced heartbeats from active clients
• --no-daemon flag to opt out and use legacy per-client hyperd

Operating modes

Two independent dimensions:

Engine	Persistence	Behavior
Shared daemon (default)	Ephemeral (default)	All clients share one hyperd; each gets a temp .hyper deleted on session end
Shared daemon	--workspace <path>	All clients can collaborate on the same persistent .hyper file
--no-daemon	Ephemeral	Each client spawns its own hyperd (legacy)
--no-daemon	--workspace <path>	One client owns its own hyperd + persistent file

Crash recovery

The daemon polls hyperd every 5 seconds via Child::try_wait() (zero SQL traffic — no log noise). If hyperd has exited:

1. Drop the old HyperProcess.
2. Spawn a replacement with the same parameters.
3. Atomically rewrite daemon.json with the new endpoint.
4. Clients reconnect transparently via the existing ConnectionLost recovery path.

Clients can also signal REPORT_HYPERD_ERROR to fast-path the restart when they detect a dead endpoint before the daemon's polling tick.

Restart attempts are rate-limited to 3 per 60 seconds. The 4th attempt within the window triggers daemon shutdown so the user sees the failure clearly rather than spinning silently. The next MCP client to start will spawn a fresh daemon.

Known limitation: hung-but-alive hyperd (TCP listening but query-stuck) is not detected. Operator recovery is hyperdb-mcp daemon stop.

New CLI commands

hyperdb-mcp daemon              # Run as daemon (usually auto-spawned)
hyperdb-mcp daemon stop         # Gracefully shut down running daemon
hyperdb-mcp daemon status       # Show running daemon info
hyperdb-mcp --no-daemon         # Legacy mode: private hyperd per session

New environment variables:

• HYPERDB_STATE_DIR — override ~/.hyperdb/
• HYPERDB_DAEMON_PORT — override default port 7484
• HYPERDB_DAEMON_IDLE_TIMEOUT — override 30-minute default

Test plan

37 new tests, all passing:

• Daemon basics (26): DaemonState idle tracking + shutdown flag; discovery file write/read/overwrite/remove/stale-cleanup; health protocol (PING, HEARTBEAT, STOP, STATUS, unknown command, multi-command); single-instance lock (second bind fails with AddrInUse); idle timeout triggers shutdown / heartbeats prevent it; engine connects to shared daemon; two engines share one hyperd; persistent database file survives engine drop; ephemeral database cleaned up on drop (DETACH + delete).
• Crash recovery (11): HyperProcess::has_exited correctness on Unix and after SIGKILL; DaemonState restart-flag round-trip; restart-history rate-limit math (records under limit, rejects 4th in window, prunes after window expires); REPORT_HYPERD_ERROR health command sets the flag; full integration — kill -9 hyperd → daemon restarts it → endpoint is reachable; client report triggers restart; end-to-end engine recovery — a fresh Engine::new after a hyperd kill successfully reconnects and reads data persisted before the kill.
• All existing workspace tests pass unchanged
• cargo clippy --workspace --tests and cargo fmt --check clean
• Manual smoke: 7 MCP clients sharing 1 daemon-managed hyperd verified via ps aux

Adversarial review

Per request, both plan and code went through adversarial review. Findings caught and fixed before commit:

• Plan phase: Windows is_running() was a no-op (always returned true); plan v1 would have shipped a Windows-broken feature. Fixed by switching to Child::try_wait() which is correct on both platforms. Also caught: alive-but-hung detection was overstated as covered (it isn't — documented as a known limitation).
• Code phase: Spurious-double-restart race — if a client REPORT_HYPERD_ERROR arrived during a restart, the next monitor tick would consume the stale flag and restart the just-spawned healthy hyperd. Fixed by draining the flag after a successful restart.
• Code phase: TestDaemon::Drop waited only 200ms — could leave hyperd running between tests under load. Fixed by polling the daemon's health port until unreachable, with a 10s timeout.

Commits

• 178e40e — feat: single-instance daemon for shared hyperd across MCP clients
• fbccfa8 — docs: describe shared-daemon operating modes and CLI surface
• 760575c — feat(daemon): detect and restart crashed hyperd