feat(docker): support workload image updates preserving sandbox identity and data #5484

User Story

I use OpenShell's Docker driver on Apple Silicon macOS with Docker Desktop for a stateful conversational workload. I need to install a corrected workload image, reopen the client, and continue the same conversations and tool access while verifying that the existing sandbox actually runs the corrected runtime. Requiring users to recreate their workspace after each image update is not a sustainable upgrade flow.

Problem Statement

OpenShell has a storage-preserving SandboxUpdates.UpdateSandbox contract, but the published documentation limits workload changes to Kubernetes. Metadata-only updates work across drivers. The inspected v0.1.7 Docker driver does not implement workload image updates; ordinary Stop/Start retains the original workload container and image.

Related work: #758, #4623 and merged PR #4873. This request is the Docker-specific extension of that workflow, not a second update API.

Impact / Why This Matters

Importing a corrected image or updating the client/gateway does not replace the workload image of an existing Docker sandbox. Delete/recreate can change sandbox identity and lose data stored in the container's writable layer. A client-managed backup/recreation/rebinding procedure would make each integration responsible for native lifecycle and crash recovery.

The requirement is to retain intended user state while changing executable runtime bytes. Keeping the old container running successfully is not evidence that the correction was applied.

Proposed Design

Extend the existing stopped-sandbox update workflow to the Docker driver, with an explicit supported persistent-data layout and retention contract. The Docker container may be replaced internally, but the OpenShell sandbox ID and its intended persistent state must remain associated with the same sandbox.

Notional interaction through the existing CLI and generated/raw RPC client:

StopSandbox(scope, name) -> wait for Stopped
GetSandbox(scope, name) -> current resource version
SandboxUpdates.UpdateSandbox(
  scope, name, image = corrected immutable image reference,
  expected_resource_version, request_id
) -> same sandbox ID, still Stopped
StartSandbox(scope, name) -> corrected image, retained user state

This does not propose a new sandbox update CLI command. Reuse the existing authorization, activation, conflict and retry contract. Keep trusted supervisor/sandbox-runtime updates distinct from workload image updates.

Please confirm the intended Docker storage boundary and how this fits the existing update contract before implementation. A first implementation could require sandboxes created with a supported persistent-data layout. Existing writable-layer-only sandboxes must then refuse safely before mutation and document the limitation; transparent migration of those installations need not be part of the initial contribution.

Acceptance Criteria

• An authorized caller can update a stopped Docker sandbox's workload image through the existing update contract. It remains stopped until explicitly started.
• The OpenShell sandbox ID, workspace association, applicable policy and access bindings remain valid across the update. A new underlying container is not adopted merely because its name matches.
• An A-to-B integration test verifies the new image/runtime bytes, retained conversation/session state and user files, and a subsequent successful workload operation after restart.
• The supported persistent-data layout and treatment of writable-layer data are documented. New-image workspace seeding does not overwrite initialized user data; retaining data does not mask the corrected runtime with old executable files.
• Running-sandbox updates, stale versions, incompatible storage layouts, missing images and ownership conflicts fail without unintended data deletion or identity reassociation.
• Interruption during replacement or gateway restart can reconcile the accepted operation without duplicate workloads, concurrent writers or premature startup. Unrecoverable cases preserve intended data and report an actionable failure.
• Stop/Start and explicit removal remain available as documented. Stop retains data; delete follows the documented retention policy and does not remove unrelated resources. A pending update does not require a successful update or network request before documented local cleanup can proceed.
• Docker-specific integration coverage includes update/restart, retry/interruption and storage retention/removal. Docker Desktop on Apple Silicon is the motivating environment; other Docker environments are qualified separately.

Alternatives Considered

• Stop/Start or gateway upgrade only: retains the original workload image; does not apply the corrected image.
• Delete/recreate and client-owned restore: changes native identity and moves data/connection recovery into every client integration.
• Persistent mounts alone: can retain selected files but do not provide OpenShell's image-update lifecycle or recovery contract.
• Direct Docker container replacement: bypasses OpenShell's resource tracking, admission and lifecycle coordination.
• Middleware, interceptors or credential providers: can participate in existing operations but do not themselves supply the missing Docker workload mutation. Prefer an existing supported extension if maintainers identify one that satisfies this workflow.
• Patch runtime files in place: does not provide a reproducible immutable-image update and may leave base-image/interpreter changes unapplied.

Source Investigation

Source review used OpenShell v0.1.7, revision 5d25211b25a8fc887e8d5c606d57d6ab22f1d76c. The gateway already has a configuration-update coordinator, while the Docker driver's optional configuration-update implementation is absent. Docker Stop/Start reuses the workload container. These are source findings, not a claim that a Docker A-to-B update has passed native qualification.

The current sandbox update documentation was also reviewed and explicitly documents Kubernetes-only workload changes.

Checklist

• Reviewed open and closed issues and the published documentation; no Docker-specific issue covering this workflow was found in the searches performed.
• This proposes a user workflow and observable success criteria; implementation and scope require maintainer alignment.