Skip to content

Feature request: sandbox update --image (in-place image swap preserving PVC) #758

Closed as not planned

Description

@novarii

Problem

sandbox delete + sandbox create is currently the only way to update a sandbox's image. For stateful workloads this causes:

  1. PVC orphaning — the old persistent volume survives but is not reattached to the new sandbox. The new sandbox gets a fresh PVC.
  2. State loss — any runtime state on the persistent volume (auth sessions, conversation history, scheduled jobs) is effectively lost.
  3. No rollback — once deleted, the old sandbox and its state binding are gone.

This makes OpenShell unsafe for stateful long-running services that need frequent image updates (e.g., config changes, hotfixes) without downtime or state loss.

Use case

We run a WhatsApp-connected daemon inside an OpenShell sandbox. The daemon stores Baileys auth state and conversation sessions on /sandbox/state (persistent volume). During early deployment we need to iterate on config and code multiple times per day. Each sandbox delete + sandbox create cycle:

  • Destroys the WhatsApp session (requires QR code rescan with the client)
  • Loses in-flight conversation state
  • Accumulates orphaned PVCs

Proposed solution

A sandbox update command (or similar) that swaps the container image while preserving the existing PVC binding:

openshell sandbox update <name> --image <new-image> [--policy <policy.yaml>]

Expected behavior:

  • Stop the running container
  • Replace the image reference
  • Restart with the same PVC attached at the same mount point
  • Optionally update the policy file
  • Optionally support --rollback to revert to the previous image

This is analogous to how Kubernetes handles StatefulSet image updates — the pod is replaced but the PVC persists and rebinds.

Alternatives considered

  • Patch files in-place inside the sandbox — works for writable paths but Landlock (correctly) makes /app and /etc read-only, so code and baked config can't be updated without an image swap.
  • Use dev policy with relaxed Landlock — defeats the security model.
  • kubectl patch the underlying workload directly — bypasses OpenShell's state tracking and policy enforcement, fragile.

Environment

  • OpenShell with Landlock filesystem enforcement
  • Persistent volumes mounted at /sandbox/
  • Stateful daemon workloads (WhatsApp, session persistence)

🤖 Generated with Claude Code

Activity

  1. added theissue type on Apr 7, 2026
  2. self-assigned this
    on Apr 7, 2026
  3. rossmorey commented on Apr 8, 2026

    @rossmorey

    I hit this exact problem and found a workaround that's been reliable: patch the Sandbox CR directly instead of delete+create.

    The key insight is that openshell sandbox delete destroys the PVC, but the underlying k8s Sandbox resource supports in-place image updates that preserve it. The workspace-init container checks for a .workspace-initialized sentinel file on the PVC — if it exists, it skips the copy from the image, so your runtime state survives the pod restart.

    Here's the full flow I've been using:

    # 1. Build new image and import into the k3s cluster
    IMAGE_TAG="my-sandbox:$(date +%s)"
    docker build --no-cache -t "$IMAGE_TAG" -f Dockerfile .
    docker save "$IMAGE_TAG" | \
      docker exec -i openshell-cluster-openshell ctr --namespace k8s.io images import -
    
    # 2. Stop your workload (optional, but cleaner)
    ssh -o StrictHostKeyChecking=no openshell-primary 'pkill my-daemon 2>/dev/null; true'
    
    # 3. Swap the image — pod restarts, PVC stays
    SANDBOX="primary"  # your sandbox name
    openshell doctor exec -- kubectl patch sandbox "$SANDBOX" -n openshell --type=json -p "
      [
        {\"op\": \"replace\", \"path\": \"/spec/podTemplate/spec/containers/0/image\", \"value\": \"$IMAGE_TAG\"},
        {\"op\": \"replace\", \"path\": \"/spec/podTemplate/spec/initContainers/0/image\", \"value\": \"$IMAGE_TAG\"}
      ]"
    
    # 4. Wait for pod to come back
    sleep 5
    openshell sandbox list  # should show Ready

    Agree that a native sandbox update --image would be the right solution — this bypasses OpenShell's state tracking, which isn't ideal. But it's been stable as a stopgap.

  4. mjamiv commented on Apr 19, 2026

    @mjamiv
    Contributor

    +1 — operator perspective from a production deployment on the cluster image. Current behavior forces a discipline of host-side tarball backups before every supervisor upgrade or pod operation, since /sandbox/ is wiped on reschedule. PVC preservation across image swap would retire a multi-step backup/restore procedure that adds real time per sandbox and creates incident risk if an operator forgets a step.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions