Skip to content

Backend pod crashloop on k3d/Tilt: exec /app/services/<<service-name>>/.venv/bin/poetry: no such file or directory #142

@a-klos

Description

@a-klos

Summary

When running the backend in a k3d cluster with Tilt (Helm deploy), the container repeatedly restarts with:

exec /app/services/rag-backend/.venv/bin/poetry: no such file or directory

This does not reproduce on latest macOS; it happens on Ubuntu 22.04 LTS with Docker 28.3.2.


Environment

  • OS: Ubuntu 22.04 LTS
  • Docker: 28.3.2
  • Orchestrator: k3d
  • Deployer: Tilt + Helm
  • Namespace: rag

Logs / Symptoms

Tilt events show repeated restarts of admin-backend, backend and document-extractor. Find below a sample log of the backend service:

Detected container restart. Pod: backend-78c4b4c758-frdpr. Container: backend.
[backend] exec /app/services/rag-backend/.venv/bin/poetry: no such file or directory
[event: pod rag/backend-78c4b4c758-frdpr] Back-off restarting failed container backend in pod backend-78c4b4c758-frdpr_rag(813c9098-6d6c-4ce7-8bf4-bb13faaae6d5)

Expected behavior

Backend container starts successfully and the application process runs without crashlooping.

Actual behavior

Container exits immediately because the entrypoint/command tries to exec Poetry from a path that does not exist at runtime.


Likely Root Cause

Tilt/Helm live code sync/volume mount is overlaying /app/services/rag-backend inside the container and hiding the .venv directory created at build time. The image likely contains .venv/bin/poetry, but the runtime mount replaces the whole project directory, so the Poetry binary (and/or its shebang interpreter) is missing.

This can be why it “works on macOS”: the dev flow there may not mount the same path, or Docker Desktop differences mean the venv isn’t overlaid.

Labels: bug, dev-experience, k8s, tilt, helm, python, poetry

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No fields configured for Bug.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions