You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This is a key thing for us to finish replacing podman-bootc. I think a key thing here is we should act as close as possible to the Linux implementation. Ideally bcvk ephemeral works.
We can easily scrape out the kernel+initramfs from the target container image back to the host and launch krun (or via macadam) from there. The trickier bit here will be replicating the "mount target container image's root" when the files live inside the rootfs of the podman-machine VM. I am not sure if we can easily do the virtiofs thing. Maybe we could try using e.g. NFS (eww?).
Reacted by Gerard Braad | RGRV, Jeff Ligon and Anthony Byrne
With newer macs (M3+) nested virtualization is supported in linux guests, I don’t know if running a nested VM inside the podman machine VM could achieve what you are after?
With newer macs (M3+) nested virtualization is supported in linux guests, I don’t know if running a nested VM inside the podman machine VM could achieve what you are after?
It's a bit tricky; I think sometimes folks would want that, other times not. The biggest thing here is it scopes resource usage of the L2 VMs to the max resources of the L1 (podman machine) VM.
For macadam SELinux is enabled, while for bcvk this is disabled.
We only emit selinux=0 for bcvk ephemeral right now, we definitely don't do that (i.e. selinux is enforcing) for the bcvk to-disk and bcvk libvirt flows.
The trickier bit here will be replicating the "mount target container image's root" when the files live inside the rootfs of the podman-machine VM. I am not sure if we can easily do the virtiofs thing. Maybe we could try using e.g. NFS (eww?).
To me a big goal of bcvk ephemeral is that it's O(1) to launch (i.e. it doesn't get slower as your image gets bigger). This is how podman run works because of overlayfs and general dynamic access.
It would not be O(1) to run e.g. mke2fs on a loop blockdev and serve that via nbd (and of course more generally doing so requires duplicating storage!). Now...could we somehow have a nbd server that was dynamic? In theory, maybe? I guess in particular with erofs it seems theoretically possible...
But really serving via virtiofsd or some other filesystem-based mechanism just meshes much more directly/natively with container images.
It would not be O(1) to run e.g. mke2fs on a loop blockdev and serve that via nbd
hadn’t looked at the linuxdisk plugin source code, I assumed/hoped this was dynamically generated, but now I see the mke2fs call in there :(
But really serving via virtiofsd or some other filesystem-based mechanism just meshes much more directly/natively with container images.
We don’t have that level of control over apple’s virtiofs implementation (ie vfkit), but maybe something could be done with the krunkit stack? I’m not familiar enough with it.
We don’t have that level of control over apple’s virtiofs implementation (ie vfkit), but maybe something could be done with the krunkit stack? I’m not familiar enough with it.
would be bcvk be an executable on the mac like bcvk run, or would it be podman machine ssh -> bcvk run?
if its the latter, then we need to rethink this concept. The solution to supporting another OS shouldn't be "lets just get them to a Linux shell so they can do linux-y things"
Activity
cfergeau commentedon Sep 24, 2025
9p might also be an option for the file sharing.
With newer macs (M3+) nested virtualization is supported in linux guests, I don’t know if running a nested VM inside the podman machine VM could achieve what you are after?
cgwalters commentedon Sep 24, 2025
Also explicitly cross-referencing crc-org/macadam#140
It's a bit tricky; I think sometimes folks would want that, other times not. The biggest thing here is it scopes resource usage of the L2 VMs to the max resources of the L1 (podman machine) VM.
gbraad commentedon Sep 24, 2025
For macadam SELinux is enabled, while for
bcvkthis is disabled. This is one of those things that people want to test withcgwalters commentedon Sep 24, 2025
We only emit
selinux=0forbcvk ephemeralright now, we definitely don't do that (i.e. selinux is enforcing) for thebcvk to-diskandbcvk libvirtflows.cgwalters commentedon Sep 24, 2025
Split that issue out to #22
cfergeau commentedon Sep 25, 2025
Does this need to be writable? or read-only is fine? https://manpages.ubuntu.com/manpages/questing/en/man1/nbdkit-linuxdisk-plugin.1.html might work for that.
vfkitcan use a disk image served over nbd.cgwalters commentedon Sep 25, 2025
bcvk ephemeralalways injectssystemd.volatile=overlaytoday, so we don't depend on writes to the backing store.To me a big goal of
bcvk ephemeralis that it's O(1) to launch (i.e. it doesn't get slower as your image gets bigger). This is howpodman runworks because of overlayfs and general dynamic access.It would not be O(1) to run e.g.
mke2fson a loop blockdev and serve that via nbd (and of course more generally doing so requires duplicating storage!). Now...could we somehow have a nbd server that was dynamic? In theory, maybe? I guess in particular witherofsit seems theoretically possible...But really serving via virtiofsd or some other filesystem-based mechanism just meshes much more directly/natively with container images.
cfergeau commentedon Sep 26, 2025
hadn’t looked at the linuxdisk plugin source code, I assumed/hoped this was dynamically generated, but now I see the
mke2fscall in there :(We don’t have that level of control over apple’s virtiofs implementation (ie vfkit), but maybe something could be done with the krunkit stack? I’m not familiar enough with it.
cgwalters commentedon Sep 26, 2025
Did coreos/cap-std-ext#80 to help with this
cgwalters commentedon Sep 26, 2025
cc @germag
jtligon commentedon Oct 24, 2025
would be bcvk be an executable on the mac like
bcvk run, or would it bepodman machine ssh->bcvk run?if its the latter, then we need to rethink this concept. The solution to supporting another OS shouldn't be "lets just get them to a Linux shell so they can do linux-y things"
cgwalters commentedon Oct 24, 2025
No the plan is the former; doing anything bcvk-like from inside a VM would add a whole massive level of complexity.
cgwalters commentedon Feb 25, 2026
OK on this topic I did #211 - in a nutshell, supporting apple/container would probably be quite a bit easier than supporting podman machine.
jtligon commentedon Feb 25, 2026
Need a review of some swift code? Let me know!
build: Add macOS stub compilation support
build: Add macOS stub compilation support
WIP: Add macOS support via krunkit/libkrun (bcvk-krun crate)