Skip to content

IPv6 traffic blocked by eBPF filter in docker-container buildx builders #828

Open

Description

@eest

Bug report

Bug description

When running software tests in a docker-container buildx builder I noticed a
program was unable to connect to ::1 even if a listening socket was created.

It looks like a eBPF egress filter is attached to the /docker/buildx cgroup with
empty IPv6 allowlists so all IPv6 traffic including loopback is dropped. I would expect it to have similar entries to the IPv4-related maps.

It is possible this is the same problem that was experienced in moby/buildkit#5813.

Here is the cgroup_skb eBPF filter attached at cgroup_inet_egress with the
name egress:

bpftool prog show id 2
2: cgroup_skb  name egress  tag 3629986f5d8e9bc6  gpl
	loaded_at 2026-11-06T23:11:10+0000  uid 0
	xlated 744B  jited 536B  memlock 4096B  map_ids 15,13,11,16,14

The related maps in question:

bpftool map show id 11
11: queue  name blocked_packets  flags 0x0
	key 0B  value 8B  max_entries 1000  memlock 8312B
bpftool map show id 13
13: hash  name allowed_map  flags 0x0
	key 4B  value 4B  max_entries 10000  memlock 903696B
bpftool map show id 14
14: hash  name allowed_map6  flags 0x0
	key 16B  value 4B  max_entries 10000  memlock 983776B
bpftool map show id 15
15: lpm_trie  name allowed_trie  flags 0x1
	key 8B  value 8B  max_entries 1024  memlock 216B
bpftool map show id 16
16: lpm_trie  name allowed_trie6  flags 0x1
	key 20B  value 8B  max_entries 1024  memlock 0B

Contents of allowed_trie (IPv4 networks):

bpftool map dump id 15
key: 08 00 00 00 0a 00 00 00  value: 00 00 00 00 00 00 00 00
key: 08 00 00 00 7f 00 00 00  value: 00 00 00 00 00 00 00 00
key: 0c 00 00 00 ac 10 00 00  value: 00 00 00 00 00 00 00 00
key: 18 00 00 00 c0 a8 41 00  value: 00 00 00 00 00 00 00 00
key: 10 00 00 00 c0 a8 00 00  value: 00 00 00 00 00 00 00 00
key: 00 00 00 00 00 00 00 00  value: 00 00 00 00 00 00 00 00
Found 6 elements

Translation of IPv4 networks:

08 00 00 00 0a 00 00 00: 10.0.0.0/8
08 00 00 00 7f 00 00 00: 127.0.0.0/8
0c 00 00 00 ac 10 00 00: 172.16.0.0/12
18 00 00 00 c0 a8 41 00: 192.168.65.0/24
10 00 00 00 c0 a8 00 00: 192.168.0.0/16
00 00 00 00 00 00 00 00: 0.0.0.0/0

Contents of allowed_trie6 (IPv6 networks):

bpftool map dump id 16
Found 0 elements

Contents of allowed_map6 (IPv6 addresses):

bpftool map dump id 14
Found 0 elements

Reproduction steps

Here is a Dockerfile to test connectivity:

FROM python:3.15-slim
RUN python3 - <<'EOF'
import socket

server = socket.socket(socket.AF_INET6)
server.bind(("::1", 12345))
server.listen()

client = socket.socket(socket.AF_INET6)
client.settimeout(3)
client.connect(("::1", 12345))

print("ipv6 loopback ok")
EOF

Building it using the default builder (using the docker driver) works:

docker buildx build --builder desktop-linux --no-cache --progress=plain .
[...]
#5 [2/2] RUN python3 - <<'EOF'
#5 0.123 ipv6 loopback ok
#5 DONE 0.1s
[...]

Creating a builder that uses the docker-container driver and trying to use
that instead fails:

docker buildx create --name repro --driver docker-container
docker buildx build --builder repro --no-cache --progress=plain .
[...]
#5 [2/2] RUN python3 - <<'EOF'
#5 3.155 Traceback (most recent call last):
#5 3.155   File "<stdin>", line 9, in <module>
#5 3.155 TimeoutError: timed out
#5 ERROR: process "/bin/sh -c python3 - <<'EOF'\nimport socket\n\nserver = socket.socket(socket.AF_INET6)\nserver.bind((\"::1\", 12345))\nserver.listen()\n\nclient = socket.socket(socket.AF_INET6)\nclient.settimeout(3)\nclient.connect((\"::1\", 12345))\n\nprint(\"ipv6 loopback ok\")\nEOF" did not complete successfully: exit code: 1
------
 > [2/2] RUN python3 - <<'EOF':
3.155 Traceback (most recent call last):
3.155   File "<stdin>", line 9, in <module>
3.155 TimeoutError: timed out
------
[...]

We can verify the eBPF filter is the issue by temporarily inserting the
equivalent of "::1" into allowed_map6 if we have access to a privileged
builder and a more involved Dockerfile:

FROM python:3.15-slim
RUN apt-get update && apt-get install -y --no-install-recommends bpftool
COPY <<'EOF' /check.py
import socket

server = socket.socket(socket.AF_INET6)
server.bind(("::1", 12345))
server.listen()

client = socket.socket(socket.AF_INET6)
client.settimeout(3)
client.connect(("::1", 12345))

print("ipv6 loopback ok")
EOF
RUN --security=insecure <<'EOF'
set -e
key="00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01"
echo "--- before:"; python3 /check.py || true
bpftool map update name allowed_map6 key hex $key value hex 00 00 00 00
trap 'bpftool map delete name allowed_map6 key hex $key' EXIT
echo "--- after adding ::1 to allowed_map6:"; python3 /check.py
EOF
docker buildx create --name repro-privileged --driver docker-container --buildkitd-flags '--allow-insecure-entitlement security.insecure'
docker buildx build --builder repro-privileged --allow security.insecure --no-cache --progress=plain .
[...]
#9 [4/4] RUN --security=insecure <<'EOF' (set -e...)
#9 0.025 --- before:
#9 3.174 Traceback (most recent call last):
#9 3.174   File "/check.py", line 9, in <module>
#9 3.174     client.connect(("::1", 12345))
#9 3.174     ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^
#9 3.174 TimeoutError: timed out
#9 3.181 --- after adding ::1 to allowed_map6:
#9 3.192 ipv6 loopback ok
#9 DONE 3.2s
[...]

Inspecting the state of the Docker Desktop VM directly we can see the egress
filter is attached at two locations where one is /docker/buildx which matches the
cgroup-parent setting. This is probably why the builders inherit the filter:

# docker run --rm -it --privileged --pid=host --cgroupns=host debian:trixie-slim bash
# apt-get update && apt-get install -y --no-install-recommends bpftool
# bpftool cgroup tree
CgroupPath
ID       AttachType      AttachFlags     Name
[...]
/sys/fs/cgroup/docker/buildx
    2        cgroup_inet_egress multi           egress
[...]
/sys/fs/cgroup/restricted
    2        cgroup_inet_egress multi           egress

Additionally while digging into this with the help of an LLM I supplied it with
the contents of the loaded eBPF program:

bpftool prog dump xlated id 2
    0: (bf) r6 = r1
    1: (bf) r3 = r10
    2: (07) r3 += -20
    3: (b4) w2 = 0
    4: (b4) w4 = 20
    5: (85) call bpf_skb_load_bytes#15022224
    6: (bf) r1 = r0
    7: (18) r2 = 0x80000000
    9: (5f) r1 &= r2
   10: (b4) w0 = 0
   11: (55) if r1 != 0x0 goto pc+80
   12: (71) r1 = *(u8 *)(r10 -20)
   13: (74) w1 >>= 4
   14: (16) if w1 == 0x6 goto pc+34
   15: (56) if w1 != 0x4 goto pc+76
   16: (b4) w1 = 32
   17: (63) *(u32 *)(r10 -60) = r1
   18: (61) r1 = *(u32 *)(r10 -4)
   19: (63) *(u32 *)(r10 -56) = r1
   20: (bf) r2 = r10
   21: (07) r2 += -60
   22: (18) r1 = map[id:15]
   24: (85) call trie_lookup_elem#377176
   25: (bf) r1 = r0
   26: (b4) w0 = 1
   27: (55) if r1 != 0x0 goto pc+64
   28: (bf) r2 = r10
   29: (07) r2 += -4
   30: (18) r1 = map[id:13]
   32: (85) call __htab_map_lookup_elem#329680
   33: (15) if r0 == 0x0 goto pc+1
   34: (07) r0 += 56
   35: (bf) r1 = r0
   36: (b4) w0 = 1
   37: (55) if r1 != 0x0 goto pc+54
   38: (61) r1 = *(u32 *)(r10 -4)
   39: (63) *(u32 *)(r10 -80) = r1
   40: (61) r1 = *(u32 *)(r10 -8)
   41: (63) *(u32 *)(r10 -76) = r1
   42: (bf) r2 = r10
   43: (07) r2 += -80
   44: (18) r1 = map[id:11]
   46: (b7) r3 = 2
   47: (85) call queue_stack_map_push_elem#388888
   48: (05) goto pc+42
   49: (bf) r3 = r10
   50: (07) r3 += -60
   51: (bf) r1 = r6
   52: (b4) w2 = 0
   53: (b4) w4 = 40
   54: (85) call bpf_skb_load_bytes#15022224
   55: (bf) r1 = r0
   56: (b4) w0 = 0
   57: (18) r2 = 0x80000000
   59: (5f) r1 &= r2
   60: (55) if r1 != 0x0 goto pc+31
   61: (b4) w0 = 1
   62: (61) r1 = *(u32 *)(r10 -36)
   63: (16) if w1 == 0x2ff goto pc+28
   64: (73) *(u8 *)(r10 -76) = r1
   65: (b4) w1 = 128
   66: (63) *(u32 *)(r10 -80) = r1
   67: (61) r1 = *(u32 *)(r10 -32)
   68: (73) *(u8 *)(r10 -75) = r1
   69: (61) r1 = *(u32 *)(r10 -28)
   70: (73) *(u8 *)(r10 -74) = r1
   71: (61) r1 = *(u32 *)(r10 -24)
   72: (73) *(u8 *)(r10 -73) = r1
   73: (bf) r2 = r10
   74: (07) r2 += -80
   75: (18) r1 = map[id:16]
   77: (85) call trie_lookup_elem#377176
   78: (bf) r1 = r0
   79: (b4) w0 = 1
   80: (55) if r1 != 0x0 goto pc+11
   81: (bf) r2 = r10
   82: (07) r2 += -36
   83: (18) r1 = map[id:14]
   85: (85) call __htab_map_lookup_elem#329680
   86: (15) if r0 == 0x0 goto pc+1
   87: (07) r0 += 64
   88: (bf) r1 = r0
   89: (b4) w0 = 1
   90: (55) if r1 != 0x0 goto pc+1
   91: (b4) w0 = 0
   92: (95) exit

... and it had this to say:

The 20-byte lookup key for allowed_trie6 is built at offset -80
(instructions 62–72). Each of the four 32-bit words of the destination address
is loaded, but only one byte of each is stored (*(u8 *) at -76 to -73), so
the key contains address bytes 0, 4, 8 and 12, and its other 12 address bytes
are never written. As a result, only ::/0 and prefixes of /8 or shorter can
match reliably; longer entries such as ::1/128 cannot, as the result depends
on leftover kernel stack contents. allowed_map6 is unaffected, since it is
looked up with the full address (instructions 81–85).

I have not verified this bug claim but thought it might be a good idea to include just in case.

Platform

macOS

Version information

Client:
 Version:           29.9.2
 API version:       1.56
 Go version:        go1.26.10
 Git commit:        a82e528
 Built:             Tue Oct 20 18:43:59 2026
 OS/Arch:           darwin/arm64
 Context:           desktop-linux

Server: Docker Desktop 4.98.0 (245422)
 Engine:
  Version:          29.9.2
  API version:      1.56 (minimum version 1.40)
  Go version:       go1.26.10
  Git commit:       0367cbe
  Built:            Tue Oct 20 18:44:14 2026
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          v2.4.2
  GitCommit:        b7a25261e029d68978bbf290df0a62fc819b3c51
 runc:
  Version:          1.5.2
  GitCommit:        v1.5.2-0-g29dd3dc2
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

Diagnostics ID

No response

Activity

  1. added theissue type on Nov 7, 2026
  2. eest commented on Nov 7, 2026

    @eest
    Author

    Just to follow up on this, given that the filter is inherited from the default cgroup-parent, creating a builder with a custom cgroup-parent outside /docker/buildx seems to avoid the filter (of course also avoiding what the filter is supposed to protect against).

    Steps to try it, using the basic Dockerfile from the initial description:

    docker buildx create --name repro-custom-cgroup --driver docker-container --driver-opt cgroup-parent=/docker/repro-custom-cgroup
    docker buildx build --builder repro-custom-cgroup --no-cache --progress=plain .
    [...]
    #6 [2/2] RUN python3 - <<'EOF'
    #6 0.074 ipv6 loopback ok
    #6 DONE 0.2s
    
  3. bsousaa commented on Nov 9, 2026

    @bsousaa
    Collaborator

    please include a diagnostics id in your issue report

  4. eest commented on Nov 9, 2026

    @eest
    Author

    @bsousaa not trying to be difficult but from what I can tell I cant get a diagnostics ID without having the app upload some sort of diagnostics package which I am not comfortable with. The report above should have everything in it to reproduce the issue locally.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions