Supports multiple IP address assignment for nerdctl run and nerdctl compose up

[kbrierly]

Per (#3556 (comment)) I am opening an issue on what seems to be CNI concurrency issue.

I am having CNI issues when using multiple networks. One is macvlan network and the other is a bridge network. I've also attempted dual macvlan with similar results. What happens looks like an ordering issue. Sometimes the bridge interface would try and come up as a macvlan for example or as below shows the macvlan tries to come up as a bridge . I made a basic test compose file with hello-world and it happens there are well. Nothing is currently assigned the ip. I've tried other unused ip's as well.

I found this when trying to use 2.0.0-rc3 and I retried when 2.0.0 was released and the problem persists. The same compose file works without issue in docker compose.

This is the result from a "nerdctl compose up". The home.local is macvlan, proxy.home.local is a bridge. Single network services start with no issues.

# nerdctl compose up  
INFO[0000] Ensuring image hello-world                   
INFO[0000] Creating container test                      
FATA[0003] failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error running hook #0: error running hook: exit status 1, stdout: , stderr: time="2024-11-09T11:06:52-06:00" level=fatal msg="failed to call cni.Setup: plugin type=\"bridge\" failed (add): failed to allocate all requested IPs: 10.0.0.171": unknown 
FATA[0003] error while creating container test: exit status 1

The hello-world compose file:

services:
  hello:
    image: hello-world
    container_name: test
    hostname: test
    networks:
      home.local:
        ipv4_address: 10.0.0.171
        mac_address: 02:42:0a:00:01:47
      proxy.home.local:
          ipv4_address: 10.100.100.127
     
networks:
    home.local:
        name: home.local
        external: true
    proxy.home.local:
        name: proxy.home.local
        external: true

Network creation commands and cni versions:

# nerdctl network create -d macvlan --subnet=10.0.0.0/24 --gateway=10.0.0.1 -o parent=bond0 -o macvlan_mode=bridge home.local
# nerdctl network create --subnet=10.100.100.0/24 proxy.home.local

# /opt/cni/bin/macvlan -v
CNI macvlan plugin v1.6.0
CNI protocol versions supported: 0.1.0, 0.2.0, 0.3.0, 0.3.1, 0.4.0, 1.0.0, 1.1.0
# /opt/cni/bin/bridge -v
CNI bridge plugin v1.6.0
CNI protocol versions supported: 0.1.0, 0.2.0, 0.3.0, 0.3.1, 0.4.0, 1.0.0, 1.1.0

[Zheaoli]

Bug confirmed, Let us figure it out

[Zheaoli]

OK, here's the detail

I'm not sure if this a bug or not. For now, we do not support multiple IP address assignment yet.

https://github.com/containerd/nerdctl/blob/main/pkg/cmd/container/create.go#L634-L662

Your compose file will be converted to the command nerdctl run --pull=never --net=home.local --ip=10.0.0.171 --mac-address=02:42:0a:00:01:47 --net=proxy.home.local --ip=10.100.100.127 --hostname=test --restart=no hello-world

The last IP address will override the first IP address. So the issue comes here.

I'm not sure we should support multiple IP address assignment. WDYT @AkihiroSuda

[AkihiroSuda]

I'm not sure we should support multiple IP address assignment.

It should correspond to whether Docker supports multiple IP address assignment

[Zheaoli]

I'm not sure we should support multiple IP address assignment.

It should correspond to whether Docker supports multiple IP address assignment

Same idea, Let us figure it out