setup environment with kind

[BurlyLuo]

setup with kind:

INFO[2024-08-26 07:47:24] Setting log directory: /var/log/afxdp-k8s-plugins/ 
INFO[2024-08-26 07:47:24] Setting log file: afxdp-dp.log               
INFO[2024-08-26 07:47:24] Setting log level: debug                     
INFO[2024-08-26 07:47:24] Switching to debug log format                
INFO[2024-08-26 07:47:24] [main.go:75] [main] Starting AF_XDP Device Plugin                
INFO[2024-08-26 07:47:24] [main.go:78] [main] Checking if host meets requirements          
DEBU[2024-08-26 07:47:24] [main.go:171] [checkHost] Checking kernel version                      
DEBU[2024-08-26 07:47:24] [main.go:197] [checkHost] Kernel version: 6.8.0-40-generic meets minimum requirements 
DEBU[2024-08-26 07:47:24] [main.go:200] [checkHost] Checking host for Libbpf                     
DEBU[2024-08-26 07:47:24] [host.go:85] [HasLibbpf] Directory /usr/lib64/ does not exist         
DEBU[2024-08-26 07:47:24] [main.go:207] [checkHost] Libbpf found on host:                        
DEBU[2024-08-26 07:47:24] [main.go:209] [checkHost]     /usr/lib/libbpf.so.0                        
DEBU[2024-08-26 07:47:24] [main.go:209] [checkHost]     /usr/lib/libbpf.so.0.5.0                    
INFO[2024-08-26 07:47:24] [main.go:88] [main] Host meets requirements                      
INFO[2024-08-26 07:47:24] [main.go:91] [main] Getting device pools                         
DEBU[2024-08-26 07:47:24] [config.go:111] [GetPoolConfigs] Unprivileged BPF is allowed on this host     
DEBU[2024-08-26 07:47:24] [config.go:130] [GetPoolConfigs] eth0 is not a physical device, removing from list of host devices 
DEBU[2024-08-26 07:47:24] [config.go:130] [GetPoolConfigs] net0 is not a physical device, removing from list of host devices 
DEBU[2024-08-26 07:47:24] [config.go:130] [GetPoolConfigs] lo is not a physical device, removing from list of host devices 
DEBU[2024-08-26 07:47:24] [config.go:130] [GetPoolConfigs] tunl0 is not a physical device, removing from list of host devices 
DEBU[2024-08-26 07:47:24] [config.go:145] [GetPoolConfigs] Host devices:
{}                             
INFO[2024-08-26 07:47:24] [config.go:149] [GetPoolConfigs] Processing Pool: myPool                      
DEBU[2024-08-26 07:47:24] [config.go:163] [GetPoolConfigs] Using default UDS timeout: 30 seconds        
DEBU[2024-08-26 07:47:24] [config.go:273] [getDeviceListOfDriverType] Exit discovery.                              
[root@wluo k8snetworkplumbingwg]$ ll

when setup with kind. the af-xdp pod plugin shown above errors:
DEBU[2024-08-26 07:47:24] [config.go:130] [GetPoolConfigs] net0 is not a physical device, removing from list of host devices
YAML details:

# 1.prep noCNI env
cat <<EOF | kind create cluster --name=cni-multus --image=kindest/node:v1.27.3 --config=-
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
networking:
        disableDefaultCNI: true
nodes:
        - role: control-plane
        - role: worker
          extraMounts:
            - hostPath: /tmp/afxdp_dp/
              containerPath: /tmp/afxdp_dp/
              propagation: Bidirectional
              selinuxRelabel: false
        - role: worker
          extraMounts:
            - hostPath: /tmp/afxdp_dp2/
              containerPath: /tmp/afxdp_dp/
              propagation: Bidirectional
              selinuxRelabel: false

containerdConfigPatches:
- |-
  [plugins."io.containerd.grpc.v1.cri".registry.mirrors."192.168.2.100:5000"]
    endpoint = ["http://192.168.2.100:5000"]
EOF

# 2.remove taints
controller_node_ip=`kubectl get node -o wide --no-headers | grep -E "control-plane|bpf1" | awk -F " " '{print $6}'`
kubectl taint nodes $(kubectl get nodes -o name | grep control-plane) node-role.kubernetes.io/control-plane:NoSchedule-
kubectl get nodes -o wide
kubectl label node cni-multus-worker cndp="true"
kubectl label node cni-multus-worker2 cndp="true"

./2-setup-clab.sh

for i in $(docker ps --format '{{.Names}}'| grep cni-multus- | grep -v clab);do docker exec -it $i bash -c "sysctl kernel.unprivileged_bpf_disabled=0";done

# 3. install CNI[Calico v3.23.2]
kubectl apply -f ./k8snetworkplumbingwg

# 4. wait all pods ready
kubectl wait --timeout=100s --for=condition=Ready=true pods --all -A

[root@wluo 13-kind-multus-af-xdp]$ 

[root@wluo 13-kind-multus-af-xdp]$ cat 2-setup-clab.sh 
#!/bin/bash
set -v

{ ip l s br-pool0 down;ip l d br-pool0; } > /dev/null 2>&1
ip l a br-pool0 type bridge && ip l s br-pool0 up

cat <<EOF>clab.yaml | clab deploy -t clab.yaml -
name: cni-multus
topology:
  nodes:
    gw0:
      kind: linux
      image: 192.168.2.100:5000/vyos/vyos:1.4.9
      cmd: /sbin/init
      binds:
        - /lib/modules:/lib/modules
        - ./startup-conf/gw0-boot.cfg:/opt/vyatta/etc/config/config.boot
 
    br-pool0:
      kind: bridge

    server1:
      kind: linux
      image: 192.168.2.100:5000/nettool
      network-mode: container:cni-multus-control-plane

    server2:
      kind: linux
      image: 192.168.2.100:5000/nettool
      network-mode: container:cni-multus-worker

    server3:
      kind: linux
      image: 192.168.2.100:5000/nettool
      network-mode: container:cni-multus-worker2


  links:
    - endpoints: ["br-pool0:br-pool0-net0", "server1:net0"]
    - endpoints: ["br-pool0:br-pool0-net1", "server2:net0"]
    - endpoints: ["br-pool0:br-pool0-net2", "server3:net0"]

    - endpoints: ["gw0:eth1", "br-pool0:br-pool0-net3"]
EOF

the net0 is a veth device it should be virtual device. but the logs show it's not a physical device. so remove it.
not sure how can we setup the testbed at with kind.

kind node details:

[root@wluo 13-kind-multus-af-xdp]$ lo 3908967a6c43 ip a 
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: tunl0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN group default qlen 1000
    link/ipip 0.0.0.0 brd 0.0.0.0
188: eth0@if189: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default 
    link/ether 02:42:ac:12:00:02 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 172.18.0.2/16 brd 172.18.255.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fc00:1234:5678:abcd::2/64 scope global nodad 
       valid_lft forever preferred_lft forever
    inet6 fe80::42:acff:fe12:2/64 scope link 
       valid_lft forever preferred_lft forever
200: net0@if201: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9500 qdisc noqueue state UP group default 
    link/ether aa:c1:ab:06:28:f4 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet6 fe80::a8c1:abff:fe06:28f4/64 scope link 
       valid_lft forever preferred_lft forever
[root@wluo 13-kind-multus-af-xdp]$ 

Or we should build the image at Makfile rather use the image which already pushed at image: afxdp-device-plugin:latest