containerd手动配置容器网络

机器详情

操作系统：Ubuntu 22.04.4 LTS
内核版本：5.15.0-112-generic
containerd版本：v1.7.1
nerdctl版本：1.0.0

nerdctl启动一个不带网络的容器

nerdctl  run -d --network none swr.cn-north-4.myhuaweicloud.com/ctl456/nginx:latest

获取容器ID、PID与network namespace路径

nerdctl ps

nerdctl inspect 容器ID -f '{{ .State.Pid }}'

此时可以查看容器网络命名空间中的网络接口，可以看到网络命名空间内只有一个网络回环接口lo，并没有其他任何配置

nsenter -t PID -n ip a

network namespace路径

/proc/PID/ns/net

准备bridge插件的执行配置文件

vim bridge.json

{"cniVersion": "1.0.0","name": "dbnet","type": "bridge","bridge": "mycni0","isGateway": true,"keyA": ["some more","plugin specific","configuration"],"ipam": {"type": "host-local","subnet": "10.1.0.0/16","routes": [{"dst": "0.0.0.0/0"}]},"dns": {"nameservers": ["10.1.0.1"]}
}

通过下面的命令调用bridge插件

CNI_COMMAND=ADD CNI_CONTAINERID=容器ID CNI_NETNS=network namespace路径 CNI_IFNAME=eth0 CNI_PATH=/opt/cni/bin /opt/cni/bin/bridge < ~/bridge.json

成功返回如下的内容

root@ubuntu:~# CNI_COMMAND=ADD CNI_CONTAINERID=3cc3646b6e9c CNI_NETNS=/proc/1377/ns/net CNI_IFNAME=eth0 CNI_PATHcni/bin /opt/cni/bin/bridge < ~/bridge.json
{"cniVersion": "1.0.0","interfaces": [{"name": "mycni0","mac": "12:15:f7:e2:95:cd"},{"name": "veth9bfbdf99","mac": "22:0d:c2:3d:48:ca"},{"name": "eth0","mac": "3a:98:85:45:f5:af","sandbox": "/proc/1377/ns/net"}],"ips": [{"interface": 2,"address": "10.1.0.2/16","gateway": "10.1.0.1"}],"routes": [{"dst": "0.0.0.0/0"}],"dns": {"nameservers": ["10.1.0.1"]}
root@ubuntu:~# 

可以再次提供如下的命令查看容器IP

nsenter -t PID -n ip a

查看物理机的IP
查看物理机路由

可以通过容器的IP访问到nginx服务

准备tuning插件文件

vim tuning.json​​

{"cniVersion": "1.0.0","name": "dbnet","type": "tuning","sysctl": {"net.core.somaxconn": "500"},"runtimeConfig": {"mac": "00:11:22:33:44:66"}, /*替换capabilities,将eth0的mac值调整为测试值*/"prevResult": { /*调用bridge插件放回的内容*/"interfaces": [{"name": "mycni0","mac": "12:15:f7:e2:95:cd"},{"name": "veth9bfbdf99","mac": "22:0d:c2:3d:48:ca"},{"name": "eth0","mac": "3a:98:85:45:f5:af","sandbox": "/proc/1377/ns/net"}],"ips": [{"interface": 2,"address": "10.1.0.2/16","gateway": "10.1.0.1"}],"routes": [{"dst": "0.0.0.0/0"}],"dns": {"nameservers": ["10.1.0.1"]}}
}

执行下面的命令调用tuning插件

CNI_COMMAND=ADD CNI_CONTAINERID=容器ID CNI_NETNS=network namespace路径 CNI_IFNAME=eth0 CNI_PATH=/opt/cni/bin /opt/cni/bin/tuning < ~/tuning.json

成功返回如下的内容

root@ubuntu:~# CNI_COMMAND=AD​​D​​ CNI_CONTAI​​NERID=3cc3646b6e9c CNI_NETNS=/proc/1377/ns/net CNI_IFNAME=eth0 CNI_PATH=/opt/cni/bin /opt/cni/bin/tuning < ~/tuning.json​​ 
{"cniVersion": "1.0.0","interfaces": [{"name": "mycni0","mac": "12:15:f7:e2:95:cd"},{"name": "veth9bfbdf99","mac": "22:0d:c2:3d:48:ca"},{"name": "eth0","mac": "00:11:22:33:44:66","sandbox": "/proc/1377/ns/net"}],"ips": [{"interface": 2,"address": "10.1.0.2/16","gateway": "10.1.0.1"}],"routes": [{"dst": "0.0.0.0/0"}],"dns": {"nameservers": ["10.1.0.1"]}
}root@ubuntu:~# 

可以通过如下的命令查看容器IP的mac地址是否修改

nsenter -t PID -n ip a

准备portmap插件文件

vim portmap.json

{"cniVersion": "1.0.0","name": "dbnet","type": "portmap","runtimeConfig": {"portMappings": [{"hostPort": 8080,"containerPort": 80,"protocol": "tcp"}]},"prevResult": {"interfaces": [{"name": "mycni0","mac": "12:15:f7:e2:95:cd"},{"name": "veth9bfbdf99","mac": "22:0d:c2:3d:48:ca"},{"name": "eth0","mac": "00:11:22:33:44:66","sandbox": "/proc/1377/ns/net"}],"ips": [{"interface": 2,"address": "10.1.0.2/16","gateway": "10.1.0.1"}],"routes": [{"dst": "0.0.0.0/0"}],"dns": {"nameservers": ["10.1.0.1"]}}
}

执行下面的命令调用portmap插件

CNI_COMMAND=ADD CNI_CONTAINERID=容器ID CNI_NETNS=network namespace路径 CNI_IFNAME=eth0 CNI_PATH=/opt/cni/bin /opt/cni/bin/portmap < ~/portmap.json

成功返回如下的内容

root@ubuntu:~# CNI_COMMAND=ADD CNI_CONTAINERID=3cc3646b6e9c CNI_NETNS=/proc/1377/ns/net CNI_IFNAME=eth0 CNI_PATH=/opt/cni/bin /opt/cni/bin/portmap <​​ ~/portmap.json 
{"cniVersion": "1.0.0","interfaces": [{"name": "mycni0","mac": "12:15:f7:e2:95:cd"},{"name": "veth9bfbdf99","mac": "22:0d:c2:3d:48:ca"},{"name": "eth0","mac": "00:11:22:33:44:66","sandbox": "/proc/1377/ns/net"}],"ips": [{"interface": 2,"address": "10.1.0.2/16","gateway": "10.1.0.1"}],"routes": [{"dst": "0.0.0.0/0"}],"dns": {"nameservers": ["10.1.0.1"]}
}root@ubuntu:~#

可以通过物理及的IP:8080访问到容器的nginx服务

删除网络

创建网络时，容器运行时按照顺序依次调用bridge、tuning、portmap插件，而删除网络时，则按照相反的顺序依次调用portmap、tuning、bridge插件。

CNI_COMMAND=DEL CNI_CONTAINERID=容器ID CNI_NETNS=network namespace路径 CNI_IFNAME=eth0 CNI_PATH=/opt/cni/bin /opt/cni/bin/portmap < ~/portmap.json

CNI_COMMAND=DEL CNI_CONTAINERID=容器ID CNI_NETNS=network namespace路径 CNI_IFNAME=eth0 CNI_PATH=/opt/cni/bin /opt/cni/bin/tuning < ~/tuning.json

vim bridge-del.json

{"cniVersion": "1.0.0","name": "dbnet","type": "bridge","bridge": "mycni0","isGateway": true,"keyA": ["some more","plugin specific","configuration"],"ipam": {"type": "host-local","subnet": "10.1.0.0/16","routes": [{"dst": "0.0.0.0/0"}]},"dns": {"nameservers": ["10.1.0.1"]},"prevResult": {"interfaces": [{"name": "mycni0","mac": "12:15:f7:e2:95:cd"},{"name": "veth9bfbdf99","mac": "22:0d:c2:3d:48:ca"},{"name": "eth0","mac": "3a:98:85:45:f5:af","sandbox": "/proc/1377/ns/net"}],"ips": [{"interface": 2,"address": "10.1.0.2/16","gateway": "10.1.0.1"}],"routes": [{"dst": "0.0.0.0/0"}],"dns": {"nameservers": ["10.1.0.1"]}}
}

CNI_COMMAND=DEL CNI_CONTAINERID=容器ID CNI_NETNS=network namespace路径 CNI_IFNAME=eth0 CNI_PATH=/opt/cni/bin /opt/cni/bin/bridge < ~/bridge-del.json