调用 iptables 导致 docker 中“不支持操作”

Question

iptables我在 Pod 中使用时遇到错误：

root@chris-sshuttle-k8stest:~# iptables -t nat -nL
iptables: Operation not supported.

---

如果我直接使用 docker 运行图像，但它工作正常：

docker run --cap-add=NET_ADMIN -it --rm chrissound/sshuttle-k8stest:v2 /bin/bash
root@e857b0d4152a:/# iptables -t nat -nL
Chain PREROUTING (policy ACCEPT)
target     prot opt source               destination         
...

另外，输出为capsh --print：

root@chris-sshuttle-k8stest:~# capsh --print
Current: = cap_chown,cap_dac_override,cap_dac_read_search,cap_fowner,cap_fsetid,cap_kill,cap_setgid,cap_setuid,cap_setpcap,cap_linux_immutable,cap_net_bind_service,cap_net_broadcast,cap_net_admin,cap_net_raw,cap_ipc_lock,cap_ipc_owner,cap_sys_module,cap_sys_rawio,cap_sys_chroot,cap_sys_ptrace,cap_sys_pacct,cap_sys_admin,cap_sys_boot,cap_sys_nice,cap_sys_resource,cap_sys_time,cap_sys_tty_config,cap_mknod,cap_lease,cap_audit_write,cap_audit_control,cap_setfcap,cap_mac_override,cap_mac_admin,cap_syslog,cap_wake_alarm,cap_block_suspend,cap_audit_read+eip
Bounding set =cap_chown,cap_dac_override,cap_dac_read_search,cap_fowner,cap_fsetid,cap_kill,cap_setgid,cap_setuid,cap_setpcap,cap_linux_immutable,cap_net_bind_service,cap_net_broadcast,cap_net_admin,cap_net_raw,cap_ipc_lock,cap_ipc_owner,cap_sys_module,cap_sys_rawio,cap_sys_chroot,cap_sys_ptrace,cap_sys_pacct,cap_sys_admin,cap_sys_boot,cap_sys_nice,cap_sys_resource,cap_sys_time,cap_sys_tty_config,cap_mknod,cap_lease,cap_audit_write,cap_audit_control,cap_setfcap,cap_mac_override,cap_mac_admin,cap_syslog,cap_wake_alarm,cap_block_suspend,cap_audit_read
Securebits: 00/0x0/1'b0
 secure-noroot: no (unlocked)
 secure-no-suid-fixup: no (unlocked)
 secure-keep-caps: no (unlocked)
uid=0(root)
gid=0(root)
groups=

其中确实有net_admin：

root@chris-sshuttle-k8stest:~# capsh --print | grep net_admin
Current: = cap_chown,cap_dac_override,cap_dac_read_search,cap_fowner,cap_fsetid,cap_kill,cap_setgid,cap_setuid,cap_setpcap,cap_linux_immutable,cap_net_bind_service,cap_net_broadcast,cap_net_admin,cap_net_raw,cap_ipc_lock,cap_ipc_owner,cap_sys_module,cap_sys_rawio,cap_sys_chroot,cap_sys_ptrace,cap_sys_pacct,cap_sys_admin,cap_sys_boot,cap_sys_nice,cap_sys_resource,cap_sys_time,cap_sys_tty_config,cap_mknod,cap_lease,cap_audit_write,cap_audit_control,cap_setfcap,cap_mac_override,cap_mac_admin,cap_syslog,cap_wake_alarm,cap_block_suspend,cap_audit_read+eip
Bounding set =cap_chown,cap_dac_override,cap_dac_read_search,cap_fowner,cap_fsetid,cap_kill,cap_setgid,cap_setuid,cap_setpcap,cap_linux_immutable,cap_net_bind_service,cap_net_broadcast,cap_net_admin,cap_net_raw,cap_ipc_lock,cap_ipc_owner,cap_sys_module,cap_sys_rawio,cap_sys_chroot,cap_sys_ptrace,cap_sys_pacct,cap_sys_admin,cap_sys_boot,cap_sys_nice,cap_sys_resource,cap_sys_time,cap_sys_tty_config,cap_mknod,cap_lease,cap_audit_write,cap_audit_control,cap_setfcap,cap_mac_override,cap_mac_admin,cap_syslog,cap_wake_alarm,cap_block_suspend,cap_audit_read

pod.yaml：

apiVersion: v1
kind: Pod
metadata:
  name: chris-sshuttle-k8stest
  labels:
    name: chris-sshuttle-k8stest 
spec:
  containers:
  - name: sshuttle
    image: chrissound/sshuttle-k8stest:v2
    command: ["sleep", "10000000"]
    securityContext:
      privileged: true
      capabilities:
        add: ["NET_ADMIN","NET_RAW"]

---

额外调试：

通过 SSH 连接到 k8s 节点并检查 docker 容器，一切似乎都是正确的：

$ docker inspect 6f96802d7e13 | grep -B 4 -A 4 NET_AD
            "AutoRemove": false,
            "VolumeDriver": "",
            "VolumesFrom": null,
            "CapAdd": [
                "NET_ADMIN",
                "NET_RAW"
            ],
            "CapDrop": null,
            "Dns": null,

Answer 1

看来错误与 iptables 有关。非常感谢@KFC_ 对此进行调查。

python:3.7-slim奇怪的是，当我安装后再次从图像运行它时iptables：我得到了额外的输出：

# iptables -t nat -nL
# Warning: iptables-legacy tables present, use iptables-legacy to see them
iptables: Operation not supported.

在这里找到解决方案：https ://github.com/docker/libnetwork/issues/2331

update-alternatives --set iptables /usr/sbin/iptables-legacy