Pod 无法在 kubernetes 中分配大页

Question

我正在 kubernetes 中运行一个 pod，在主机中分配大页，并在 pod 中定义大页。kubernetes 工作线程位于虚拟机中。VM（主机）分配了大页面。但 Pod 无法分配大页。当尝试写入第一个大页分配时，应用程序会收到 SIGBUS。

pod 定义包括大页面：

    securityContext:
      allowPrivilegeEscalation: true
      privileged: true
      runAsUser: 0
      capabilities:
        add: ["SYS_ADMIN", "IPC_LOCK"]
    resources:
      requests:
        intel.com/intel_sriov_netdevice : 2
        memory: 2Gi
        hugepages-2Mi: 4Gi
      limits:
        intel.com/intel_sriov_netdevice : 2
        memory: 2Gi
        hugepages-2Mi: 4Gi
    volumeMounts:
    - mountPath: /sys
      name: sysfs
    - mountPath: /dev/hugepages
      name: hugepage
      readOnly: false
  volumes:
  - name: hugepage
    emptyDir:
      medium: HugePages
  - name: sysfs
    hostPath:
      path: /sys
    

托管 Pod 的虚拟机分配了大页面：

cat /proc/meminfo | grep -i hug
AnonHugePages:         0 kB
HugePages_Total:    4096
HugePages_Free:     4096
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB

下面的代码在托管 Pod 的虚拟机中运行良好，我可以看到在 中创建的大页面文件/dev/hugepages，而且在进程运行时 HugePages_Free 计数器也会减少。

cat /proc/meminfo | grep -i hug
AnonHugePages:         0 kB
HugePages_Total:    4096
HugePages_Free:     4096
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB

但是，如果我在 pod 中运行相同的代码，我会在尝试写入分配的第一个大页时收到 SIGBUS。

VM 上的结果（托管 Pod）

root@k8s-1:~# cat /proc/meminfo | grep -i hug
AnonHugePages:         0 kB
HugePages_Total:    4096
HugePages_Free:     4096
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
root@k8s-1:~# ./mmap  &
[1] 19428
root@k8s-1:~# address returned 0x7ffff7800000
address returned 0x7ffff7600000
address returned 0x7ffff7400000
address returned 0x7ffff7200000
address returned 0x7ffff7000000
address returned 0x7ffff6e00000
address returned 0x7ffff6c00000
address returned 0x7ffff6a00000
address returned 0x7ffff6800000
address returned 0x7ffff6600000
address returned 0x7ffff6400000
address returned 0x7ffff6200000
address returned 0x7ffff6000000
address returned 0x7ffff5e00000
address returned 0x7ffff5c00000
address returned 0x7ffff5a00000

root@k8s-1:~# cat /proc/meminfo | grep -i hug
AnonHugePages:         0 kB
HugePages_Total:    4096
HugePages_Free:     4080
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB

pod 中的结果：

Program received signal SIGBUS, Bus error.
0x00005555555547cb in write_bytes ()
(gdb) where
#0  0x00005555555547cb in write_bytes ()
#1  0x00005555555548a6 in main ()

Answer 1

在worker节点上配置hugepage后，需要重新启动kubelet才能通知K8S Hugepage资源。（当我在 K8S 1.16 版本中尝试这个时，我遇到了同样的问题，也许现在它效果更好了？）