DPDK 示例应用程序在 EAL 后中止:无法获取大页文件上的 fd

Nag*_*gev 3 huge-pages dpdk

克隆dpdk git 存储库并构建helloworld应用程序后,出现以下错误:

$ ./examples/helloworld/build/helloworld
EAL: Detected 4 lcore(s)
EAL: Detected 1 NUMA nodes
EAL: Multi-process socket /run/user/1000/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'VA'
EAL: No available hugepages reported in hugepages-1048576kB
EAL: Probing VFIO support...
EAL: VFIO support initialized
EAL: Couldn't get fd on hugepage file
EAL: error allocating rte services array
EAL: FATAL: rte_service_init() failed
EAL: rte_service_init() failed
PANIC in main():
Cannot init EAL
5: [./examples/helloworld/build/helloworld(+0x11de) [0x56175faac1de]]
4: [/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3) [0x7f31f60fe0b3]]
3: [./examples/helloworld/build/helloworld(+0x111c) [0x56175faac11c]]
2: [/lib/x86_64-linux-gnu/librte_eal.so.20.0(__rte_panic+0xc5) [0x7f31f62d537e]]
1: [/lib/x86_64-linux-gnu/librte_eal.so.20.0(rte_dump_stack+0x32) [0x7f31f62ecc52]]
Aborted (core dumped)
Run Code Online (Sandbox Code Playgroud)

检查了大页支持,看起来不错:

$ cat /proc/meminfo | grep -i huge
AnonHugePages:         0 kB
ShmemHugePages:        0 kB
FileHugePages:         0 kB
HugePages_Total:     256
HugePages_Free:      255
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
Hugetlb:          524288 kB

$ mount | grep huge
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime,pagesize=2M)

$ cat /proc/filesystems | grep huge
nodev   hugetlbfs

$ cat /proc/sys/vm/nr_hugepages
256
Run Code Online (Sandbox Code Playgroud)

我在相关问题中看到了解决方法;使用以下选项运行它--no-huge,该选项有效:

$ ./examples/helloworld/build/helloworld --no-huge
EAL: Detected 4 lcore(s)
EAL: Detected 1 NUMA nodes
EAL: Static memory layout is selected, amount of reserved memory can be adjusted with -m or --socket-mem
EAL: Multi-process socket /run/user/1000/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'VA'
EAL: Probing VFIO support...
EAL: VFIO support initialized
EAL: PCI device 0000:02:01.0 on NUMA socket -1
EAL:   Invalid NUMA socket, default to 0
EAL:   probe driver: 8086:100f net_e1000_em
EAL: PCI device 0000:03:00.0 on NUMA socket -1
EAL:   Invalid NUMA socket, default to 0
EAL:   probe driver: 15ad:7b0 net_vmxnet3
hello from core 1
hello from core 2
hello from core 3
hello from core 0
Run Code Online (Sandbox Code Playgroud)

但这是一个有限的解决方案。

Nag*_*gev 5

TL;DR 使用sudo

\n

按照 @VipinVarghese 的建议运行--log-level=eal,8显示这是一个权限问题:

\n
$ ./examples/helloworld/build/helloworld --log-level=eal,8\nEAL: Detected lcore 0 as core 0 on socket 0\nEAL: Detected lcore 1 as core 0 on socket 0\nEAL: Detected lcore 2 as core 0 on socket 0\nEAL: Detected lcore 3 as core 0 on socket 0\nEAL: Support maximum 128 logical core(s) by configuration.\nEAL: Detected 4 lcore(s)\nEAL: Detected 1 NUMA nodes\nEAL: open shared lib /usr/lib/x86_64-linux-gnu/dpdk/pmds-20.0/librte_pmd_qede.so\nEAL: Registered [vdev] bus.\nEAL: Registered [pci] bus.\nEAL: Registered [eth] device class.\nEAL: open shared lib /usr/lib/x86_64-linux-gnu/dpdk/pmds-20.0/librte_pmd_aesni_mb.so\n...\nEAL: Ask a virtual area of 0x61000 bytes\nEAL: Virtual area found at 0xd00600000 (size = 0x61000)\nEAL: Memseg list allocated: 0x800kB at socket 0\nEAL: Ask a virtual area of 0x400000000 bytes\nEAL: Virtual area found at 0xd00800000 (size = 0x400000000)\nEAL: TSC frequency is ~2590000 KHz\nEAL: Master lcore 0 is ready (tid=7fc11ed01d00;cpuset=[0])\nEAL: lcore 2 is ready (tid=7fc116ffd700;cpuset=[2])\nEAL: lcore 3 is ready (tid=7fc1167fc700;cpuset=[3])\nEAL: lcore 1 is ready (tid=7fc1177fe700;cpuset=[1])\nEAL: Trying to obtain current memory policy.\nEAL: Setting policy MPOL_PREFERRED for socket 0\nEAL: get_seg_fd(): open failed: Permission denied\nEAL: Couldn\'t get fd on hugepage file\nEAL: attempted to allocate 1 segments, but only 0 were allocated\nEAL: Restoring previous memory policy: 0\nEAL: error allocating rte services array\nEAL: FATAL: rte_service_init() failed\nEAL: rte_service_init() failed\nPANIC in main():\nCannot init EAL\n5: [./examples/helloworld/build/helloworld(+0x11de) [0x56459e5391de]]\n4: [/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3) [0x7fc11eed00b3]]\n3: [./examples/helloworld/build/helloworld(+0x111c) [0x56459e53911c]]\n2: [/lib/x86_64-linux-gnu/librte_eal.so.20.0(__rte_panic+0xc5) [0x7fc11f0a737e]]\n1: [/lib/x86_64-linux-gnu/librte_eal.so.20.0(rte_dump_stack+0x32) [0x7fc11f0bec52]]\nAborted (core dumped)\n
Run Code Online (Sandbox Code Playgroud)\n

尝试解决权限问题(EAL:get_seg_fd():打开失败:权限被拒绝),但只有当我以root身份运行它时才有效:

\n
$ sudo ./examples/helloworld/build/helloworld\nEAL: Detected 4 lcore(s)\nEAL: Detected 1 NUMA nodes\nEAL: Multi-process socket /var/run/dpdk/rte/mp_socket\nEAL: Selected IOVA mode \'PA\'\nEAL: No available hugepages reported in hugepages-1048576kB\nEAL: Probing VFIO support...\nEAL: VFIO support initialized\nEAL: PCI device 0000:02:01.0 on NUMA socket -1\nEAL:   Invalid NUMA socket, default to 0\nEAL:   probe driver: 8086:100f net_e1000_em\nEAL: PCI device 0000:03:00.0 on NUMA socket -1\nEAL:   Invalid NUMA socket, default to 0\nEAL:   probe driver: 15ad:7b0 net_vmxnet3\nhello from core 1\nhello from core 2\nhello from core 3\nhello from core 0\n
Run Code Online (Sandbox Code Playgroud)\n

事实证明,这是正确的方法,尽管文档似乎认为这是显而易见的。“6.2. 运行示例应用程序”部分没有提及所需的 root 权限,摘录如下:

\n
\n

将 DPDK 应用程序二进制文件复制到您的目标,然后按如下方式运行应用程序(假设平台每个处理器插槽有四个内存通道,并且核心 0-3 存在并用于运行应用程序):

\n

./dpdk-helloworld -l 0-3 -n 4

\n
\n

不过,这一点在文档后面提到,请参阅“ 8.2. 在没有 Root 权限的情况下运行 DPDK 应用程序”,其中有明确的说明:

\n
\n

以下说明将允许使用较旧的\nLinux 内核版本以非 root 身份运行 DPDK。但是,从版本 4.0 开始,内核不允许非特权进程从页面映射文件读取物理地址信息,从而使这些进程无法使用需要物理地址的硬件设备

\n
\n

常见问题解答中也有提示:

\n
\n
    \n
  1. \xe2\x80\x9cEAL: map_all_hugepages(): open failed: Permission returned 无法初始化内存\xe2\x80\x9d 是什么意思?这很可能是由于测试应用程序未使用 sudo 运行来将用户提升为超级用户。或者,应用程序也可以以常规用户身份运行。\n 有关详细信息,请参阅 DPDK 入门\n指南。
  2. \n
\n
\n

还有一封涉及该主题的电子邮件:

\n
2016-07-07 16:47, Jez Higgins:\n> Is it possible to get DPDK up and running as non-root - if so, can\n> anyone guide me to what I\'m missing? Or should I be giving this up as a\n> bad job?\n\nYou can try the --no-huge option.\nBut most of drivers won\'t work without hugepage currently.\nA rework of the memory allocation is needed to make it work better.\n
Run Code Online (Sandbox Code Playgroud)\n

那是四年前的事了。也许已经有一个不需要sudoor的解决方案--no-huge?如果是这样,欢迎其他答案。现在,我就这么做。

\n