看门狗：BUG：软锁定 - CPU#6 卡住了 23 秒

Question

我尝试了在谷歌上找到的所有解决方案...我找不到我的服务器崩溃的原因...

Aug  5 17:11:08  kernel: [ 2300.084576] watchdog: BUG: soft lockup - CPU#6 stuck for 23s! [VM Thread:4054]
Aug  5 17:11:08  kernel: [ 2300.084578] Modules linked in: veth nf_conntrack_netlink nfnetlink xfrm_user xfrm_algo br_netfilter bridge stp llc rpcsec_gss_krb5 auth_rpcgss aufs nfsv4 nfs lockd grace fscache overlay isofs xt_nat xt_MASQUERADE xt_addrtype iptable_nat nf_nat xt_tcpudp xt_conntrack nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 iptable_filter bpfilter nls_iso8859_1 dm_multipath scsi_dh_rdac scsi_dh_emc scsi_dh_alua ppdev kvm_intel kvm ipmi_si input_leds joydev ipmi_devintf ipmi_msghandler video parport_pc parport acpi_pad sch_fq_codel drm sunrpc ip_tables x_tables autofs4 raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid0 multipath linear raid1 crct10dif_pclmul crc32_pclmul ghash_clmulni_intel hid_generic aesni_intel crypto_simd cryptd glue_helper usbhid igb hid nvme dca ahci i2c_algo_bit nvme_core libahci
Aug  5 17:11:08  kernel: [ 2300.084616] CPU: 6 PID: 4054 Comm: VM Thread Not tainted 5.4.0-42-generic #46-Ubuntu
Aug  5 17:11:08  kernel: [ 2300.084616] Hardware name: Intel Corporation S1200SP/S1200SP, BIOS S1200SP.86B.03.01.0042.013020190050 01/30/2019
Aug  5 17:11:08  kernel: [ 2300.084620] RIP: 0010:_raw_spin_lock+0x10/0x30
Aug  5 17:11:08  kernel: [ 2300.084621] Code: ff 01 00 00 75 07 4c 89 e0 41 5c 5d c3 e8 f8 f9 62 ff 4c 89 e0 41 5c 5d c3 90 0f 1f 44 00 00 31 c0 ba 01 00 00 00 f0 0f b1 17 <75> 01 c3 55 89 c6 48 89 e5 e8 c2 e1 62 ff 66 90 5d c3 66 66 2e 0f
Aug  5 17:11:08  kernel: [ 2300.084621] RSP: 0000:ffffa592c1bef760 EFLAGS: 00000246 ORIG_RAX: ffffffffffffff13
Aug  5 17:11:08  kernel: [ 2300.084622] RAX: 0000000000000000 RBX: 0000000000000100 RCX: ffff95314b79bc00
Aug  5 17:11:08  kernel: [ 2300.084622] RDX: 0000000000000001 RSI: 0000000000000588 RDI: ffff953145c1aeac
Aug  5 17:11:08  kernel: [ 2300.084623] RBP: ffffa592c1bef7b8 R08: ffff95314a5520f0 R09: 0000000000000000
Aug  5 17:11:08  kernel: [ 2300.084623] R10: 0000000000000000 R11: ffffffffffffffb8 R12: 0000000000000000
Aug  5 17:11:08  kernel: [ 2300.084623] R13: ffff953145c1ae00 R14: ffff95314b79bc00 R15: ffff953145c1aeac
Aug  5 17:11:08  kernel: [ 2300.084624] FS:  00007fa0e4151700(0000) GS:ffff953151580000(0000) knlGS:0000000000000000
Aug  5 17:11:08  kernel: [ 2300.084624] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Aug  5 17:11:08  kernel: [ 2300.084625] CR2: 0000000594832008 CR3: 000000045bf00003 CR4: 00000000003606e0
Aug  5 17:11:08  kernel: [ 2300.084625] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Aug  5 17:11:08  kernel: [ 2300.084625] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Aug  5 17:11:08  kernel: [ 2300.084626] Call Trace:
Aug  5 17:11:08  kernel: [ 2300.084628]  ? scan_swap_map_slots+0x3cd/0x510
Aug  5 17:11:08  kernel: [ 2300.084629]  get_swap_pages+0x207/0x380
Aug  5 17:11:08  kernel: [ 2300.084630]  ? rmap_walk_anon+0x16f/0x260
Aug  5 17:11:08  kernel: [ 2300.084632]  get_swap_page+0xe3/0x210
Aug  5 17:11:08  kernel: [ 2300.084633]  add_to_swap+0x1a/0x70
Aug  5 17:11:08  kernel: [ 2300.084634]  shrink_page_list+0x4b3/0xbb0
Aug  5 17:11:08  kernel: [ 2300.084648]  shrink_inactive_list+0x201/0x3e0
Aug  5 17:11:08  kernel: [ 2300.084649]  shrink_node_memcg+0x137/0x370
Aug  5 17:11:08  kernel: [ 2300.084650]  shrink_node+0xbd/0x400
Aug  5 17:11:08  kernel: [ 2300.084650]  do_try_to_free_pages+0xd7/0x3a0
Aug  5 17:11:08  kernel: [ 2300.084651]  try_to_free_mem_cgroup_pages+0xf4/0x210
Aug  5 17:11:08  kernel: [ 2300.084653]  try_charge+0x2eb/0x810
Aug  5 17:11:08  kernel: [ 2300.084654]  ? find_get_entry+0xaf/0x170
Aug  5 17:11:08  kernel: [ 2300.084655]  mem_cgroup_try_charge+0x71/0x190
Aug  5 17:11:08  kernel: [ 2300.084656]  ? pagecache_get_page+0x2d/0x300
Aug  5 17:11:08  kernel: [ 2300.084657]  mem_cgroup_try_charge_delay+0x22/0x50
Aug  5 17:11:08  kernel: [ 2300.084658]  do_swap_page+0x220/0x9f0
Aug  5 17:11:08  kernel: [ 2300.084659]  __handle_mm_fault+0x73b/0x7a0
Aug  5 17:11:08  kernel: [ 2300.084659]  handle_mm_fault+0xca/0x200
Aug  5 17:11:08  kernel: [ 2300.084661]  do_user_addr_fault+0x1f9/0x450
Aug  5 17:11:08  kernel: [ 2300.084662]  __do_page_fault+0x58/0x90
Aug  5 17:11:08  kernel: [ 2300.084663]  do_page_fault+0x2c/0xe0
Aug  5 17:11:08  kernel: [ 2300.084664]  page_fault+0x34/0x40
Aug  5 17:11:08  kernel: [ 2300.084665] RIP: 0033:0x7fa168646be3
Aug  5 17:11:08  kernel: [ 2300.084666] Code: 4c 89 6d b8 49 89 5d 00 49 c7 45 08 00 00 00 00 4c 3b 6d b0 0f 83 1d 01 00 00 4c 89 6d b0 49 89 dd 4d 39 fd 0f 83 bd 00 00 00 <49> 8b 45 00 4c 89 eb 83 e0 03 48 83 f8 03 0f 84 09 01 00 00 42 0f
Aug  5 17:11:08  kernel: [ 2300.084666] RSP: 002b:00007fa0e41501b0 EFLAGS: 00010283
Aug  5 17:11:08  kernel: [ 2300.084667] RAX: 00000005237c2908 RBX: 0000000000000004 RCX: 00007fa0e41504b0
Aug  5 17:11:08  kernel: [ 2300.084667] RDX: 0000000000000004 RSI: 0000000594831fe8 RDI: 00007fa160745850
Aug  5 17:11:08  kernel: [ 2300.084668] RBP: 00007fa0e4150230 R08: 00000005237c28e8 R09: 00007fa1607458f0
Aug  5 17:11:08  kernel: [ 2300.084668] R10: 00007fa168f52d99 R11: 000000014b7bf600 R12: 00007fa1609924d0
Aug  5 17:11:08  kernel: [ 2300.084668] R13: 0000000594832008 R14: 0000000000000240 R15: 0000000595000000

我更换了硬件，但更换了磁盘。

当我使用 Minecraft 启动 docker 容器 (pterodactyl) 时，有时它会冻结并出现上述错误。我找不到一些相关的日志...

名称-a： Linux X-X-X 5.4.0-42-generic #46-Ubuntu SMP Fri Jul 10 00:24:02 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

免费-h： total used free shared buff/cache available Mem: 31Gi 594Mi 29Gi 4.0Mi 1.1Gi 30Gi Swap: 1.0Gi 0B 1.0Gi

sysctl vm.swappiness ： vm.swappiness = 60

sudo lshw -C 内存：

  *-firmware
       description: BIOS
       vendor: Intel Corporation
       physical id: 6
       version: S1200SP.86B.03.01.0042.013020190050
       date: 01/30/2019
       size: 64KiB
       capacity: 16MiB
       capabilities: pci pnp upgrade shadowing cdboot bootselect edd int13floppy1200 int13floppy720 int13floppy2880 int5printscreen int9keyboard int14serial int17printer int10video acpi usb ls120boot zipboot biosbootspecification netboot uefi
  *-cache:0
       description: L1 cache
       physical id: 1a
       slot: L1 Cache
       size: 128KiB
       capacity: 128KiB
       capabilities: synchronous internal write-through instruction
       configuration: level=1
  *-cache:1
       description: L2 cache
       physical id: 1b
       slot: L2 Cache
       size: 1MiB
       capacity: 1MiB
       capabilities: synchronous internal write-through unified
       configuration: level=2
  *-cache:2
       description: L3 cache
       physical id: 1c
       slot: L3 Cache
       size: 8MiB
       capacity: 8MiB
       capabilities: synchronous internal write-back unified
       configuration: level=3
  *-cache
       description: L1 cache
       physical id: 19
       slot: L1 Cache
       size: 128KiB
       capacity: 128KiB
       capabilities: synchronous internal write-through data
       configuration: level=1
  *-memory
       description: System Memory
       physical id: 1e
       slot: System board or motherboard
       size: 32GiB
     *-bank:0
          description: [empty]
          vendor: Empty/NO DIMM
          physical id: 0
          slot: DIMM_A1
     *-bank:1
          description: DIMM DDR4 Synchronous 2400 MHz (0.4 ns)
          product: KHX2400C15/16G
          vendor: Kingston
          physical id: 1
          serial: A800F9241
          slot: DIMM_A2
          size: 16GiB
          width: 64 bits
          clock: 2400MHz (0.4ns)
     *-bank:2
          description: [empty]
          vendor: Empty/NO DIMM
          physical id: 2
          slot: DIMM_B1
     *-bank:3
          description: DIMM DDR4 Synchronous 2400 MHz (0.4 ns)
          product: KHX2400C15/16G
          vendor: Kingston
          physical id: 3
          serial: BE305496
          slot: DIMM_B2
          size: 16GiB
          width: 64 bits
          clock: 2400MHz (0.4ns)
  *-memory UNCLAIMED
       description: Memory controller
       product: 100 Series/C230 Series Chipset Family Power Management Controller
       vendor: Intel Corporation
       physical id: 1f.2
       bus info: pci@0000:00:1f.2
       version: 31
       width: 32 bits
       clock: 33MHz (30.3ns)
       capabilities: bus_master
       configuration: latency=0
       resources: memory:a2f10000-a2f13fff

grep -i 交换 /etc/fstab ：

UUID="X-X-X-X-X" swap swap defaults 0 0
UUID="X-X-X-X-X" swap swap defaults 0 0
/swapfile swap swap defaults 0 0

有任何想法吗 ？

Answer 1

可能的交换/内存问题。

\n

BIOS

\n

您的 BIOS 版本为 S1200SP.86B.03.01.0042.013020190050，日期为 2019 年 1 月 30 日。

\n

有更新的 BIOS 可用，日期为 2020 年 6 月，可以在此处下载下载。

\n

注意：更新 BIOS 之前请做好备份。

\n

内存测试

\n

访问https://www.memtest86.com/并免费下载/运行它们memtest来测试您的记忆力。至少完成一次所有 4/4 测试，以确认良好的记忆力。这可能需要几个小时才能完成。

\n

更新#1：

\n

正如我之前所想的......你有交换问题。

\n

您有三个交换位置，如 /etc/fstab 中所示！

\n

UUID="X-X-X-X-X" swap swap defaults 0 0\nUUID="X-X-X-X-X" swap swap defaults 0 0\n/swapfile swap swap defaults 0 0\n

\n

#关闭sudo swapoff -a 交换

\n

然后在 /etc/fstab 中注释掉上述所有三行。

\n

完全禁用交换永远是不行的。掉期太小是不合适的。你有两个问题。

\n

让我们为您的系统创建一个适当的 /swapfile。

\n

注意：错误使用dd可能会导致数据丢失。建议复制/粘贴。

\n

sudo swapoff -a           # turn off swap\nsudo rm -i /swapfile      # remove old /swapfile\n\nsudo dd if=/dev/zero of=/swapfile bs=1M count=4096\n\nsudo chmod 600 /swapfile  # set proper file protections\nsudo mkswap /swapfile     # init /swapfile\nsudo swapon /swapfile     # turn on swap\nfree -h                   # confirm 32G RAM and 4G swap\n

\n

将此行添加到 /etc/fstab...

\n

/swapfile \xc2\xa0 \xc2\xa0none \xc2\xa0 \xc2\xa0swap \xc2\xa0 \xc2\xa0sw \xc2\xa0 \xc2\xa0 \xc2\xa00   0\n

\n

然后reboot系统并验证运行。

\n

如果一切正常，您可以使用gparted/etc/fstab 中注释掉的行中显示的 UUID 来删除两个磁盘分区。这里要小心，并确保您有要删除的正确分区。然后删除 /etc/fstab 中这三行注释掉的行。

\n

Answer 2

尽管问题似乎已得到解答，但对于发现自己出现相同 CPU 错误的任何人（除了 heynnemas 答案之外），请检查您已连接的任何显卡的 PCI 电缆连接。

在断开显卡连接后，我遇到了同样的错误和问题，后来我意识到显卡的 6 针连接有故障（并且烧焦）。更换电缆后系统功能恢复正常。

我还建议检查 CPU/内存计时是否太疯狂，以及 CPU 冷却器是否正确连接（紧密）。