Mik*_*ril 4 linux ubuntu bash oom
我的老朋友OOM杀手来拜访我已经有很长时间了。现在它每周来一次，我无法弄清楚它为什么被触发。似乎我没有失去记忆。是什么导致了这种情况，可以采取哪些措施来防止这种情况发生？
内核：3.13.0-49-generic #83-Ubuntu SMP
就在 OOM 杀手被触发之前：
Fri Nov 16 01:45:01 EST 2018
COMMAND %MEM
mysqld 17.2
apache2 7.2
spamd child 7.0
apache2 7.0
apache2 6.9
apache2 6.8
apache2 6.8
apache2 6.6
apache2 6.6
apache2 6.4
apache2 6.4
/usr/sbin/spamd 6.4
spamd child 6.2
apache2 5.6
opendkim 0.6
horde-alarms 0.5
fail2ban-server 0.5
apache2 0.4
munin-node 0.2
pickup 0.1
cron 0.1
cron 0.1
cron 0.1
mem.sh 0.1
init 0.1
ps 0.1
config 0.0
tlsmgr 0.0
ntpd 0.0
qmgr 0.0
sh 0.0
sh 0.0
sh 0.0
systemd-logind 0.0
dovecot 0.0
auth 0.0
whoopsie 0.0
rsyslogd 0.0
log 0.0
vnstatd 0.0
master 0.0
dbus-daemon 0.0
anvil 0.0
cron 0.0
upstart-file-br 0.0
php5-fpm 0.0
upstart-socket- 0.0
upstart-udev-br 0.0
memcached 0.0
sshd 0.0
shellinaboxd 0.0
getty 0.0
getty 0.0
getty 0.0
getty 0.0
getty 0.0
php5-fpm 0.0
php5-fpm 0.0
/usr/sbin/postg 0.0
getty 0.0
systemd-udevd 0.0
acpid 0.0
opendkim 0.0
kthreadd 0.0
ksoftirqd/0 0.0
kworker/0:0H 0.0
rcu_sched 0.0
rcuos/0 0.0
rcu_bh 0.0
rcuob/0 0.0
migration/0 0.0
watchdog/0 0.0
khelper 0.0
kdevtmpfs 0.0
netns 0.0
writeback 0.0
kintegrityd 0.0
bioset 0.0
kworker/u3:0 0.0
kblockd 0.0
ata_sff 0.0
khubd 0.0
md 0.0
devfreq_wq 0.0
kworker/0:1 0.0
khungtaskd 0.0
kswapd0 0.0
ksmd 0.0
khugepaged 0.0
fsnotify_mark 0.0
ecryptfs-kthrea 0.0
crypto 0.0
kthrotld 0.0
vballoon 0.0
scsi_eh_0 0.0
scsi_eh_1 0.0
deferwq 0.0
charger_manager 0.0
scsi_eh_2 0.0
kpsmoused 0.0
kworker/0:2 0.0
ttm_swap 0.0
qxl_gc 0.0
jbd2/vda1-8 0.0
ext4-rsv-conver 0.0
kvm-irqfd-clean 0.0
atd 0.0
shellinaboxd 0.0
managesieve-log 0.0
kworker/u3:1 0.0
kauditd 0.0
kworker/u2:2 0.0
kworker/u2:0 0.0
Memory Space Details
Total Memory space : 2018 MB
Used Memory Space : 1055 MB
Free Memory : 962 MB
Swap memory Details
Total Swap space : 1023 MB
Used Swap Space : 164 MB
Free Swap : 859 MB
Node 0, zone DMA 9 12 6 8 7 4 1 2 2 0 0
Node 0, zone DMA32 2202 535 425 464 304 89 26 24 12 1 0
系统日志：
Nov 16 01:46:34 foobar kernel: [2269670.312226] apache2 invoked oom-killer: gfp_mask=0x200da, order=0, oom_score_adj=0
Nov 16 01:46:34 foobar kernel: [2269670.312231] apache2 cpuset=/ mems_allowed=0
Nov 16 01:46:34 foobar kernel: [2269670.312234] CPU: 0 PID: 14040 Comm: apache2 Not tainted 3.13.0-49-generic #83-Ubuntu
Nov 16 01:46:34 foobar kernel: [2269670.312235] Hardware name: DigitalOcean Droplet, BIOS 20171212 12/12/2017
Nov 16 01:46:34 foobar kernel: [2269670.312237] 0000000000000000 ffff880003f3b988 ffffffff81721b36 ffff8800041f1800
Nov 16 01:46:34 foobar kernel: [2269670.312240] ffff880003f3ba10 ffffffff8171c3f1 ffffffff81067996 ffff880003f3b9e8
Nov 16 01:46:34 foobar kernel: [2269670.312242] ffffffff810c75dc 0000000000000000 ffff88003fff7e28 0000000000000000
Nov 16 01:46:34 foobar kernel: [2269670.312244] Call Trace:
Nov 16 01:46:34 foobar kernel: [2269670.312250] [<ffffffff81721b36>] dump_stack+0x45/0x56
Nov 16 01:46:34 foobar kernel: [2269670.312257] [<ffffffff8171c3f1>] dump_header+0x7f/0x1f1
Nov 16 01:46:34 foobar kernel: [2269670.312262] [<ffffffff81067996>] ? put_online_cpus+0x56/0x80
Nov 16 01:46:34 foobar kernel: [2269670.312273] [<ffffffff810c75dc>] ? rcu_oom_notify+0xcc/0xf0
Nov 16 01:46:34 foobar kernel: [2269670.312276] [<ffffffff811529ce>] oom_kill_process+0x1ce/0x330
Nov 16 01:46:34 foobar kernel: [2269670.312280] [<ffffffff812d75f5>] ? security_capable_noaudit+0x15/0x20
Nov 16 01:46:34 foobar kernel: [2269670.312282] [<ffffffff81153104>] out_of_memory+0x414/0x450
Nov 16 01:46:34 foobar kernel: [2269670.312285] [<ffffffff81159470>] __alloc_pages_nodemask+0xa60/0xb80
Nov 16 01:46:34 foobar kernel: [2269670.312289] [<ffffffff81199dda>] alloc_pages_vma+0x9a/0x140
Nov 16 01:46:34 foobar kernel: [2269670.312292] [<ffffffff8118c5ab>] read_swap_cache_async+0xeb/0x160
Nov 16 01:46:34 foobar kernel: [2269670.312294] [<ffffffff8118c6b8>] swapin_readahead+0x98/0xe0
Nov 16 01:46:34 foobar kernel: [2269670.312297] [<ffffffff8117a536>] handle_mm_fault+0xa76/0xf10
Nov 16 01:46:34 foobar kernel: [2269670.312301] [<ffffffff8172db74>] __do_page_fault+0x184/0x570
Nov 16 01:46:34 foobar kernel: [2269670.312305] [<ffffffff811ceb6f>] ? user_path_at_empty+0x5f/0x90
Nov 16 01:46:34 foobar kernel: [2269670.312308] [<ffffffff810a02c5>] ? set_next_entity+0x95/0xb0
Nov 16 01:46:34 foobar kernel: [2269670.312312] [<ffffffff8101260b>] ? __switch_to+0x16b/0x4d0
Nov 16 01:46:34 foobar kernel: [2269670.312314] [<ffffffff8172df7a>] do_page_fault+0x1a/0x70
Nov 16 01:46:34 foobar kernel: [2269670.312316] [<ffffffff8172d5e9>] do_async_page_fault+0x29/0xe0
Nov 16 01:46:34 foobar kernel: [2269670.312320] Mem-Info:
Nov 16 01:46:34 foobar kernel: [2269670.312321] Node 0 DMA per-cpu:
Nov 16 01:46:34 foobar kernel: [2269670.312323] CPU 0: hi: 0, btch: 1 usd: 0
Nov 16 01:46:34 foobar kernel: [2269670.312324] Node 0 DMA32 per-cpu:
Nov 16 01:46:34 foobar kernel: [2269670.312325] CPU 0: hi: 186, btch: 31 usd: 93
Nov 16 01:46:34 foobar kernel: [2269670.312329] active_anon:101755 inactive_anon:101941 isolated_anon:208
Nov 16 01:46:34 foobar kernel: [2269670.312329] active_file:294 inactive_file:309 isolated_file:64
Nov 16 01:46:34 foobar kernel: [2269670.312329] unevictable:0 dirty:0 writeback:63 unstable:0
Nov 16 01:46:34 foobar kernel: [2269670.312329] free:12210 slab_reclaimable:2743 slab_unreclaimable:8287
Nov 16 01:46:34 foobar kernel: [2269670.312329] mapped:2580 shmem:6561 pagetables:20656 bounce:0
Nov 16 01:46:34 foobar kernel: [2269670.312329] free_cma:0
Nov 16 01:46:34 foobar kernel: [2269670.312331] Node 0 DMA free:4596kB min:704kB low:880kB high:1056kB active_anon:4844kB i
nactive_anon:5424kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15992kB ma
naged:15908kB mlocked:0kB dirty:0kB writeback:0kB mapped:92kB shmem:96kB slab_reclaimable:64kB slab_unreclaimable:232kB kernel
_stack:40kB pagetables:684kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Nov 16 01:46:34 foobar kernel: [2269670.312336] lowmem_reserve[]: 0 975 975 975
Nov 16 01:46:34 foobar kernel: [2269670.312338] Node 0 DMA32 free:44244kB min:44348kB low:55432kB high:66520kB act
Mic*_*ton 10
这里有很多事情你需要做。
首先，OOM 杀手是由apache2请求比可用内存多的内存触发的，而不是由 mysqld触发的。但是因为当时 mysqld 进程使用了最多的内存，它被杀死了。
考虑设置 sysctl vm.oom_kill_allocating_task，这将导致请求内存的进程被杀死。
其次，您很可能已经使用 mpm_prefork 配置了 Apache，并且正在将 PHP 作为 Apache 模块运行。这种设置很慢，性能不佳，不是线程安全的，当 PHP 失败时，失败会归咎于其他组件，因为 PHP 很好地隐藏在 Apache 内部。现代 Web 服务器设置使用 php-fpm 和带有 mpm_event 的 nginx 或 Apache。这将为您提供更好的性能和对 PHP 正在做什么的可见性（因为它很可能会占用您所有的内存）。
最后，您的 Droplet 似乎有 1GB RAM 和 1GB 交换空间。如果您经常用完，那么您的虚拟机可能太小了。考虑添加 RAM。
| 归档时间： |
|
| 查看次数： |
12407 次 |
| 最近记录： |