Unicorn自行重启 - 没有内存 - 被杀死

Question

我在DigitalOcean上运行两个Rails应用程序,512MB RAM和4个nginx进程.

rails应用程序使用Unicorn.

一个有2个工人,另一个用1个.

我的问题是第二个应用程序有1个Unicorn工作者(当有2个工作者时也存在同样的问题).会发生什么,突然我的应用程序抛出500错误.当我SSH到服务器时,我会发现应用程序的独角兽进程没有运行!

当我再次开始独角兽时,一切都会好的.

这是我的日志文件.正如您所看到的,工作人员获得了收益,然后它无法分叉,给出的理由是No Memory.

, [2014-01-24T04:12:28.080716 #8820]  INFO -- : master process ready
I, [2014-01-24T04:12:28.110834 #8824]  INFO -- : worker=0 ready

E, [2014-01-24T06:45:08.423082 #8820] ERROR -- : reaped #<Process::Status: pid 8824 SIGKILL (signal 9)> worker=0
E, [2014-01-24T06:45:08.438352 #8820] ERROR -- : Cannot allocate memory - fork(2) (Errno::ENOMEM)
/home/vocp/projects/hmd/vendor/bundle/ruby/2.0.0/gems/unicorn-4.7.0/lib/unicorn/http_server.rb:523:in `fork'
/home/vocp/projects/hmd/vendor/bundle/ruby/2.0.0/gems/unicorn-4.7.0/lib/unicorn/http_server.rb:523:in `spawn_missing_workers'
/home/vocp/projects/hmd/vendor/bundle/ruby/2.0.0/gems/unicorn-4.7.0/lib/unicorn/http_server.rb:538:in `maintain_worker_count'
/home/vocp/projects/hmd/vendor/bundle/ruby/2.0.0/gems/unicorn-4.7.0/lib/unicorn/http_server.rb:303:in `join'
/home/vocp/projects/hmd/vendor/bundle/ruby/2.0.0/gems/unicorn-4.7.0/bin/unicorn:126:in `<top (required)>'
/home/vocp/projects/hmd/vendor/bundle/ruby/2.0.0/bin/unicorn:23:in `load'
/home/vocp/projects/hmd/vendor/bundle/ruby/2.0.0/bin/unicorn:23:in `<main>'

I, [2014-01-24T08:43:53.693228 #26868]  INFO -- : Refreshing Gem list
I, [2014-01-24T08:43:56.283950 #26868]  INFO -- : unlinking existing socket=/tmp/unicorn.hmd.sock
I, [2014-01-24T08:43:56.284840 #26868]  INFO -- : listening on addr=/tmp/unicorn.hmd.sock fd=11
I, [2014-01-24T08:43:56.320075 #26868]  INFO -- : master process ready
I, [2014-01-24T08:43:56.348648 #26872]  INFO -- : worker=0 ready

E, [2014-01-24T09:10:07.251846 #26868] ERROR -- : reaped #<Process::Status: pid 26872 SIGKILL (signal 9)> worker=0
I, [2014-01-24T09:10:07.300339 #27743]  INFO -- : worker=0 ready
I, [2014-01-24T09:18:09.992675 #28039]  INFO -- : executing ["/home/vocp/projects/hmd/vendor/bundle/ruby/2.0.0/bin/unicorn", "-D", "-c", "/home/vocp/projects/hmd/config/unicorn.rb", "-E", "production", {11=>#<Kgio::UNIXServer:/tmp/unicorn.hmd.sock>}] (in /home/vocp/projects/hmd)
I, [2014-01-24T09:18:10.426852 #28039]  INFO -- : inherited addr=/tmp/unicorn.hmd.sock fd=11
I, [2014-01-24T09:18:10.427090 #28039]  INFO -- : Refreshing Gem list
E, [2014-01-24T09:18:13.456986 #28039] ERROR -- : Cannot allocate memory - fork(2) (Errno::ENOMEM)
/home/vocp/projects/hmd/vendor/bundle/ruby/2.0.0/gems/unicorn-4.7.0/lib/unicorn/http_server.rb:523:in `fork'
/home/vocp/projects/hmd/vendor/bundle/ruby/2.0.0/gems/unicorn-4.7.0/lib/unicorn/http_server.rb:523:in `spawn_missing_workers'
/home/vocp/projects/hmd/vendor/bundle/ruby/2.0.0/gems/unicorn-4.7.0/lib/unicorn/http_server.rb:153:in `start'
/home/vocp/projects/hmd/vendor/bundle/ruby/2.0.0/gems/unicorn-4.7.0/bin/unicorn:126:in `<top (required)>'
/home/vocp/projects/hmd/vendor/bundle/ruby/2.0.0/bin/unicorn:23:in `load'
/home/vocp/projects/hmd/vendor/bundle/ruby/2.0.0/bin/unicorn:23:in `<main>'
E, [2014-01-24T09:18:13.464982 #26868] ERROR -- : reaped #<Process::Status: pid 28039 exit 1> exec()-ed

这是我的 unicorn.rb

root = "/home/vocp/projects/hmd"
working_directory root
pid "#{root}/tmp/pids/unicorn.pid"
stderr_path "#{root}/log/unicorn.log"
stdout_path "#{root}/log/unicorn.log"

listen "/tmp/unicorn.hmd.sock"
worker_processes 1
timeout 30
preload_app true

# Force the bundler gemfile environment variable to
# reference the capistrano "current" symlink
before_exec do |_|
  ENV["BUNDLE_GEMFILE"] = File.join(root, 'Gemfile')
end


before_fork do |server, worker|
  defined?(ActiveRecord::Base) && ActiveRecord::Base.connection.disconnect!

  old_pid = Rails.root + '/tmp/pids/unicorn.pid.oldbin'
  if File.exists?(old_pid) && server.pid != old_pid
    begin
      Process.kill("QUIT", File.read(old_pid).to_i)
    rescue Errno::ENOENT, Errno::ESRCH
      puts "Old master alerady dead"
    end
  end
end

after_fork do |server, worker|
  defined?(ActiveRecord::Base) && ActiveRecord::Base.establish_connection
  child_pid = server.config[:pid].sub('.pid', ".#{worker.nr}.pid")
  system("echo #{Process.pid} > #{child_pid}")
end

我没有monit或god或任何监控工具.我发现它很奇怪,因为通常使用的服务器内存是380/490.没有人使用这两个应用程序除了我!他们正在发展中.

我错误配置了什么吗？为什么会这样？请帮忙.我应该配置上帝在崩溃时重启独角兽吗？

Answer 1

不幸的是,对于Unicorn内存的使用,唯一的方法是启动.如果您的rails应用程序需要,Unicorn将分配更多内存.但它不会释放它,即使它不再需要它.例如,如果您一次为索引页加载大量记录,独角兽将增加内存使用量.现在,由于512MB对于拥有3名工作人员的2个轨道应用来说并不是一个巨大的内存,这一点更加恶化.

此外,还有内存泄漏也会增加内存使用量.请参阅此文章 https://www.digitalocean.com/community/articles/how-to-optimize-unicorn-workers-in-a-ruby-on-rails-app

在文章的最后,他们引用了unicorn-worker-killer gem,以便根据max connections或max memory重新启动unicorn worker,这看起来非常简单.

就个人而言,我已经使用bluepill gem监视各个独角兽进程,并在需要时重新启动它们.

在您的情况下,我将监视所有unicorn进程,并在它们达到特定内存大小时重新启动它们.