Rails 3 + Heroku:找出dyno的内存使用情况

Question

有没有办法找出heroku web dyno使用了多少内存？假设我想写一个定期运行的rake任务并检查每个dyno的内存使用情况.我怎么能这样做？谢谢!

Answer 1

现在还有一种本地Heroku方式可以做到这一点.Heroku推出了实验室功能日志运行时指标,它将CPU负载和内存使用信息注入日志记录流.这些日志带有源ID(例如"web.1")和dyno唯一ID,因此您可以将dynos区分开来.它看起来像这样:

source=web.1 dyno=heroku.2808254.d97d0ea7-cf3d-411b-b453-d2943a50b456 sample#load_avg_1m=2.46 sample#load_avg_5m=1.06 sample#load_avg_15m=0.99
source=web.1 dyno=heroku.2808254.d97d0ea7-cf3d-411b-b453-d2943a50b456 sample#memory_total=21.00MB sample#memory_rss=21.22MB sample#memory_cache=0.00MB sample#memory_swap=0.00MB sample#memory_pgpgin=348836pages sample#memory_pgpgout=343403pages

要启用此功能,只需运行:

heroku labs:enable log-runtime-metrics
heroku restart

Answer 2

我从接受的答案中提出了建议,并使用聚合和基于阈值的差异实现了/ proc文件系统解析器.我发现它在调试Heroku上的一些Ruby 2.0内存问题时非常有用.获取代码,为方便起见,此处也包括在内.

# Memory snapshot analyzer which parses the /proc file system on *nix
#
# Example (run in Heroku console):
#
#    ms = MemorySnapshot.new
#    1.upto(10000).map { |i| Array.new(i) }; nil
#    ms.snapshot!; nil
#    ms.diff 10
#    => {"lib/ld-2.11.1.so"=>156, "heap"=>2068, "all"=>2224}
#
class MemorySnapshot

  attr_reader :previous
  attr_reader :current

  def initialize
    snapshot!
    @previous = @current
  end

  # Generates a Hash of memory elements mapped to sizes of the elements in Kb
  def snapshot!
    @previous = @current
    @current = reduce(names_with_sizes)
  end

  # Calculates the difference between the previous and the current snapshot
  # Threshold is a minimum delta in kilobytes required to include an entry
  def diff(threshold = 0)
    self.class.diff_between previous, current, threshold
  end

  # Calculates the difference between two memory snapshots
  # Threshold is a minimum delta in kilobytes required to include an entry
  def self.diff_between(before, after, threshold)
    names = (before.keys + after.keys).uniq
    names.reduce({}) do |memo, name|
      delta = after.fetch(name) { 0 } - before.fetch(name) { 0 }
      memo[name] = delta if delta.abs >= threshold
      memo
    end
  end

  private

  def reduce(matches)
    total = 0
    current_name = nil
    matches.reduce(Hash.new { 0 }) do |memo, match|
      current_name = match[:name] || current_name
      size = match[:size].to_i
      total += size
      memo[current_name] += size
      memo
    end.tap { |snapshot| snapshot['all'] = total }
  end

  def names_with_sizes
    smap_entries.map do |line|
      /((^(\/|\[)(?<name>[^ \]]+)\]?\s+)|(^))(?<size>\d+)\s/.match(line)
    end
  end

  def smap_entries
    smaps.
        gsub(/^(([^Sa-f0-9])|(S[^i]))[^\n]+\n/m, '').
        gsub(/\nSize:/m, '').
        gsub(/[0-9a-f]+-[0-9a-f]+.{6}[0-9a-f]+ [0-9a-f]+:[0-9a-f]+ [0-9a-f]+\s+/i, '').
        split("\n")
  end

  def smaps
    File.read("/proc/#{Process.pid}/smaps")
  end
end

Answer 3

由于 Heroku 在采用 Linux 的 Amazon 实例上运行，因此您可以使用proc文件系统来获取运行时系统数据。该/proc/<pid>/smaps文件包含有关正在运行的进程及其加载的所有库的内存信息。对于驻留进程大小，请执行以下操作：

f = File.open("/proc/#{Process.pid}/smaps")
f.gets # Throw away first line
l = f.gets # Contains a string like "Size:               2148 kB\n"
l =~ /(\d+) kB/  # match the size in kB 
f.close
$1.to_i  # returns matched size as integer

更新：proc 文件系统有一个更好的源，/proc/<pid>/status. 有一个条目VmRSS:用于表示总驻留内存部分和VmHWM:使用的最高驻留内存（高水位线）。有关这些字段和其他字段的更多详细信息，请参阅proc 文件系统的 Linux 内核文档。