当子进程触摸变量元数据时,如何防止内存爆炸？

Question

Linux使用COW在分叉后保持较低的内存使用率,但Perl 5变量的工作方式perl似乎打败了这种优化.例如,对于变量:

my $s = "1";

perl 真的存储:

SV = PV(0x100801068) at 0x1008272e8
  REFCNT = 1
  FLAGS = (POK,pPOK)
  PV = 0x100201d50 "1"\0
  CUR = 1
  LEN = 16

在数字上下文中使用该字符串时,它会修改struct表示数据的C :

SV = PVIV(0x100821610) at 0x1008272e8
  REFCNT = 1
  FLAGS = (IOK,POK,pIOK,pPOK)
  IV = 1
  PV = 0x100201d50 "1"\0
  CUR = 1
  LEN = 16

字符串指针本身没有改变(它仍然是0x100201d50),但现在它处于不同的C struct(PVIV而不是a PV).我根本没有修改价值,但突然间我付了COW费用.有没有办法锁定perlPerl 5变量的表示,以便节省时间(perl不必转换"0"为0第二次)hack不会损害我的内存使用？

注意,上面的表示是从这段代码生成的:

perl -MDevel::Peek -e '$s = "1"; Dump $s; $s + 0; Dump $s'

Answer 1

到目前为止，我找到的唯一解决方案是确保我强制perl执行我期望在父进程中进行的所有转换。从下面的代码中你可以看到，即使这样也只能起到一点帮助。

结果：

Useless use of addition (+) in void context at z.pl line 34.
Useless use of addition (+) in void context at z.pl line 45.
Useless use of addition (+) in void context at z.pl line 51.
before eating memory
used memory: 71
after eating memory
used memory: 119
after 100 forks that don't reference variable
used memory: 144
after children are reaped
used memory: 93
after 100 forks that touch the variables metadata
used memory: 707
after children are reaped
used memory: 93
after parent has updated the metadata
used memory: 109
after 100 forks that touch the variables metadata
used memory: 443
after children are reaped
used memory: 109

代码：

#!/usr/bin/perl

use strict;
use warnings;

use Parallel::ForkManager;

sub print_mem {
    print @_, "used memory: ", `free -m` =~ m{cache:\s+([0-9]+)}s, "\n";
}

print_mem("before eating memory\n");

my @big = ("1") x (1_024 * 1024);

my $pm = Parallel::ForkManager->new(100);

print_mem("after eating memory\n");

for (1 .. 100) {
    next if $pm->start;
    sleep 2;
    $pm->finish;
}

print_mem("after 100 forks that don't reference variable\n");

$pm->wait_all_children;

print_mem("after children are reaped\n");

for (1 .. 100) {
    next if $pm->start;
    $_ + 0 for @big; #force an update to the metadata
    sleep 2;
    $pm->finish;
}

print_mem("after 100 forks that touch the variables metadata\n");

$pm->wait_all_children;

print_mem("after children are reaped\n");

$_ + 0 for @big; #force an update to the metadata

print_mem("after parent has updated the metadata\n");

for (1 .. 100) {
    next if $pm->start;
    $_ + 0 for @big; #force an update to the metadata
    sleep 2;
    $pm->finish;
}

print_mem("after 100 forks that touch the variables metadata\n");

$pm->wait_all_children;

print_mem("after children are reaped\n");