如何在没有内存错误的情况下使用var_dump + output buffering？

Question

我在一个应用程序中使用调试辅助工具,该应用程序使用var_dump()输出缓冲来捕获变量并显示它们.但是,我遇到了大型对象的问题,最终在缓冲区中占用了太多内存.

function getFormattedOutput(mixed $var) {
  if (isTooLarge($var)) { 
    return 'Too large! Abort!'; // What a solution *might* look like
  }

  ob_start();
  var_dump($var); // Fatal error:  Allowed memory size of 536870912 bytes exhausted
  $data = ob_get_clean();

  // Return the nicely-formated data to use later
  return $data
}

有没有办法阻止这种情况？或者解决方法是检测它是否要为特定变量输出大量信息？我真的没有控制哪些变量传递给这个函数.它可以是任何类型.

Answer 1

正如所有其他人都提到你所要求的是不可能的.你唯一能做的就是尽可能好地处理它.

您可以尝试将其拆分为更小的部分然后将其组合.我已经创建了一个小测试来尝试获取内存错误.显然,一个真实世界的例子可能表现不同,但这似乎可以解决问题.

<?php
define('mem_limit', return_bytes(ini_get('memory_limit'))); //allowed memory

/*
SIMPLE TEST CLASS
*/
class test { }
$loop = 260;
$t = new Test();
for ($x=0;$x<=$loop;$x++) {
  $v = 'test'.$x;
  $t->$v = new Test();
  for ($y=0;$y<=$loop;$y++) {
    $v2 = 'test'.$y;
    $t->$v->$v2 = str_repeat('something to test! ', 200);
  }
}
/* ---------------- */


echo saferVarDumpObject($t);

function varDumpToString($v) {
  ob_start();
  var_dump($v);
  $content = ob_get_contents();
  ob_end_clean();
  return $content;
}

function saferVarDumpObject($var) {
  if (!is_object($var) && !is_array($var))
    return varDumpToString($var);

  $content = '';
  foreach($var as $v) {
    $content .= saferVarDumpObject($v);
  }
  //adding these smaller pieces to a single var works fine.
  //returning the complete larger piece gives memory error

  $length = strlen($content);
  $left = mem_limit-memory_get_usage(true);

  if ($left>$length)
    return $content; //enough memory left

  echo "WARNING! NOT ENOUGH MEMORY<hr>";
  if ($left>100) {
    return substr($content, 0, $left-100); //100 is a margin I choose, return everything you have that fits in the memory
  } else {
    return ""; //return nothing.
  }  
}

function return_bytes($val) {
    $val = trim($val);
    $last = strtolower($val[strlen($val)-1]);
    switch($last) {
        // The 'G' modifier is available since PHP 5.1.0
        case 'g':
            $val *= 1024;
        case 'm':
            $val *= 1024;
        case 'k':
            $val *= 1024;
    }

    return $val;
}
?>

更新上面的版本仍有一些错误.我重新创建它以使用类和其他一些函数

• 检查递归
• 修复单个大属性
• 模仿var_dump输出
• 关于警告的trigger_error能够捕获/隐藏它

如注释中所示,类的资源标识符与var_dump的输出不同.据我所知,其他事情是平等的.

<?php  
/*
RECURSION TEST
*/
class sibling {
  public $brother;
  public $sister;
}
$brother = new sibling();
$sister = new sibling();
$brother->sister = $sister;
$sister->sister = $brother;
Dump::Safer($brother);


//simple class
class test { }

/*
LARGE TEST CLASS - Many items
*/
$loop = 260;
$t = new Test();
for ($x=0;$x<=$loop;$x++) {
  $v = 'test'.$x;
  $t->$v = new Test();
  for ($y=0;$y<=$loop;$y++) {
    $v2 = 'test'.$y;
    $t->$v->$v2 = str_repeat('something to test! ', 200);
  }
}
//Dump::Safer($t);
/* ---------------- */


/*
LARGE TEST CLASS - Large attribute
*/
$a = new Test();
$a->t2 = new Test();
$a->t2->testlargeattribute = str_repeat('1', 268435456 - memory_get_usage(true) - 1000000);
$a->smallattr1 = 'test small1';
$a->smallattr2 = 'test small2';
//Dump::Safer($a);
/* ---------------- */

class Dump
{
  private static $recursionhash;
  private static $memorylimit;
  private static $spacing;
  private static $mimicoutput = true;


  final public static function MimicOutput($v) {
    //show results similar to var_dump or without array/object information
    //defaults to similar as var_dump and cancels this on out of memory warning
    self::$mimicoutput = $v===false ? false : true;
  }

  final public static function Safer($var) {
    //set defaults
    self::$recursionhash = array();
    self::$memorylimit = self::return_bytes(ini_get('memory_limit'));

    self::$spacing = 0;

    //echo output
    echo self::saferVarDumpObject($var);
  }  

  final private static function saferVarDumpObject($var) {
    if (!is_object($var) && !is_array($var))
      return self::Spacing().self::varDumpToString($var);

    //recursion check
    $hash = spl_object_hash($var);
    if (!empty(self::$recursionhash[$hash])) {
      return self::Spacing().'*RECURSION*'.self::Eol();
    }
    self::$recursionhash[$hash] = true;


    //create a similar output as var dump to identify the instance
    $content = self::Spacing() . self::Header($var);
    //add some spacing to mimic vardump output
    //Perhaps not the best idea because the idea is to use as little memory as possible.
    self::$spacing++;
    //Loop trough everything to output the result
    foreach($var as $k=>$v) {
      $content .= self::Spacing().self::Key($k).self::Eol().self::saferVarDumpObject($v);
    }
    self::$spacing--;
    //decrease spacing and end the object/array
    $content .= self::Spacing().self::Footer().self::Eol();
    //adding these smaller pieces to a single var works fine.
    //returning the complete larger piece gives memory error

    //length of string and the remaining memory
    $length = strlen($content);
    $left = self::$memorylimit-memory_get_usage(true);

     //enough memory left?
    if ($left>$length)
      return $content;

    //show warning  
    trigger_error('Not enough memory to dump "'.get_class($var).'" memory left:'.$left, E_USER_WARNING);
    //stop mimic output to prevent fatal memory error
    self::MimicOutput(false);
    if ($left>100) {
      return substr($content, 0, $left-100); //100 is a margin I chose, return everything you have that fits in the memory
    } else {
      return ""; //return nothing.
    }  
  }

  final private static function Spacing() {
    return self::$mimicoutput ? str_repeat(' ', self::$spacing*2) : '';
  }

  final private static function Eol() {
    return self::$mimicoutput ? PHP_EOL : '';
  }

  final private static function Header($var) {
    //the resource identifier for an object is WRONG! Its always 1 because you are passing around parts and not the actual object. Havent foundnd a fix yet
    return self::$mimicoutput ? (is_array($var) ? 'array('.count($var).')' : 'object('.get_class($var).')#'.intval($var).' ('.count((array)$var).')') . ' {'.PHP_EOL : '';
  }

  final private static function Footer() {
    return self::$mimicoutput ? '}' : '';
  }

  final private static function Key($k) {
    return self::$mimicoutput ? '['.(gettype($k)=='string' ? '"'.$k.'"' : $k ).']=>' : '';
  }

  final private static function varDumpToString($v) {
    ob_start();
    var_dump($v);

    $length = strlen($v);
    $left = self::$memorylimit-memory_get_usage(true);

     //enough memory left with some margin?
    if ($left-100>$length) {
      $content = ob_get_contents();
      ob_end_clean();
      return $content;
    }
    ob_end_clean();

    //show warning  
    trigger_error('Not enough memory to dump "'.gettype($v).'" memory left:'.$left, E_USER_WARNING);

    if ($left>100) {
      $header = gettype($v).'('.strlen($v).')';
      return $header . substr($v, $left - strlen($header));
    } else {
      return ""; //return nothing.
    }  
  }

  final private static function return_bytes($val) {
      $val = trim($val);
      $last = strtolower($val[strlen($val)-1]);
      switch($last) {
          // The 'G' modifier is available since PHP 5.1.0
          case 'g':
              $val *= 1024;
          case 'm':
              $val *= 1024;
          case 'k':
              $val *= 1024;
      }

      return $val;
  }
}
?>

Answer 2

好吧,如果物理内存有限(你看到致命的错误:)

致命错误:允许的内存大小为536870912字节

我建议在磁盘上进行输出缓冲(参见callback参数ob_start).输出缓冲工作分块,这意味着,如果仍有足够的内存将单个块保留在内存中,则可以将其存储到临时文件中.

// handle output buffering via callback, set chunksize to one kilobyte
ob_start($output_callback, $chunk_size = 1024);

但是,您必须记住,这只会在缓冲时防止致命错误.如果您现在想要返回缓冲区,则仍需要有足够的内存,或者返回文件句柄或文件路径,以便您也可以流式传输输出.

但是,您可以使用该文件来获取所需的字节大小.PHP字符串的开销不是很多IIRC,所以如果仍然有足够的内存可用于文件大小,这应该可以正常工作.您可以减去偏移量以获得一点空间并保证安全.只是尝试和错误一点它的作用.

一些示例代码(PHP 5.4):

<?php
/**
 * @link http://stackoverflow.com/questions/5446647/how-can-i-use-var-dump-output-buffering-without-memory-errors/
 */

class OutputBuffer
{
    /**
     * @var int
     */
    private $chunkSize;

    /**
     * @var bool
     */
    private $started;

    /**
     * @var SplFileObject
     */
    private $store;

    /**
     * @var bool Set Verbosity to true to output analysis data to stderr
     */
    private $verbose = true;

    public function __construct($chunkSize = 1024) {
        $this->chunkSize = $chunkSize;
        $this->store     = new SplTempFileObject();
    }

    public function start() {
        if ($this->started) {
            throw new BadMethodCallException('Buffering already started, can not start again.');
        }
        $this->started = true;
        $result = ob_start(array($this, 'bufferCallback'), $this->chunkSize);
        $this->verbose && file_put_contents('php://stderr', sprintf("Starting Buffering: %d; Level %d\n", $result, ob_get_level()));
        return $result;
    }

    public function flush() {
        $this->started && ob_flush();
    }

    public function stop() {
        if ($this->started) {
            ob_flush();
            $result = ob_end_flush();
            $this->started = false;
            $this->verbose && file_put_contents('php://stderr', sprintf("Buffering stopped: %d; Level %d\n", $result, ob_get_level()));
        }
    }

    private function bufferCallback($chunk, $flags) {

        $chunkSize = strlen($chunk);

        if ($this->verbose) {
            $level     = ob_get_level();
            $constants = ['PHP_OUTPUT_HANDLER_START', 'PHP_OUTPUT_HANDLER_WRITE', 'PHP_OUTPUT_HANDLER_FLUSH', 'PHP_OUTPUT_HANDLER_CLEAN', 'PHP_OUTPUT_HANDLER_FINAL'];
            $flagsText = '';
            foreach ($constants as $i => $constant) {
                if ($flags & ($value = constant($constant)) || $value == $flags) {
                    $flagsText .= (strlen($flagsText) ? ' | ' : '') . $constant . "[$value]";
                }
            }

            file_put_contents('php://stderr', "Buffer Callback: Chunk Size $chunkSize; Flags $flags ($flagsText); Level $level\n");
        }

        if ($flags & PHP_OUTPUT_HANDLER_FINAL) {
            return TRUE;
        }

        if ($flags & PHP_OUTPUT_HANDLER_START) {
            $this->store->fseek(0, SEEK_END);
        }

        $chunkSize && $this->store->fwrite($chunk);

        if ($flags & PHP_OUTPUT_HANDLER_FLUSH) {
            // there is nothing to d
        }

        if ($flags & PHP_OUTPUT_HANDLER_CLEAN) {
            $this->store->ftruncate(0);
        }

        return "";
    }

    public function getSize() {
        $this->store->fseek(0, SEEK_END);
        return $this->store->ftell();
    }

    public function getBufferFile() {
        return $this->store;
    }

    public function getBuffer() {
        $array = iterator_to_array($this->store);
        return implode('', $array);
    }

    public function __toString() {
        return $this->getBuffer();
    }

    public function endClean() {
        return ob_end_clean();
    }
}


$buffer  = new OutputBuffer();
echo "Starting Buffering now.\n=======================\n";
$buffer->start();

foreach (range(1, 10) as $iteration) {
    $string = "fill{$iteration}";
    echo str_repeat($string, 100), "\n";
}
$buffer->stop();

echo "Buffering Results:\n==================\n";
$size = $buffer->getSize();
echo "Buffer Size: $size (string length: ", strlen($buffer), ").\n";
echo "Peeking into buffer: ", var_dump(substr($buffer, 0, 10)), ' ...', var_dump(substr($buffer, -10)), "\n";

输出:

STDERR: Starting Buffering: 1; Level 1
STDERR: Buffer Callback: Chunk Size 1502; Flags 1 (PHP_OUTPUT_HANDLER_START[1]); Level 1
STDERR: Buffer Callback: Chunk Size 1503; Flags 0 (PHP_OUTPUT_HANDLER_WRITE[0]); Level 1
STDERR: Buffer Callback: Chunk Size 1503; Flags 0 (PHP_OUTPUT_HANDLER_WRITE[0]); Level 1
STDERR: Buffer Callback: Chunk Size 602; Flags 4 (PHP_OUTPUT_HANDLER_FLUSH[4]); Level 1
STDERR: Buffer Callback: Chunk Size 0; Flags 8 (PHP_OUTPUT_HANDLER_FINAL[8]); Level 1
STDERR: Buffering stopped: 1; Level 0
Starting Buffering now.
=======================
Buffering Results:
==================
Buffer Size: 5110 (string length: 5110).
Peeking into buffer: string(10) "fill1fill1"
 ...string(10) "l10fill10\n"

Answer 3

当您运行xdebug时,您可以限制var_dump跟踪对象的深度.在某些软件产品中,您可能会遇到一种递归,这会使var_dump的输出膨胀.除此之外,您可以提高内存限制.

见http://www.xdebug.org/docs/display

Answer 4

对不起,我认为你的问题没有办法解决.您要求确定大小以防止该大小的内存分配.PHP无法给出"它将占用多少内存"的答案,因为ZVAL结构是在PHP中使用时创建的.请参阅PHP - 14.5编程.内存管理,概述PHP的内存分配内部.

你给出了正确的提示"里面可以有任何东西",从我的角度来看,这就是问题所在.有一个架构问题会导致您描述的情况.而且我认为你试图在错误的一端解决它.

例如:您可以从php中的每种类型的开关开始,并尝试为每种大小设置限制.只要没有人想到在流程中改变内存限制,这就会持续下去.

Xdebug是一个很好的解决方案,因为它可以防止应用程序因为(甚至非关键业务)日志功能而爆炸,并且它是一个糟糕的解决方案,因为你不应该在生产中激活xdebug.

我认为内存异常是正确的行为,你不应该尝试解决它.

[rant]如果转储50兆字节或更多字符串的人不关心他/她的应用行为,他/她应该受其影响;)[/ rant]

Answer 5

我不相信有任何方法可以确定特定功能最终会占用多少内存.你可以做的一件事是使用memory_get_usage()来检查脚本在$largeVar设置之前当前正在采取多少内存,然后将其与之后的数量进行比较.这将使您对大小有所了解$largeVar,并且您可以运行试验以确定在优雅退出之前可接受的最大大小限制.

您也可以自己重新实现var_dump()函数.让函数遍历结构并在生成内容时回显生成的内容,或将其存储在临时文件中,而不是将巨大的字符串存储在内存中.这将使您获得相同的所需结果,但不会遇到您遇到的内存问题.