Chr*_*sen 173
管道缓冲区的容量因系统而异(甚至可能在同一系统上有所不同)。我不确定是否有一种快速、简单和跨平台的方法来查找管道的容量。
例如,Mac OS X 默认使用 16384 字节的容量,但如果对管道进行大量写入,则可以切换到 65336 字节的容量,或者如果内核内存过多,则会切换到单个系统页面的容量被管道缓冲区使用(参见xnu/bsd/sys/pipe.h, 和xnu/bsd/kern/sys_pipe.c; 因为它们来自 FreeBSD,同样的行为也可能发生在那里)。
一个 Linux pipe(7)手册页说,自 Linux 2.6.11 以来,管道容量为 65536 字节,而在此之前的单个系统页面(例如(32 位)x86 系统上为 4096 字节)。代码 ( include/linux/pipe_fs_i.h, 和fs/pipe.c) 似乎使用了 16 个系统页面(即,如果系统页面为 4 KiB,则为 64 KiB),但可以通过管道上的fcntl调整每个管道的缓冲区(最大容量默认为 1048576字节,但可以通过/proc/sys/fs/pipe-max-size))更改。
这是我用来测试系统上管道容量的一些bash / perl组合:
#!/bin/bash
test $# -ge 1 || { echo "usage: $0 write-size [wait-time]"; exit 1; }
test $# -ge 2 || set -- "$@" 1
bytes_written=$(
{
exec 3>&1
{
perl -e '
$size = $ARGV[0];
$block = q(a) x $size;
$num_written = 0;
sub report { print STDERR $num_written * $size, qq(\n); }
report; while (defined syswrite STDOUT, $block) {
$num_written++; report;
}
' "$1" 2>&3
} | (sleep "$2"; exec 0<&-);
} | tail -1
)
printf "write size: %10d; bytes successfully before error: %d\n" \
"$1" "$bytes_written"
Run Code Online (Sandbox Code Playgroud)
这是我在 Mac OS X 10.6.7 系统上以各种写入大小运行它时发现的结果(请注意写入大于 16KiB 的变化):
% /bin/bash -c 'for p in {0..18}; do /tmp/ts.sh $((2 ** $p)) 0.5; done'
write size: 1; bytes successfully before error: 16384
write size: 2; bytes successfully before error: 16384
write size: 4; bytes successfully before error: 16384
write size: 8; bytes successfully before error: 16384
write size: 16; bytes successfully before error: 16384
write size: 32; bytes successfully before error: 16384
write size: 64; bytes successfully before error: 16384
write size: 128; bytes successfully before error: 16384
write size: 256; bytes successfully before error: 16384
write size: 512; bytes successfully before error: 16384
write size: 1024; bytes successfully before error: 16384
write size: 2048; bytes successfully before error: 16384
write size: 4096; bytes successfully before error: 16384
write size: 8192; bytes successfully before error: 16384
write size: 16384; bytes successfully before error: 16384
write size: 32768; bytes successfully before error: 65536
write size: 65536; bytes successfully before error: 65536
write size: 131072; bytes successfully before error: 0
write size: 262144; bytes successfully before error: 0
Run Code Online (Sandbox Code Playgroud)
Linux 3.19 上的相同脚本:
/bin/bash -c 'for p in {0..18}; do /tmp/ts.sh $((2 ** $p)) 0.5; done'
write size: 1; bytes successfully before error: 65536
write size: 2; bytes successfully before error: 65536
write size: 4; bytes successfully before error: 65536
write size: 8; bytes successfully before error: 65536
write size: 16; bytes successfully before error: 65536
write size: 32; bytes successfully before error: 65536
write size: 64; bytes successfully before error: 65536
write size: 128; bytes successfully before error: 65536
write size: 256; bytes successfully before error: 65536
write size: 512; bytes successfully before error: 65536
write size: 1024; bytes successfully before error: 65536
write size: 2048; bytes successfully before error: 65536
write size: 4096; bytes successfully before error: 65536
write size: 8192; bytes successfully before error: 65536
write size: 16384; bytes successfully before error: 65536
write size: 32768; bytes successfully before error: 65536
write size: 65536; bytes successfully before error: 65536
write size: 131072; bytes successfully before error: 0
write size: 262144; bytes successfully before error: 0
Run Code Online (Sandbox Code Playgroud)
注意:PIPE_BUFC 头文件中定义的值(以及 的pathconf值_PC_PIPE_BUF)不指定管道的容量,而是指定可以原子写入的最大字节数(请参阅POSIX write(2))。
/* Differs from PIPE_BUF in that PIPE_SIZE is the length of the actual
memory allocation, whereas PIPE_BUF makes atomicity guarantees. */
Run Code Online (Sandbox Code Playgroud)
Asa*_*vic 45
这个 shell-line 也可以显示管道缓冲区大小:
M=0; while true; do dd if=/dev/zero bs=1k count=1 2>/dev/null; \
M=$(($M+1)); echo -en "\r$M KB" 1>&2; done | sleep 999
Run Code Online (Sandbox Code Playgroud)
(将 1k 块发送到阻塞的管道,直到缓冲区已满)...一些测试输出:
64K (intel-debian), 32K (aix-ppc), 64K (jslinux bellard.org) ...Ctrl+C.
Run Code Online (Sandbox Code Playgroud)
使用 printf 的最短 bash-one-liner:
M=0; while printf A; do >&2 printf "\r$((++M)) B"; done | sleep 999
Run Code Online (Sandbox Code Playgroud)
小智 8
以下是一些仅使用 shell 命令探索实际管道缓冲区容量的进一步替代方法:
# get pipe buffer size using Bash
yes produce_this_string_as_output | tee >(sleep 1) | wc -c
# portable version
( (sleep 1; exec yes produce_this_string_as_output) & echo $! ) |
(pid=$(head -1); sleep 2; kill "$pid"; wc -c </dev/stdin)
# get buffer size of named pipe
sh -c '
rm -f fifo
mkfifo fifo
yes produce_this_string_as_output | tee fifo | wc -c &
exec 3<&- 3<fifo
sleep 1
exec 3<&-
rm -f fifo
'
# Mac OS X
#getconf PIPE_BUF /
#open -e /usr/include/limits.h /usr/include/sys/pipe.h
# PIPE_SIZE
# BIG_PIPE_SIZE
# SMALL_PIPE_SIZE
# PIPE_MINDIRECT
Run Code Online (Sandbox Code Playgroud)
这是对 Ubuntu 12.04 YMMV 的快速而肮脏的黑客攻击
cat >pipesize.c
#include <unistd.h>
#include <errno.h>
#include </usr/include/linux/fcntl.h>
#include <stdio.h>
void main( int argc, char *argv[] ){
int fd ;
long pipesize ;
if( argc>1 ){
// if command line arg, associate a file descriptor with it
fprintf( stderr, "sizing %s ... ", argv[1] );
fd = open( argv[1], O_RDONLY|O_NONBLOCK );
}else{
// else use STDIN as the file descriptor
fprintf( stderr, "sizing STDIN ... " );
fd = 0 ;
}
fprintf( stderr, "%ld bytes\n", (long)fcntl( fd, F_GETPIPE_SZ ));
if( errno )fprintf( stderr, "Uh oh, errno is %d\n", errno );
if( fd )close( fd );
}
gcc -o pipesize pipesize.c
mkfifo /tmp/foo
./pipesize /tmp/foo
>sizing /tmp/foo ... 65536 bytes
date | ./pipesize
>sizing STDIN ... 65536 bytes
Run Code Online (Sandbox Code Playgroud)
如果您需要 Python>=3.3 中的值,这里有一个简单的方法(假设您可以运行 call out to dd):
from subprocess import Popen, PIPE, TimeoutExpired
p = Popen(["dd", "if=/dev/zero", "bs=1"], stdin=PIPE, stdout=PIPE)
try:
p.wait(timeout=1)
except TimeoutExpired:
p.kill()
print(len(p.stdout.read()))
Run Code Online (Sandbox Code Playgroud)