zLib透明写模式"wT"性能下降

Question

我希望zLib透明模式(gzptintf())和常规fprintf()一样快.我发现带有"wT"的zLib gzprintf()比fprintf()慢2.5倍.有关此性能问题的解决方法吗？

细节:

我在Linux上使用libz.so.1.2.8(fedora 22,内核4.0.5,Intel(R)Core(TM)i7-3770 CPU @ 3.40GHz)为我的事件跟踪收集器提供输出文件压缩选项.为了保持传统兼容性,我需要透明文件格式写入模式.

正如我所见,gzopen中的选项"T"允许写入没有压缩的文件而没有gzip头记录.

问题在于性能.透明模式比简单标准fprintf慢约2.5倍.

这是快速测试结果(值在TSC中):

    zLib]$ ./zlib_transparent
Performance fprintf vs gzprintf (transparent):
     fprintf 22883026324
zLib transp  62305122876
ratio 2.72277

此测试的来源:

#include <stdio.h>
#include <zlib.h>
#include <iostream>
#include <sstream>
#include <iomanip>

#define NUMITERATIONS 10000000
static double buffer[NUMITERATIONS];

static __inline__ unsigned long long rdtsc(void){
    unsigned hi, lo;
    __asm__ __volatile__ ("rdtsc" : "=a"(lo), "=d"(hi));
    return ( (unsigned long long)lo)|( ((unsigned long long)hi)<<32 );
}

long long test_fprintf(double *buffer){
    long long t = rdtsc();
#ifdef USE_FPRINTF
    double tmp = 0;
    FILE *file = fopen("fprintf_file.txt", "w");
    for (int i = 0; i < NUMITERATIONS; ++i) {
        fprintf(file, "[%f:%f]\n", buffer[i], buffer[i] - tmp);
        tmp = buffer[i] + i;
    }
    fclose(file);
#endif
    return rdtsc() - t;
}

long long test_zlib_transparent(double *buffer){
    long long t = rdtsc();
#ifdef USE_ZLIB
    double tmp = 0;
    gzFile file = gzopen("zlib_file.txt.gz", "wT");
    for (int i = 0; i < NUMITERATIONS; ++i) {
        gzprintf(file, "[%f:%f]\n", buffer[i], buffer[i] - tmp);
        tmp = buffer[i] + i;
    }
    gzclose(file);
#endif
    return rdtsc() - t;
}


int main(){
    std::cout << "Performance fprintf vs gzprintf (transparent):" << std::endl;
    long long dPrint = test_fprintf(buffer);
    std::cout << "     fprintf " << dPrint << std::endl;

    long long dStream = test_zlib_transparent(buffer);
    std::cout << "zLib transp  " << dStream << std::endl;

    std::cout << "ratio " << double(dStream)/double(dPrint) << std::endl;
    return 0;
}

建立:

g++ -g -O3 -DUSE_ZLIB=1 -DUSE_FPRINTF=1 zlib_transparent.cpp -o zlib_transparent –lz

谢谢

谢尔盖

Answer 1

我的错.(我写道gzprintf().)

write()被经常被召唤.如果fprintf()用snprintf()and 替换,你将获得与zlib大致相同的性能write().

我将在下一版本的zlib中对此进行改进.如果您想尝试一下,请应用此差异.我不知道它将如何在Linux上运行,但在Mac OS X上,gzprintf()透明模式现在比它快 10%fprintf().(没想到.)