在解析文件时,Rust比Python慢

Question

我希望使用Rust来加速目前用Python编写的一些文本处理程序.

为了测试两种语言的性能,我决定在一个非常简单的任务上测试它们:

• STDIN逐行读入文件.
• 如果该行以该行开头>,则将该行保存到headers.txt文件中.
• 否则,将行保存到sequences.txt文件中.

对于此测试,我使用的是一个包含1000万行的fasta文件,如下所示:

$ head uniparc_active-head.fasta
>UPI0000000001 status=active
MGAAASIQTTVNTLSERISSKLEQEANASAQTKCDIEIGNFYIRQNHGCNLTVKNMCSAD
ADAQLDAVLSAATETYSGLTPEQKAYVPAMFTAALNIQTSVNTVVRDFENYVKQTCNSSA
VVDNKLKIQNVIIDECYGAPGSPTNLEFINTGSSKGNCAIKALMQLTTKATTQIAPKQVA
GTGVQFYMIVIGVIILAALFMYYAKRMLFTSTNDKIKLILANKENVHWTTYMDTFFRTSP
MVIATTDMQN
>UPI0000000002 status=active
MMTPENDEEQTSVFSATVYGDKIQGKNKRKRVIGLCIRISMVISLLSMITMSAFLIVRLN
QCMSANEAAITDAAVAVAAASSTHRKVASSTTQYDHKESCNGLYYQGSCYILHSDYQLFS
DAKANCTAESSTLPNKSDVLITWLIDYVEDTWGSDGNPITKTTSDYQDSDVSQEVRKYFC

这是我的Python脚本:

import fileinput

with open('headers.txt', 'w') as hof, \
        open('sequences.txt', 'w') as sof:
    for line in fileinput.input():
        if line[0] == '>':
            hof.write(line)
        else:
            sof.write(line)

和我的Rust脚本(我编译cargo build --release):

use std::io;
use std::fs::File;
use std::io::Write;
use std::io::BufRead;

fn main() {
    let stdin = io::stdin();
    let mut headers = File::create("headers.txt").unwrap();
    let mut sequences = File::create("sequences.txt").unwrap();

    for line in stdin.lock().lines() {
        let line = line.unwrap();
        match &line[..1] {
            ">" => writeln!(headers, "{}", line).unwrap(),
            _ => writeln!(sequences, "{}", line).unwrap(),
        }
    }
}

运行一些基准测试:

Python 2.7

$ time bash -c 'cat uniparc_active-head.fasta | python2 src/main.py'
real    0m11.704s
user    0m6.996s
sys     0m1.100s

Python 3.5

$ time bash -c 'cat uniparc_active-head.fasta | python3 src/main.py'
real    0m16.788s
user    0m12.508s
sys     0m1.576s

PyPy 5.3.1

$ time bash -c 'cat uniparc_active-head.fasta | pypy src/main.py'
real    0m6.526s
user    0m1.536s
sys     0m0.884s

Rust 1.14.0

$ cargo build --release
$ time bash -c 'cat uniparc_active-head.fasta | target/release/parse_text'
real    0m17.493s
user    0m2.728s
sys     0m15.408s

因此,Rust比PyPy慢约3倍,甚至比Python 3慢.

任何人都可以对此有所启发吗？我在Rust代码中犯了错误吗？如果没有,我应该坚持使用Python/PyPy处理文本文件,还是有另一种语言可以更好地完成这项工作？

Answer 1

正如@ BurntSushi5所建议的那样,取而代之

let mut headers = File::create("headers.txt").unwrap();
let mut sequences = File::create("sequences.txt").unwrap();

同

let mut headers = io::BufWriter::new(File::create("headers.txt").unwrap());
let mut sequences = io::BufWriter::new(File::create("sequences.txt").unwrap());

使速度达到我的预期:

$ time bash -c 'cat uniparc_active-head.fasta | target/release/parse_text'
real    0m5.645s
user    0m1.396s
sys     0m0.804s