Python比D ??更快 IO操作似乎减慢了D很多......发生了什么？

Question

我为汇编翻译编写了一个虚拟机语言,用于我正在学习的计算机系统课程(使用nand2tetris课程).我最初是用Python编写的,但是因为我正在学习D,我以为我会翻译它.D在语法上与Python非常接近,所以它并不太难.我认为D,作为一种性能语言并且已经编译,至少和Python一样快,在大文件上,会快得多.但事实恰恰相反!尽管算法相同,但当我构造一个非常大的文件进行编译时,D的执行速度仍然比python略慢.在大约500000行的文件上,python一直需要大约2.6秒完成,而D一直占用大约3.这不是一个巨大的差距,但值得注意的是python会更快.

我不想暗示我天真地认为python实际上比D总体更快; 然而,在这种情况下,至少看起来D不是直观地更快.我很感激我的D代码中可能的性能下降的一些输入.我认为瓶颈可能是IO操作,但我不确定.

源代码如下.细节并不重要; 分配了一些汇编语言模板,然后通过虚拟机语言进行线性传递,将每条指令转换为等效的汇编代码块.

编辑:在使用D代码重新编译之后dmd -O -release -inline -m64,D在输入上以2.20秒的时间输出胜利者.然而,问题仍然存在,为什么几乎相同的代码,D似乎比python执行速度慢.

编辑2:使用下面的建议,我从使用简单的字符串列表切换到使用a appender!string(),并将时间改进了显着的数量.但值得一提的是,如果你有一堆字符串appender,请不要使用如下命令将它们写入文件:

auto outputfile = File("foo.txt","w");
foreach(str; my_appender.data)
   outputfile.write(str);

而是写一些类似于:

auto outputfile = File("foo.txt","w");
outputfile.write(my_appender.data);

第二个例子比使用简单的例子提供了小的性能提升string[].但是使用第一个给我带来了巨大的性能损失,使执行时间翻了一番.

更改为a appender!string(),上述大文件的编译花费了大约2.75秒(对于Python的2.8),其中原始版本大约需要3.执行此操作,并使用优化标记dmd,总编译时间为1.98秒!:)

蟒蛇:

#!/usr/bin/python

import sys

operations_dict = {"add":"+", "sub":"-",
                   "and":"&", "or":"|",
                   "not":"!", "neg":"-",
                   "lt":"JLT", "gt":"JGT",
                   "eq":"JEQ", "leq":"JLE",
                   "geq":"JGE"}

vars_dict = {"this":("THIS","M"),
             "that":("THAT","M"),
             "argument":("ARG","M",),
             "local":("LCL","M",),
             "static":("f.%d","M",),
             "temp":("TEMP","A",)}

start = "@SP\nAM=M-1\n"
end = "@SP\nM=M+1\n"

binary_template = start + "D=M\n\
@SP\n\
AM=M-1\n\
M=M%sD\n" + end

unary_template = start + "M=%sM\n" + end

comp_template = start + "D=M\n\
@SP\n\
AM=M-1\n\
D=M-D\n\
@COMP.%d.TRUE\n\
D;%s\n\
@COMP.%d.FALSE\n\
0;JMP\n\
(COMP.%d.TRUE)\n\
@SP\n\
A=M\n\
M=-1\n\
@SP\n\
M=M+1\n\
@COMP.%d.END\n\
0;JMP\n\
(COMP.%d.FALSE)\n\
@SP\n\
A=M\n\
M=0\n" + end + "(COMP.%d.END)\n"

push_tail_template = "@SP\n\
A=M\n\
M=D\n\
@SP\n\
M=M+1\n"

push_const_template = "@%d\nD=A\n" + push_tail_template

push_var_template = "@%d\n\
D=A\n\
@%s\n\
A=%s+D\n\
D=M\n" + push_tail_template

push_staticpointer_template = "@%s\nD=M\n" + push_tail_template

pop_template = "@%d\n\
D=A\n\
@%s\n\
D=%s+D\n\
@R13\n\
M=D\n\
@SP\n\
AM=M-1\n\
D=M\n\
@R13\n\
A=M\n\
M=D\n"

pop_staticpointer_template = "@SP\n\
AM=M-1\n\
D=M\n\
@%s\n\
M=D"


type_dict = {"add":"arithmetic", "sub":"arithmetic",
                   "and":"arithmetic", "or":"arithmetic",
                   "not":"arithmetic", "neg":"arithmetic",
                   "lt":"arithmetic", "gt":"arithmetic",
                   "eq":"arithmetic", "leq":"arithmetic",
                   "geq":"arithmetic", 
                   "push":"memory", "pop":"memory"}

binary_ops = ["add", "sub", "and", "or"]
unary_ops = ["not", "neg"]
comp_ops = ["lt", "gt", "eq", "leq", "geq"]


op_count = 0
line_count = 0
output = ["// Assembly file generated by my awesome VM compiler\n"]

def compile_operation(op):
    global line_count

    if (op[0:2] == "//") or (len(op.split()) == 0):
        return ""

    # print "input: " + op
    operation = op.split()[0]
    header = "// '" + op +  "' (line " + str(line_count) + ")\n"
    line_count += 1

    if type_dict[operation] == "arithmetic":
        return header + compile_arithmetic(op)
    elif type_dict[operation] == "memory":
        return header + compile_memory(op)

def compile_arithmetic(op):
    global op_count
    out_string = ""
    if op in comp_ops:
        out_string += comp_template % (op_count, operations_dict[op], op_count, \
            op_count, op_count, op_count, op_count)
        op_count += 1
    elif op in unary_ops:
        out_string += unary_template % operations_dict[op]
    else:
        out_string += binary_template % operations_dict[op]
    return out_string

def compile_memory(op):
    global output
    instructions = op.split()
    inst = instructions[0]
    argtype = instructions[1]
    val = int(instructions[2])
    if inst == "push":
        if argtype == "constant":
            return push_const_template % val
        elif argtype == "static":
            return push_staticpointer_template % ("f." + str(val))
        elif argtype == "pointer":
            if val == 0:
                return push_staticpointer_template % ("THIS")
            else:
                return push_staticpointer_template % ("THAT")
        else:
            return push_var_template % (val, vars_dict[argtype][0], vars_dict[argtype][1])
    elif inst == "pop":
        if argtype != "constant":
            if argtype == "static":
                return pop_staticpointer_template % ("f." + str(val))
            elif argtype == "pointer":
                if val == 0:
                    return pop_staticpointer_template % "THIS"
                else:
                    return pop_staticpointer_template % "THAT"
            else:
                return pop_template % (val, vars_dict[argtype][0], vars_dict[argtype][1])

def main():
    global output

    if len(sys.argv) == 1:
        inputfname = "test.txt"
    else:
        inputfname = sys.argv[1]
    outputfname = inputfname.split('.')[0] + ".asm"

    inputf = open(inputfname)
    output += ["// Input filename: %s\n" % inputfname]
    for line in inputf.readlines():
        output += [compile_operation(line.strip())]

    outputf = open(outputfname, 'w')
    for outl in output:
        outputf.write(outl)

    outputf.write("(END)\n@END\n0;JMP");
    inputf.close()
    outputf.close()
    print "Output written to " + outputfname


if __name__ == "__main__":
    main()

d:

import std.stdio, std.string, std.conv, std.format, std.c.stdlib;

string[string] operations_dict, type_dict;
string[][string] vars_dict;
string[] arithmetic, memory, comp_ops, unary_ops, binary_ops, lines, output;
string start, end, binary_template, unary_template,
        comp_template, push_tail_template, push_const_template,
        push_var_template, push_staticpointer_template,
        pop_template, pop_staticpointer_template;
int op_count, line_count;

void build_dictionaries() {
    vars_dict = ["this":["THIS","M"],
                 "that":["THAT","M"],
                 "argument":["ARG","M"],
                 "local":["LCL","M"],
                 "static":["f.%d","M"],
                 "temp":["TEMP","A"]];

    operations_dict = ["add":"+", "sub":"-",
                   "and":"&", "or":"|",
                   "not":"!", "neg":"-",
                   "lt":"JLT", "gt":"JGT",
                   "eq":"JEQ", "leq":"JLE",
                   "geq":"JGE"];

    type_dict = ["add":"arithmetic", "sub":"arithmetic",
                   "and":"arithmetic", "or":"arithmetic",
                   "not":"arithmetic", "neg":"arithmetic",
                   "lt":"arithmetic", "gt":"arithmetic",
                   "eq":"arithmetic", "leq":"arithmetic",
                   "geq":"arithmetic", 
                   "push":"memory", "pop":"memory"];

    binary_ops = ["add", "sub", "and", "or"];
    unary_ops = ["not", "neg"];
    comp_ops = ["lt", "gt", "eq", "leq", "geq"];
}

bool is_in(string s, string[] list) {
    foreach (str; list)
        if (str==s) return true;
    return false;
}

void build_strings() {
    start = "@SP\nAM=M-1\n";
    end = "@SP\nM=M+1\n";

    binary_template = start ~ "D=M\n"
    "@SP\n"
    "AM=M-1\n"
    "M=M%sD\n" ~ end;

    unary_template = start ~ "M=%sM\n" ~ end;

    comp_template = start ~ "D=M\n"
    "@SP\n"
    "AM=M-1\n"
    "D=M-D\n"
    "@COMP.%s.TRUE\n"
    "D;%s\n"
    "@COMP.%s.FALSE\n"
    "0;JMP\n"
    "(COMP.%s.TRUE)\n"
    "@SP\n"
    "A=M\n"
    "M=-1\n"
    "@SP\n"
    "M=M+1\n"
    "@COMP.%s.END\n"
    "0;JMP\n"
    "(COMP.%s.FALSE)\n"
    "@SP\n"
    "A=M\n"
    "M=0\n" ~ end ~ "(COMP.%s.END)\n";

    push_tail_template = "@SP\n"
    "A=M\n"
    "M=D\n"
    "@SP\n"
    "M=M+1\n";

    push_const_template = "@%s\nD=A\n" ~ push_tail_template;

    push_var_template = "@%s\n"
    "D=A\n"
    "@%s\n"
    "A=%s+D\n"
    "D=M\n" ~ push_tail_template;

    push_staticpointer_template = "@%s\nD=M\n" ~ push_tail_template;

    pop_template = "@%s\n"
    "D=A\n"
    "@%s\n"
    "D=%s+D\n"
    "@R13\n"
    "M=D\n"
    "@SP\n"
    "AM=M-1\n"
    "D=M\n"
    "@R13\n"
    "A=M\n"
    "M=D\n";

    pop_staticpointer_template = "@SP\n"
    "AM=M-1\n"
    "D=M\n"
    "@%s\n"
    "M=D";
}

void init() {
    op_count = 0;
    line_count = 0;
    output = ["// Assembly file generated by my awesome VM compiler\n"];
    build_strings();
    build_dictionaries();
}

string compile_operation(string op) {
    if (op.length == 0 || op[0..2] == "//")
        return "";
    string operation = op.split()[0];
    string header = "// '" ~ op ~  "' (line " ~ to!string(line_count) ~ ")\n";
    ++line_count;

    if (type_dict[operation] == "arithmetic")
        return header ~ compile_arithmetic(op);
    else
        return header ~ compile_memory(op);
}

string compile_arithmetic(string op) {
    if (is_in(op, comp_ops)) {
        string out_string = format(comp_template, op_count, operations_dict[op], op_count, 
            op_count, op_count, op_count, op_count);
        op_count += 1;
        return out_string;
    } else if (is_in(op, unary_ops))
        return format(unary_template, operations_dict[op]);
    else
        return format(binary_template, operations_dict[op]);
}

string compile_memory(string op) {
    string[] instructions = op.split();
    string inst = instructions[0];
    string argtype = instructions[1];
    int val = to!int(instructions[2]);
    if (inst == "push") {
        if (argtype == "constant") {
            return format(push_const_template, val);
        } else if (argtype == "static")
            return format(push_staticpointer_template, ("f." ~ to!string(val)));
        else if (argtype == "pointer")
            if (val == 0)
                return format(push_staticpointer_template, "THIS");
            else
                return format(push_staticpointer_template, "THAT");
        else
            return format(push_var_template, val, vars_dict[argtype][0], vars_dict[argtype][1]);
    } else {
        if (argtype != "constant") {
            if (argtype == "static")
                return format(pop_staticpointer_template, ("f." ~ to!string(val)));
            else if (argtype == "pointer") {
                if (val == 0)
                    return format(pop_staticpointer_template, "THIS");
                else
                    return format(pop_staticpointer_template, "THAT");
            }
            else
                return format(pop_template, val, vars_dict[argtype][0], vars_dict[argtype][1]);
        } else {
            return "";
        }
    }
}

void main(string args[]) {
    init();
    if (args.length < 2) {
        writefln("usage: %s <filename>", args[0]);
        exit(0);
    }
    string inputfname = args[1];
    string outputfname = args[1].split(".")[0] ~ ".asm";

    auto inputf = File(inputfname, "r");
    output ~= format("// Input filename: %s\n", inputfname);
    foreach (line; inputf.byLine) {
        output ~= compile_operation(to!string(line).strip);
    }
    inputf.close();

    auto outputf = File(outputfname, "w");
    foreach (outl; output)
        outputf.write(outl);

    outputf.write("(END)\n@END\n0;JMP");
    outputf.close();
    writeln("Compilation successful. Output written to " ~ outputfname);
}

Answer 1

对于output变量使用Appender(docs):

import std.array : appender;

void main() {
   auto output = appender!string("// Assembly file generated by my awesome VM compiler\n");
   //...
   output.put(format("// Input filename: %s\n", inputfname));
   foreach (line; inputf.byLine) {
       output.put(compile_operation(line.to!string().strip()));
   }
   //...
   outputf.write(output.data());
   //...
}

此外,我建议你应该改变你type_dict的东西,int[string]并使用整数常量.

int[string] type_dict;

const TYPE_ARITHMETIC = 0,
    TYPE_MEMORY = 1;

//...
type_dict = ["add": TYPE_ARITHMETIC, "push": TYPE_MEMORY]; // etc
//...

//...
if (type_dict[operation] == TYPE_ARITHMETIC) {
    //...
}
//...

使用canFind方法(docs)而不是自定义is_in.甚至尝试SortedRange(docs).