如何提高二进制数据转换的Dart性能？

Question

为大型德国公司Future Technologies Group做一些咨询工作我已经向Dart移植了大约6000行Java服务器端软件.这应该有助于回答Dart是否可以有效地在服务器上使用的问题.(由于搜索了为客户端和服务器端编程使用一种语言的优势,这本身将为Dart开绿灯.)

了解Dart(我非常喜欢与之合作)让我期望相对于Java的性能损失为30-50%但是在任何情况下都不会低于100%(慢两倍),这是所提到的决策过程的截止以上.

港口进展顺利.我学到了很多.单元测试很好.但是性能结果非常糟糕......与Java程序相比总体上要慢七倍.

对代码进行概要分析揭示了两个主要原因:数据转换和文件I/O. 也许我做错了什么？在我回到我的客户并取消他们的Dart研究之前,我想搜索一些关于如何改进的建议.让我们从数据转换开始,将原生Dart数据类型转换为各种二进制格式,这些格式可用于有效传输和存储数据.

通常这些转换很简单且非常快,因为实际上没有任何东西可以从使用的内部格式转换而是主要存储在缓冲区中.我创建了一个基准程序,它以某种方式反映了我的程序中这些转换的典型用法:

import 'dart:typed_data';
import 'package:benchmark_harness/benchmark_harness.dart';

// Create a new benchmark by extending BenchmarkBase
class ConversionBenchmark extends BenchmarkBase {

  Uint8List result;

  ConversionBenchmark() : super("Conversion");

  // The benchmark code.
  void run() {
    const int BufSize = 262144; // 256kBytes
    const int SetSize = 64;     // one "typical" set of data, gets repeated
    ByteData buffer = new ByteData(BufSize);
    double doubleContent = 0.0; // used to simulate double content
    int intContent = 0;         // used to simulate int content
    int offset = 0;
    for (int j = 0; j < buffer.lengthInBytes / SetSize; j++) {
      // The following represents some "typical" conversion mix:
      buffer.setFloat64(offset, doubleContent); offset += 8; doubleContent += 0.123;
      for (int k = 0; k < 8; k++) { // main use case
        buffer.setFloat32(offset, doubleContent); offset += 4; doubleContent += 0.123;
      }
      buffer.setInt32(offset, intContent); offset += 4; intContent++;
      buffer.setInt32(offset, intContent); offset += 4; intContent++;
      buffer.setInt16(offset, intContent); offset += 2; intContent++;
      buffer.setInt16(offset, intContent); offset += 2; intContent++;
      buffer.setInt8(offset, intContent); offset += 1; intContent++;
      buffer.setInt8(offset, intContent); offset += 1; intContent++;
      buffer.buffer.asUint8List(offset).setAll(0, "AsciiStrng".codeUnits); offset += 10;
        // [ByteData] knows no other mechanism to transfer ASCII strings in
      assert((offset % SetSize) == 0); // ensure the example content fits [SetSize] bytes
    }
    result = buffer.buffer.asUint8List(); // only this can be used for further processing
  }
}

main() {
  new ConversionBenchmark().report();
}

它基于来自的基准线束 https://github.com/dart-lang/benchmark_harness.为了进行比较,我使用了以下Java程序,该程序基于来自https://github.com/bono8106/benchmark_harness_java的Dart基准测试线程的端口:

package ylib.tools;

import java.nio.ByteBuffer;

public class ConversionBenchmark extends BenchmarkBase {

  public ByteBuffer result;

  public ConversionBenchmark() { super("Conversion"); }

  // The benchmark code.
  @Override protected void run() {
    final int BufSize = 262144; // 256kBytes
    final int SetSize = 64;     // one "typical" set of data, gets repeated
    ByteBuffer buffer = ByteBuffer.allocate(BufSize);
    double doubleContent = 0.0; // used to simulate double content
    int intContent = 0;         // used to simulate int content
    for (int j = 0; j < (buffer.capacity() / SetSize); j++) {
      // The following represents some "typical" conversion mix:
      buffer.putDouble(doubleContent); doubleContent += 0.123;
      for (int k = 0; k < 8; k++) { // main use case
        buffer.putFloat((float)doubleContent); doubleContent += 0.123;
      }
      buffer.putInt(intContent); intContent++;
      buffer.putInt(intContent); intContent++;
      buffer.putShort((short)intContent); intContent++;
      buffer.putShort((short)intContent); intContent++;
      buffer.put((byte)intContent); intContent++;
      buffer.put((byte)intContent); intContent++;
      buffer.put("AsciiStrng".getBytes());
      //assert((buffer.position() % SetSize) == 0); // ensure the example content fits [SetSize] bytes
    }
    buffer.flip(); // needed for further processing
    result = buffer; // to avoid the compiler optimizing away everything
  }

  public static void main(String[] args) {
    new ConversionBenchmark().report();
  }
}

Java代码的运行速度几乎比英特尔Windows 7机器上的Dart代码快10倍.两者都在各自的VM上以生产模式运行.

代码中是否存在明显错误？或者有不同的Dart课程可以完成这项工作吗？有关为什么Dart在这些简单的转换中速度如此之慢的任何解释？或者我对Dart VM性能有完全错误的期望？

Answer 1

确实,与直接类型阵列访问相比,Dart VM上字节数据方法(ByteData.setXYZ和ByteData.getXYZ)的性能非常糟糕.我们开始研究这个问题,初步结果很有希望[1].

与此同时,您可以使用类型化数组(在[2]处完整代码)将您自己的转换滚动到大端,从而解决这种不幸的性能回归:

/// Writer wraps a fixed size Uint8List and writes values into it using
/// big-endian byte order.
class Writer {
  /// Output buffer.
  final Uint8List out;

  /// Current position within [out].
  var position = 0;

  Writer._create(this.out);

  factory Writer(size) {
    final out = new Uint8List(size);
    if (Endianness.HOST_ENDIAN == Endianness.LITTLE_ENDIAN) {
      return new _WriterForLEHost._create(out);
    } else {
      return new _WriterForBEHost._create(out);
    }
  }

  writeFloat64(double v);

}

/// Lists used for data convertion (alias each other).
final Uint8List _convU8 = new Uint8List(8);
final Float32List _convF32 = new Float32List.view(_convU8.buffer);
final Float64List _convF64 = new Float64List.view(_convU8.buffer);

/// Writer used on little-endian host.
class _WriterForLEHost extends Writer {
  _WriterForLEHost._create(out) : super._create(out);

  writeFloat64(double v) {
    _convF64[0] = v;
    out[position + 7] = _convU8[0];
    out[position + 6] = _convU8[1];
    out[position + 5] = _convU8[2];
    out[position + 4] = _convU8[3];
    out[position + 3] = _convU8[4];
    out[position + 2] = _convU8[5];
    out[position + 1] = _convU8[6];
    out[position + 0] = _convU8[7];
    position += 8;
  }
}

在您的测试中对此手动转换进行基准测试可获得大约6倍的改进:

import 'dart:typed_data';
import 'package:benchmark_harness/benchmark_harness.dart';
import 'writer.dart';

class ConversionBenchmarkManual extends BenchmarkBase {

  Uint8List result;

  ConversionBenchmarkManual() : super("Conversion (MANUAL)");

  // The benchmark code.
  void run() {
    const int BufSize = 262144; // 256kBytes
    const int SetSize = 64;     // one "typical" set of data, gets repeated

    final w = new Writer(BufSize);

    double doubleContent = 0.0; // used to simulate double content
    int intContent = 0;         // used to simulate int content
    int offset = 0;
    for (int j = 0; j < (BufSize / SetSize); j++) {
      // The following represents some "typical" conversion mix:
      w.writeFloat64(doubleContent); doubleContent += 0.123;
      for (int k = 0; k < 8; k++) { // main use case
        w.writeFloat32(doubleContent); doubleContent += 0.123;
      }
      w.writeInt32(intContent); intContent++;
      w.writeInt32(intContent); intContent++;
      w.writeInt16(intContent); intContent++;
      w.writeInt16(intContent); intContent++;
      w.writeInt8(intContent);  intContent++;
      w.writeInt8(intContent);  intContent++;
      w.writeString("AsciiStrng");
      assert((offset % SetSize) == 0); // ensure the example content fits [SetSize] bytes
    }
    result = w.out; // only this can be used for further processing
  }
}

[1] https://code.google.com/p/dart/issues/detail?id=22107

[2] https://gist.github.com/mraleph/4eb5ccbb38904075141e