如何在Java中快速编写大型二进制数据?

Dov*_*Dov 4 java nio binary-data

我正在写一个STL文件,它由一个80字节的头,一个4字节的整数组成,然后是50个字节的记录,每个字节由浮点数和一个短整数组成.

使用RandomAccessFile我可以轻松地写入数据,但速度非常慢.它使用与DataOutputStream相同的接口.如果有一种简单的方法来缓冲数据输出流,我可以使用它,但烦人的部分是需要写出所有记录,并在最后计算三角形输出的数量,并将该整数写入字节81- 84.

直截了当但缓慢的方式,只关注大部分工作,即写下每个方面:

public static void writeBinarySTL(Triangle t, RandomAccessFile d) throws IOException {
    d.writeFloat((float)t.normal.x);
    d.writeFloat((float)t.normal.y);
    d.writeFloat((float)t.normal.z);
    d.writeFloat((float)t.p1.x);
    d.writeFloat((float)t.p1.y);
    d.writeFloat((float)t.p1.z);
    d.writeFloat((float)t.p2.x);
    d.writeFloat((float)t.p2.y);
    d.writeFloat((float)t.p2.z);
    d.writeFloat((float)t.p3.x);
    d.writeFloat((float)t.p3.y);
    d.writeFloat((float)t.p3.z);
    d.writeShort(0);
}
Run Code Online (Sandbox Code Playgroud)

是否有任何优雅的方法将这种二进制数据写入块状快速I/O类?

在我看来,STL文件格式应该首先是低字节,Java可能是高字节优先.所以也许我的所有writeFloats都是徒劳的,我必须找到一个手动转换,以便它以小端形式出现?

如果必须,我愿意关闭文件,最后在randomaccessfile中重新打开,寻找字节81并写入计数.

因此,此编辑是对应该起作用的问题的两个答案的回应.第一个是添加BufferedWriter.结果非常快.我知道这台笔记本电脑是高端的SSD,但我并不期待这种性能,更不用说来自Java了.只需缓冲输出,在0.5秒内写入96Mb文件,在1.5秒内写入196Mb.

为了了解nio是否会提供更高的性能,我试图通过@ sturcotte06实现解决方案.代码不会尝试写标题,我只关注每个三角形的50字节记录.

public static void writeBinarySTL2(Shape3d s, String filename) {
    java.nio.file.Path filePath = Paths.get(filename);

    // Open a channel in write mode on your file.
    try (WritableByteChannel channel = Files.newByteChannel(filePath, StandardOpenOption.CREATE)) {
        // Allocate a new buffer.
        ByteBuffer buf = ByteBuffer.allocate(50 * 1024);
        ArrayList<Triangle> triangles = s.triangles;
        // Write your triangle data to the buffer.
        for (int i = 0; i < triangles.size(); i += 1024) {
            for (int j = i; j < i + 1024; ++j)
                writeBinarySTL(triangles.get(j), buf);
            buf.flip(); // stop modifying buffer so it can be written to disk
            channel.write(buf);  // Write your buffer's data.
        }
        channel.close();
    } catch (IOException e) {
        e.printStackTrace();
    }
}
Run Code Online (Sandbox Code Playgroud)

我尝试了WRITE(文档中说需要现有文件)和CREATE文档声称要写入现有文件或创建新文件.

这两个选项都无法创建Sphere902_bin2.stl文件

java.nio.file.NoSuchFileException: Sphere902_bin2.stl
sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:79)
        at 
sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
        at 
sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)
        at 
Run Code Online (Sandbox Code Playgroud)

sun.nio.fs.WindowsFileSystemProvider.newByteChannel(WindowsFileSystemProvider.java:230)at java.nio.file.Files.newByteChannel(Files.java:361)at java.nio.file.Files.newByteChannel(Files.java:407)在edu.stevens.scad.STL.writeBinarySTL2(STL.java:105)

我不相信写入字节的代码是相关的,但这是我提出的:

public static void writeBinarySTL(Triangle t, ByteBuffer buf) {
    buf.putInt(Integer.reverseBytes(Float.floatToIntBits((float)t.normal.x)));
    buf.putInt(Integer.reverseBytes(Float.floatToIntBits((float)t.normal.y)));
    buf.putInt(Integer.reverseBytes(Float.floatToIntBits((float)t.normal.y)));
    buf.putInt(Integer.reverseBytes(Float.floatToIntBits((float)t.p1.x)));
    buf.putInt(Integer.reverseBytes(Float.floatToIntBits((float)t.p1.y)));
    buf.putInt(Integer.reverseBytes(Float.floatToIntBits((float)t.p1.z)));
    buf.putInt(Integer.reverseBytes(Float.floatToIntBits((float)t.p2.x)));
    buf.putInt(Integer.reverseBytes(Float.floatToIntBits((float)t.p2.y)));
    buf.putInt(Integer.reverseBytes(Float.floatToIntBits((float)t.p2.z)));
    buf.putInt(Integer.reverseBytes(Float.floatToIntBits((float)t.p3.x)));
    buf.putInt(Integer.reverseBytes(Float.floatToIntBits((float)t.p3.y)));
    buf.putInt(Integer.reverseBytes(Float.floatToIntBits((float)t.p3.z)));
    buf.putShort((short)0);
}
Run Code Online (Sandbox Code Playgroud)

这是一个MWE,显示在写入超出缓冲区大小时代码不起作用:

package language;
import java.io.*;
import java.nio.*;
import java.nio.file.*;
import java.nio.channels.*;

public class FastWritenio {
    public static void writeUsingPrintWriter() throws IOException {
        PrintWriter pw = new PrintWriter(new FileWriter("test.txt"));
        pw.print("testing");
        pw.close();
    }
    public static void writeUsingnio(int numTrials, int bufferSize, int putsPer) throws IOException {
        String filename = "d:/test.dat";
        java.nio.file.Path filePath = Paths.get(filename);
        WritableByteChannel channel = Files.newByteChannel(filePath, StandardOpenOption.CREATE, StandardOpenOption.WRITE, StandardOpenOption.READ);
        ByteBuffer buf = ByteBuffer.allocate(bufferSize);
        for (int t = 0; t < numTrials; ++t) {
            for (int i = 0; i < putsPer; i ++) {
                buf.putInt(i);
            }
            buf.flip(); // stop modifying buffer so it can be written to disk
            channel.write(buf);  // Write your buffer's data.
            // Without this line, it crashes:  buf.flip();
            // but with it this code is very slow.
        }
        channel.close();
    }
    public static void main(String[] args) throws IOException {
        writeUsingPrintWriter();
        long t0 = System.nanoTime();
        writeUsingnio(1024*256, 8*1024, 2048);
        System.out.println((System.nanoTime()-t0)*1e-9);

    }
}
Run Code Online (Sandbox Code Playgroud)

stu*_*e06 6

使用nio的ByteBuffer类:

public static void writeBinarySTL(Triangle t, ByteBuffer buf) {
    buf.putFloat((float)t.normal.x);
    // ...
}

// Create a new path to your file on the default file system.
Path filePath = Paths.get("file.txt");

// Open a channel in write mode on your file.
try (WritableByteChannel channel = Files.newByteChannel(filePath, StandardOpenOption.WRITE)) {
    // Allocate a new buffer.
    ByteBuffer buf = ByteBuffer.allocate(8192);

    // Write your triangle data to the buffer.
    writeBinarySTL(t, buf);

    // Flip from write mode to read mode.
    buf.flip();

    // Write your buffer's data.
    channel.write(buf);
}
Run Code Online (Sandbox Code Playgroud)

ByteBuffer教程

ByteBuffer javadoc

WriteableByteChannel javadoc

文件javadoc

编辑:

public static boolean tryWriteBinarySTL(Triangle t, ByteBuffer buf) {
    final int triangleBytes = 50; // set this.
    if (buf.remaining() < triangleBytes) {
       return false;
    }

    buf.putFloat((float)t.normal.x);
    // ...
    return true;
}

// Create a new path to your file on the default file system.
Path filePath = Paths.get("file.txt");

// Open a channel in write mode on your file.
try (WritableByteChannel channel = Files.newByteChannel(filePath, StandardOpenOption.WRITE)) {
    // Allocate a new buffer.
    ByteBuffer buf = ByteBuffer.allocate(8192);

    // Write your triangle data to the buffer.
    for (Triangle triangle : triangles) {
        while (!tryWriteBinarySTL(triangle, buf) ) {
            // Flush buffer.
            buf.flip();
            while (buf.hasRemaining()) {
                channel.write(buf);
            }

            buf.flip();
        }
    }

    // Write remaining.
    buf.flip();
    channel.write(buf);
}
Run Code Online (Sandbox Code Playgroud)


use*_*421 5

如果有一种简单的缓冲方式

有:

DataOutputStream dos = new DataOutputStream(new BufferedOutputStream(...));
Run Code Online (Sandbox Code Playgroud)