用Java和C++读取二进制文件

Question

我有一个二进制文件(大约100 MB),我需要快速阅读.在C++中,我可以将文件加载到char指针中,并通过递增指针来遍历它.这当然会非常快.

在Java中有没有相对快速的方法来做到这一点？

Answer 1

如果使用内存映射文件或常规缓冲区,您将能够以硬件允许的速度快速读取数据.

File tmp = File.createTempFile("deleteme", "bin");
tmp.deleteOnExit();
int size = 1024 * 1024 * 1024;

long start0 = System.nanoTime();
FileChannel fc0 = new FileOutputStream(tmp).getChannel();
ByteBuffer bb = ByteBuffer.allocateDirect(32 * 1024).order(ByteOrder.nativeOrder());

for (int i = 0; i < size; i += bb.capacity()) {
    fc0.write(bb);
    bb.clear();
}
long time0 = System.nanoTime() - start0;
System.out.printf("Took %.3f ms to write %,d MB using ByteBuffer%n", time0 / 1e6, size / 1024 / 1024);

long start = System.nanoTime();
FileChannel fc = new FileInputStream(tmp).getChannel();
MappedByteBuffer buffer = fc.map(FileChannel.MapMode.READ_ONLY, 0, size);
LongBuffer longBuffer = buffer.order(ByteOrder.nativeOrder()).asLongBuffer();
long total = 0; // used to prevent a micro-optimisation.
while (longBuffer.remaining() > 0)
    total += longBuffer.get();
fc.close();
long time = System.nanoTime() - start;
System.out.printf("Took %.3f ms to read %,d MB MemoryMappedFile%n", time / 1e6, size / 1024 / 1024);

long start2 = System.nanoTime();
FileChannel fc2 = new FileInputStream(tmp).getChannel();
bb.clear();
while (fc2.read(bb) > 0) {
    while (bb.remaining() > 0)
        total += bb.get();
    bb.clear();
}
fc2.close();
long time2 = System.nanoTime() - start2;
System.out.printf("Took %.3f ms to read %,d MB File via NIO%n", time2 / 1e6, size / 1024 / 1024);

版画

Took 305.243 ms to write 1,024 MB using ByteBuffer
Took 286.404 ms to read 1,024 MB MemoryMappedFile
Took 155.598 ms to read 1,024 MB File via NIO

这是一个比你想要的大10倍的文件.这很快,因为数据被缓存在内存中(我有一个SSD驱动器).如果您拥有快速硬件,则可以非常快速地读取数据.

Answer 2

当然,您可以使用内存映射文件.

以下是两个带示例代码的好链接:

• 用Java思考:内存映射文件
• Java技巧:如何创建内存映射文件

如果你不想走这条路线,只需使用一个普通的InputStream(比如DataInputStream将它包裹在一个BufferedInputStream.