用Java读取纯文本文件

Question

似乎有不同的方法来读取和写入Java中的文件数据.

我想从文件中读取ASCII数据.可能的方式和差异是什么？

Answer 1

我最喜欢读取小文件的方法是使用BufferedReader和StringBuilder.它非常简单而且非常简单(虽然不是特别有效,但对大多数情况来说足够好):

BufferedReader br = new BufferedReader(new FileReader("file.txt"));
try {
    StringBuilder sb = new StringBuilder();
    String line = br.readLine();

    while (line != null) {
        sb.append(line);
        sb.append(System.lineSeparator());
        line = br.readLine();
    }
    String everything = sb.toString();
} finally {
    br.close();
}

有人指出,在Java 7之后你应该使用try-with-resources(即自动关闭)功能:

try(BufferedReader br = new BufferedReader(new FileReader("file.txt"))) {
    StringBuilder sb = new StringBuilder();
    String line = br.readLine();

    while (line != null) {
        sb.append(line);
        sb.append(System.lineSeparator());
        line = br.readLine();
    }
    String everything = sb.toString();
}

当我读这样的字符串时,我通常想要每行执行一些字符串处理,所以我就去实现这个.

虽然如果我只想将文件读入String,我总是使用Apache Commons IO和类IOUtils.toString()方法.您可以在这里查看来源:

http://www.docjar.com/html/api/org/apache/commons/io/IOUtils.java.html

FileInputStream inputStream = new FileInputStream("foo.txt");
try {
    String everything = IOUtils.toString(inputStream);
} finally {
    inputStream.close();
}

使用Java 7更简单:

try(FileInputStream inputStream = new FileInputStream("foo.txt")) {     
    String everything = IOUtils.toString(inputStream);
    // do something with everything string
}

Answer 2

ASCII是一个TEXT文件,因此您可以用它Readers来阅读.Java还支持使用二进制文件进行读取InputStreams.如果正在读取的文件很大,那么您可能希望在a BufferedReader之上使用a FileReader来提高读取性能.

阅读本文,了解如何使用Reader

我还建议你下载并阅读这本名为Thinking In Java的精彩(但免费)的书

在Java 7中:

new String(Files.readAllBytes(...))

(docs) 或

Files.readAllLines(...)

(文档)

在Java 8中:

Files.lines(..).forEach(...)

(文档)

Answer 3

最简单的方法是Scanner在Java和FileReader对象中使用该类.简单的例子:

Scanner in = new Scanner(new FileReader("filename.txt"));

Scanner 有几种方法可以读取字符串,数字等...您可以在Java文档页面上查找有关此内容的更多信息.

例如,将整个内容读入String:

StringBuilder sb = new StringBuilder();
while(in.hasNext()) {
    sb.append(in.next());
}
in.close();
outString = sb.toString();

此外,如果您需要特定的编码,您可以使用此代码而不是FileReader:

new InputStreamReader(new FileInputStream(fileUtf8), StandardCharsets.UTF_8)

Answer 4

这是一个简单的解决方案:

String content;

content = new String(Files.readAllBytes(Paths.get("sample.txt")));

Answer 5

这是另一种不使用外部库的方法:

import java.io.File;
import java.io.FileReader;
import java.io.IOException;

public String readFile(String filename)
{
    String content = null;
    File file = new File(filename); // For example, foo.txt
    FileReader reader = null;
    try {
        reader = new FileReader(file);
        char[] chars = new char[(int) file.length()];
        reader.read(chars);
        content = new String(chars);
        reader.close();
    } catch (IOException e) {
        e.printStackTrace();
    } finally {
        if(reader != null){
            reader.close();
        }
    }
    return content;
}

Answer 6

我必须以不同方式进行基准测试.我将评论我的发现,但简而言之,最快的方法是在FileInputStream上使用普通的旧BufferedInputStream.如果必须读取许多文件,则三个线程将总执行时间减少到大约一半,但添加更多线程将逐渐降低性能,直到完成二十个线程所需的时间比仅使用一个线程要长三倍.

假设您必须读取文件并对其内容执行有意义的操作.在这些示例中,这里是从日志中读取行并计算包含超过特定阈值的值的行.所以我假设单行Java 8 Files.lines(Paths.get("/path/to/file.txt")).map(line -> line.split(";"))不是一个选项.

我测试了Java 1.8,Windows 7以及SSD和HDD驱动器.

我写了六个不同的实现:

rawParse:在FileInputStream上使用BufferedInputStream,然后逐行读取行.这比任何其他单线程方法都要好,但对于非ASCII文件来说可能非常不方便.

lineReaderParse:在FileReader上使用BufferedReader,逐行读取,通过调用String.split()拆分行.这比rawParse慢大约20%.

lineReaderParseParallel:这与lineReaderParse相同,但它使用多个线程.在所有情况下,这是最快的选择.

nioFilesParse:使用java.nio.files.Files.lines()

nioAsyncParse:使用带有完成处理程序和线程池的AsynchronousFileChannel.

nioMemoryMappedParse:使用内存映射文件.这是一个糟糕的主意,执行时间至少比任何其他实现长三倍.

这些是在四核i7和SSD驱动器上读取每个4 MB的204个文件的平均时间.这些文件是动态生成的,以避免磁盘缓存.

rawParse                11.10 sec
lineReaderParse         13.86 sec
lineReaderParseParallel  6.00 sec
nioFilesParse           13.52 sec
nioAsyncParse           16.06 sec
nioMemoryMappedParse    37.68 sec

我发现在SSD上运行或SSD驱动器之间的差异比我预期的差异大约快15%.这可能是因为文件是在未分段的HDD上生成的并且它们是顺序读取的,因此旋转驱动器几乎可以作为SSD执行.

我对nioAsyncParse实现的低性能感到惊讶.要么我以错误的方式实现了某些东西,要么使用NIO实现多线程实现,并且完成处理程序执行与使用java.io API的单线程实现相同(甚至更糟).此外,具有CompletionHandler的异步解析在代码行中要长得多,并且比在旧流上的直接实现更难以正确实现.

现在六个实现后跟一个包含它们的类,加上一个可参数化的main()方法,允许播放文件数,文件大小和并发度.请注意,文件的大小变化加上减去20%.这是为了避免由于所有文件大小完全相同而产生的任何影响.

rawParse

public void rawParse(final String targetDir, final int numberOfFiles) throws IOException, ParseException {
    overrunCount = 0;
    final int dl = (int) ';';
    StringBuffer lineBuffer = new StringBuffer(1024);
    for (int f=0; f<numberOfFiles; f++) {
        File fl = new File(targetDir+filenamePreffix+String.valueOf(f)+".txt");
        FileInputStream fin = new FileInputStream(fl);
        BufferedInputStream bin = new BufferedInputStream(fin);
        int character;
        while((character=bin.read())!=-1) {
            if (character==dl) {

                // Here is where something is done with each line
                doSomethingWithRawLine(lineBuffer.toString());
                lineBuffer.setLength(0);
            }
            else {
                lineBuffer.append((char) character);
            }
        }
        bin.close();
        fin.close();
    }
}

public final void doSomethingWithRawLine(String line) throws ParseException {
    // What to do for each line
    int fieldNumber = 0;
    final int len = line.length();
    StringBuffer fieldBuffer = new StringBuffer(256);
    for (int charPos=0; charPos<len; charPos++) {
        char c = line.charAt(charPos);
        if (c==DL0) {
            String fieldValue = fieldBuffer.toString();
            if (fieldValue.length()>0) {
                switch (fieldNumber) {
                    case 0:
                        Date dt = fmt.parse(fieldValue);
                        fieldNumber++;
                        break;
                    case 1:
                        double d = Double.parseDouble(fieldValue);
                        fieldNumber++;
                        break;
                    case 2:
                        int t = Integer.parseInt(fieldValue);
                        fieldNumber++;
                        break;
                    case 3:
                        if (fieldValue.equals("overrun"))
                            overrunCount++;
                        break;
                }
            }
            fieldBuffer.setLength(0);
        }
        else {
            fieldBuffer.append(c);
        }
    }
}

lineReaderParse

public void lineReaderParse(final String targetDir, final int numberOfFiles) throws IOException, ParseException {
    String line;
    for (int f=0; f<numberOfFiles; f++) {
        File fl = new File(targetDir+filenamePreffix+String.valueOf(f)+".txt");
        FileReader frd = new FileReader(fl);
        BufferedReader brd = new BufferedReader(frd);

        while ((line=brd.readLine())!=null)
            doSomethingWithLine(line);
        brd.close();
        frd.close();
    }
}

public final void doSomethingWithLine(String line) throws ParseException {
    // Example of what to do for each line
    String[] fields = line.split(";");
    Date dt = fmt.parse(fields[0]);
    double d = Double.parseDouble(fields[1]);
    int t = Integer.parseInt(fields[2]);
    if (fields[3].equals("overrun"))
        overrunCount++;
}

lineReaderParseParallel

public void lineReaderParseParallel(final String targetDir, final int numberOfFiles, final int degreeOfParalelism) throws IOException, ParseException, InterruptedException {
    Thread[] pool = new Thread[degreeOfParalelism];
    int batchSize = numberOfFiles / degreeOfParalelism;
    for (int b=0; b<degreeOfParalelism; b++) {
        pool[b] = new LineReaderParseThread(targetDir, b*batchSize, b*batchSize+b*batchSize);
        pool[b].start();
    }
    for (int b=0; b<degreeOfParalelism; b++)
        pool[b].join();
}

class LineReaderParseThread extends Thread {

    private String targetDir;
    private int fileFrom;
    private int fileTo;
    private DateFormat fmt = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
    private int overrunCounter = 0;

    public LineReaderParseThread(String targetDir, int fileFrom, int fileTo) {
        this.targetDir = targetDir;
        this.fileFrom = fileFrom;
        this.fileTo = fileTo;
    }

    private void doSomethingWithTheLine(String line) throws ParseException {
        String[] fields = line.split(DL);
        Date dt = fmt.parse(fields[0]);
        double d = Double.parseDouble(fields[1]);
        int t = Integer.parseInt(fields[2]);
        if (fields[3].equals("overrun"))
            overrunCounter++;
    }

    @Override
    public void run() {
        String line;
        for (int f=fileFrom; f<fileTo; f++) {
            File fl = new File(targetDir+filenamePreffix+String.valueOf(f)+".txt");
            try {
            FileReader frd = new FileReader(fl);
            BufferedReader brd = new BufferedReader(frd);
            while ((line=brd.readLine())!=null) {
                doSomethingWithTheLine(line);
            }
            brd.close();
            frd.close();
            } catch (IOException | ParseException ioe) { }
        }
    }
}

nioFilesParse

public void nioFilesParse(final String targetDir, final int numberOfFiles) throws IOException, ParseException {
    for (int f=0; f<numberOfFiles; f++) {
        Path ph = Paths.get(targetDir+filenamePreffix+String.valueOf(f)+".txt");
        Consumer<String> action = new LineConsumer();
        Stream<String> lines = Files.lines(ph);
        lines.forEach(action);
        lines.close();
    }
}


class LineConsumer implements Consumer<String> {

    @Override
    public void accept(String line) {

        // What to do for each line
        String[] fields = line.split(DL);
        if (fields.length>1) {
            try {
                Date dt = fmt.parse(fields[0]);
            }
            catch (ParseException e) {
            }
            double d = Double.parseDouble(fields[1]);
            int t = Integer.parseInt(fields[2]);
            if (fields[3].equals("overrun"))
                overrunCount++;
        }
    }
}

nioAsyncParse

public void nioAsyncParse(final String targetDir, final int numberOfFiles, final int numberOfThreads, final int bufferSize) throws IOException, ParseException, InterruptedException {
    ScheduledThreadPoolExecutor pool = new ScheduledThreadPoolExecutor(numberOfThreads);
    ConcurrentLinkedQueue<ByteBuffer> byteBuffers = new ConcurrentLinkedQueue<ByteBuffer>();

    for (int b=0; b<numberOfThreads; b++)
        byteBuffers.add(ByteBuffer.allocate(bufferSize));

    for (int f=0; f<numberOfFiles; f++) {
        consumerThreads.acquire();
        String fileName = targetDir+filenamePreffix+String.valueOf(f)+".txt";
        AsynchronousFileChannel channel = AsynchronousFileChannel.open(Paths.get(fileName), EnumSet.of(StandardOpenOption.READ), pool);
        BufferConsumer consumer = new BufferConsumer(byteBuffers, fileName, bufferSize);
        channel.read(consumer.buffer(), 0l, channel, consumer);
    }
    consumerThreads.acquire(numberOfThreads);
}


class BufferConsumer implements CompletionHandler<Integer, AsynchronousFileChannel> {

        private ConcurrentLinkedQueue<ByteBuffer> buffers;
        private ByteBuffer bytes;
        private String file;
        private StringBuffer chars;
        private int limit;
        private long position;
        private DateFormat frmt = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");

        public BufferConsumer(ConcurrentLinkedQueue<ByteBuffer> byteBuffers, String fileName, int bufferSize) {
            buffers = byteBuffers;
            bytes = buffers.poll();
            if (bytes==null)
                bytes = ByteBuffer.allocate(bufferSize);

            file = fileName;
            chars = new StringBuffer(bufferSize);
            frmt = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
            limit = bufferSize;
            position = 0l;
        }

        public ByteBuffer buffer() {
            return bytes;
        }

        @Override
        public synchronized void completed(Integer result, AsynchronousFileChannel channel) {

            if (result!=-1) {
                bytes.flip();
                final int len = bytes.limit();
                int i = 0;
                try {
                    for (i = 0; i < len; i++) {
                        byte by = bytes.get();
                        if (by=='\n') {
                            // ***
                            // The code used to process the line goes here
                            chars.setLength(0);
                        }
                        else {
                                chars.append((char) by);
                        }
                    }
                }
                catch (Exception x) {
                    System.out.println(
                        "Caught exception " + x.getClass().getName() + " " + x.getMessage() +
                        " i=" + String.valueOf(i) + ", limit=" + String.valueOf(len) +
                        ", position="+String.valueOf(position));
                }

                if (len==limit) {
                    bytes.clear();
                    position += len;
                    channel.read(bytes, position, channel, this);
                }
                else {
                    try {
                        channel.close();
                    }
                    catch (IOException e) {
                    }
                    consumerThreads.release();
                    bytes.clear();
                    buffers.add(bytes);
                }
            }
            else {
                try {
                    channel.close();
                }
                catch (IOException e) {
                }
                consumerThreads.release();
                bytes.clear();
                buffers.add(bytes);
            }
        }

        @Override
        public void failed(Throwable e, AsynchronousFileChannel channel) {
        }
};

完全可以实现所有案例

https://github.com/sergiomt/javaiobenchmark/blob/master/FileReadBenchmark.java

Answer 7

以下是三种工作和测试方法:

运用 BufferedReader

package io;
import java.io.*;
public class ReadFromFile2 {
    public static void main(String[] args)throws Exception {
        File file = new File("C:\\Users\\pankaj\\Desktop\\test.java");
        BufferedReader br = new BufferedReader(new FileReader(file));
        String st;
        while((st=br.readLine()) != null){
            System.out.println(st);
        }
    }
}

运用 Scanner

package io;

import java.io.File;
import java.util.Scanner;

public class ReadFromFileUsingScanner {
    public static void main(String[] args) throws Exception {
        File file = new File("C:\\Users\\pankaj\\Desktop\\test.java");
        Scanner sc = new Scanner(file);
        while(sc.hasNextLine()){
            System.out.println(sc.nextLine());
        }
    }
}

运用 FileReader

package io;
import java.io.*;
public class ReadingFromFile {

    public static void main(String[] args) throws Exception {
        FileReader fr = new FileReader("C:\\Users\\pankaj\\Desktop\\test.java");
        int i;
        while ((i=fr.read()) != -1){
            System.out.print((char) i);
        }
    }
}

使用Scanner该类在没有循环的情况下读取整个文件

package io;

import java.io.File;
import java.io.FileNotFoundException;
import java.util.Scanner;

public class ReadingEntireFileWithoutLoop {

    public static void main(String[] args) throws FileNotFoundException {
        File file = new File("C:\\Users\\pankaj\\Desktop\\test.java");
        Scanner sc = new Scanner(file);
        sc.useDelimiter("\\Z");
        System.out.println(sc.next());
    }
}

Answer 8

其中的方法org.apache.commons.io.FileUtils也可能非常方便,例如:

/**
 * Reads the contents of a file line by line to a List
 * of Strings using the default encoding for the VM.
 */
static List readLines(File file)

Answer 9

你想用文字做什么？文件是否足够小以适应内存？我会尝试找到最简单的方法来处理文件以满足您的需求.FileUtils库非常适合这种情况.

for(String line: FileUtils.readLines("my-text-file"))
    System.out.println(line);

Answer 10

我记录了15种用Java读取文件的方法,然后使用各种文件大小测试它们的速度 - 从1 KB到1 GB,以下是执行此操作的前三种方法:

1. java.nio.file.Files.readAllBytes()

   经测试可在Java 7,8和9中使用.

   import java.io.File;
   import java.io.IOException;
   import java.nio.file.Files;
   
   public class ReadFile_Files_ReadAllBytes {
     public static void main(String [] pArgs) throws IOException {
       String fileName = "c:\\temp\\sample-10KB.txt";
       File file = new File(fileName);
   
       byte [] fileBytes = Files.readAllBytes(file.toPath());
       char singleChar;
       for(byte b : fileBytes) {
         singleChar = (char) b;
         System.out.print(singleChar);
       }
     }
   }
   

2. java.io.BufferedReader.readLine()

   经测试可在Java 7,8,9中使用.

   import java.io.BufferedReader;
   import java.io.FileReader;
   import java.io.IOException;
   
   public class ReadFile_BufferedReader_ReadLine {
     public static void main(String [] args) throws IOException {
       String fileName = "c:\\temp\\sample-10KB.txt";
       FileReader fileReader = new FileReader(fileName);
   
       try (BufferedReader bufferedReader = new BufferedReader(fileReader)) {
         String line;
         while((line = bufferedReader.readLine()) != null) {
           System.out.println(line);
         }
       }
     }
   }
   

3. java.nio.file.Files.lines()

   这被测试在Java 8和9中工作,但由于lambda表达式要求而无法在Java 7中工作.

   import java.io.File;
   import java.io.IOException;
   import java.nio.file.Files;
   import java.util.stream.Stream;
   
   public class ReadFile_Files_Lines {
     public static void main(String[] pArgs) throws IOException {
       String fileName = "c:\\temp\\sample-10KB.txt";
       File file = new File(fileName);
   
       try (Stream linesStream = Files.lines(file.toPath())) {
         linesStream.forEach(line -> {
           System.out.println(line);
         });
       }
     }
   }
   

Answer 11

下面是以Java 8方式进行的单行程序.假设text.txt文件位于Eclipse项目目录的根目录中.

Files.lines(Paths.get("text.txt")).collect(Collectors.toList());

Answer 12

Java 11中引入了最直观的方法Files.readString

\n

import java.io.*;\nimport java.nio.file.Files;\nimport java.nio.file.Paths;\n\npublic class App {\n    public static void main(String args[]) throws IOException {\n        String content = Files.readString(Paths.get("D:\\\\sandbox\\\\mvn\\\\my-app\\\\my-app.iml"));\n        System.out.print(content);\n    }\n}\n

\n

PHP 几十年来一直享有这种奢侈！\xe2\x98\xba

\n

Answer 13

使用BufferedReader:

import java.io.BufferedReader;
import java.io.FileNotFoundException;
import java.io.FileReader;
import java.io.IOException;

BufferedReader br;
try {
    br = new BufferedReader(new FileReader("/fileToRead.txt"));
    try {
        String x;
        while ( (x = br.readLine()) != null ) {
            // Printing out each line in the file
            System.out.println(x);
        }
    }
    catch (IOException e) {
        e.printStackTrace();
    }
}
catch (FileNotFoundException e) {
    System.out.println(e);
    e.printStackTrace();
}

Answer 14

这基本上与Jesus Ramos的答案完全相同,除了使用File而不是FileReader加迭代来逐步浏览文件的内容.

Scanner in = new Scanner(new File("filename.txt"));

while (in.hasNext()) { // Iterates each line in the file
    String line = in.nextLine();
    // Do something with line
}

in.close(); // Don't forget to close resource leaks

...投掷 FileNotFoundException

Answer 15

可能没有缓冲I/O那么快,但相当简洁:

    String content;
    try (Scanner scanner = new Scanner(textFile).useDelimiter("\\Z")) {
        content = scanner.next();
    }

该\Z模式告诉Scanner分隔符是EOF.

Answer 16

实际上，缓冲流类的性能要好得多，以至于NIO.2 API包含专门返回这些流类的方法，部分是为了鼓励您始终在应用程序中使用缓冲流。

这是一个例子：

Path path = Paths.get("/myfolder/myfile.ext");
try (BufferedReader reader = Files.newBufferedReader(path)) {
    // Read from the stream
    String currentLine = null;
    while ((currentLine = reader.readLine()) != null)
        //do your code here
} catch (IOException e) {
    // Handle file I/O exception...
}

您可以替换此代码

BufferedReader reader = Files.newBufferedReader(path);

与

BufferedReader br = new BufferedReader(new FileReader("/myfolder/myfile.ext"));

我推荐这篇文章，以学习Java NIO和IO的主要用法。