相关疑难解决方法(0)

合并大文件而不将整个文件加载到内存中?

我想将包含字符串的大文件合并到一个文件中,并尝试使用nio2.我不想将整个文件加载到内存中,所以我尝试使用BufferedReader:

public void mergeFiles(filesToBeMerged) throws IOException{

Path mergedFile = Paths.get("mergedFile");
Files.createFile(mergedFile);

List<Path> _filesToBeMerged = filesToBeMerged;

try (BufferedWriter writer = Files.newBufferedWriter(mergedFile,StandardOpenOption.APPEND)) {
        for (Path file : _filesToBeMerged) {
// this does not work as write()-method does not accept a BufferedReader
            writer.append(Files.newBufferedReader(file));
        }
    } catch (IOException e) {
        System.err.println(e);
    }

}
Run Code Online (Sandbox Code Playgroud)

我尝试了这个,这个工作,hower,字符串的格式(例如新行等不会被复制到合并文件):

...
try (BufferedWriter writer = Files.newBufferedWriter(mergedFile,StandardOpenOption.APPEND)) {
        for (Path file : _filesToBeMerged) {
//              writer.write(Files.newBufferedReader(file));
            String line = null;


BufferedReader reader = Files.newBufferedReader(file);
            while ((line = reader.readLine()) != null) { …
Run Code Online (Sandbox Code Playgroud)

java java-8 nio2

5
推荐指数
1
解决办法
7462
查看次数

是否可以在 Spring Batch 中跨单个文件进行分区?

我读过有关 spring-batch 中的分区的内容,我找到了一个演示分区的示例。该示例从 CSV 文件中读取人员,进行一些处理并将数据插入数据库。因此,在此示例中,1 个分区 = 1 个文件,因此分区器实现如下所示:

public class MultiResourcePartitioner implements Partitioner {

    private final Logger logger = LoggerFactory.getLogger(MultiResourcePartitioner.class);
    public static final String FILE_PATH = "filePath";

    private static final String PARTITION_KEY = "partition";

    private final Collection<Resource> resources;


    public MultiResourcePartitioner(Collection<Resource> resources) {
        this.resources = resources;
    }

    @Override
    public Map<String, ExecutionContext> partition(int gridSize) {
        Map<String, ExecutionContext> map = new HashMap<>(gridSize);
        int i = 0;
        for (Resource resource : resources) {
            ExecutionContext context = new ExecutionContext();
            context.putString(FILE_PATH, getPath(resource)); //Depends on what …
Run Code Online (Sandbox Code Playgroud)

java spring multithreading partitioning spring-batch

5
推荐指数
1
解决办法
3176
查看次数