在Java中并行或异步下载多个文件

Question

在这里，我试图一个接一个地下载多个文件：

环境-Java 1.6

public List<Attachment> download(List<Attachment> attachments)
{
  for(Attachment attachment : attachments) {
    attachment.setDownStatus("Failed");
    String destLocation = "C:\Users\attachments";
    try {
        String attUrl = attachment.getUrl();
        String fileName = attachment.getFileName();            
        URL url = new URL(attUrl);
        File fileLocation = new File(destLoc, fileName);
        FileUtils.copyURLToFile(url, fileLocation);
        if(fileLocation.exists()) {
           attachment.setDownStatus("Completed");
         }
       } catch(Exception e) {
          attachment.setDownStatus("Failed");
       } finally {
          attachment.setDestLocation(destLocation);
       }
   }
  return attachments;
}

我正在从提供的URL（http://cdn.octafinance.com/wp-content/uploads/2015/07/google-hummingbird.jpg）下载文件。

FileUtils.copyURLToFile(url, fileLocation);

上面的代码完美地完成了下载工作，没有任何问题。

我的问题：
如果附件列表更多，则将花费更多时间，因此我想使其成为异步或并行过程，而不是顺序下载。

Answer 1

将Java 8 Streams与ForkJoinPool结合使用

public List<Attachment> download(List<Attachment> attachments) throws InterruptedException, ExecutionException {

    ForkJoinPool forkJoinPool = new ForkJoinPool(attachments.size());

    return forkJoinPool.submit(() -> processAttachments(attachments)).get();
}

private List<Attachment> processAttachments(List<Attachment> attachments) {
    return attachments.stream().parallel().map(attachment -> processSingleAttachment(attachment)).collect(Collectors.toList());
}

private Attachment processSingleAttachment(Attachment attachment){
     //business logic to download single attachment
    .
    .
}

Answer 2

其实仔细一看，Boris的代码是有问题的，有时确实不会设置一些东西。这是一个更好的版本来解决这个问题：

public List<Attachment> download(List<Attachment> attachments) {
  ExecutorService executorService = Executors.newCachedThreadPool();
  List<Future<Attachment>> futures = new ArrayList<Future<Attachment>>();
  for (final Attachment attachment : attachments) {
    futures.add(executorService.submit(new Callable<Attachment>() {
      @Override
      public Attachment call() throws Exception {
        return doDownload(attachment);
      }
    }));
  }
  for (Future<Attachment> future: futures) {
    try {
      future.get();
    } catch (Exception ex) {
      // Do something
    }
  }
  return attachments;
}

private Attachment doDownload(Attachment attachment) throws Exception {
  attachment.setDownStatus("Failed");
  attachment.setDestLocation("C:\\Users\\attachments");
  String attUrl = attachment.getUrl();
  String fileName = attachment.getFileName();
  URL url = new URL(attUrl);
  File fileLocation = new File(attachment.getDestLocation(), fileName);
  FileUtils.copyURLToFile(url, fileLocation);
  if (fileLocation.exists()) {
    attachment.setDownStatus("Completed");
  }
  return attachment;
}

然而，考虑到您的结构Attachment和使用方式，这绝对不是最佳的。我没有解决这个问题：我只是回答了问题。