Java下载海量文件,在一段时间后在互联网网址上显示连接关机/重置

Question

我正在构建一个swing应用程序,通过Internet下载多个文件并保存到windows文件共享.我已经使用了SwingWroker,它在内部使用ExecutorService,它在内部对它们进行排队并一次下载10个,但由于某种原因,下载后说2 - 3 MB的文件它会停止并转移到下一个下载文件,它们将被批量下载10个因为SwingWorker已将其固定为Executor Service的线程数.

我必须在Windows文件共享中写这些文件,我使用nio.FileChannels来做到这一点.有50-60个文件,每个重约300MB - 500MB.文件链接位于我通过登录使用登录页面上的凭据(带有帖子请求)登录的网页,然后我CookieHandler.setDefault(new CookieManager())在开始时指定,因此它对我来说就像浏览器一样.

另一个观察是当我在本地下载它们(而不是Windows服务器共享)时,它们确实可以正常工作.

这是我正在使用的代码

import java.io.File;
import java.io.FileOutputStream;
import java.net.URL;
import java.net.URLConnection;
import java.nio.channels.Channels;
import java.nio.channels.FileChannel;
import java.nio.channels.ReadableByteChannel;

import javax.swing.SwingWorker;

public class DownloadProcess extends SwingWorker<Boolean, String> {

  private String urlPath, filePath;
  public DownloadProcess(String urlPath, String filePath){
    this.urlPath = urlPath;
    this.filePath = filePath;
  }             
  @Override
  protected Boolean doInBackground() {
    boolean taskState = true;
    URLConnection httpConn = null;
    ReadableByteChannel readableByteChannel = null;
    FileOutputStream fileOutputStream = null;
    FileChannel fileOutputChannel = null;
    try{
      //String filePath = "\\\\fileshare.server\\xyz.txt";
      //String urlPath = "http://example.com/anyBigFile.1GB.docx";
      File localFile = new File(filePath);//File share
      boolean itsThere = localFile!=null && localFile.exists();
      long done = itsThere ? localFile.length() : 0;
      URL url = new URL(urlPath);
      httpConn = url.openConnection();
      httpConn.setRequestProperty("Connection", "keep-alive");
      if(itsThere) {
        httpConn.setRequestProperty("Range","bytes="+done+"-");
      }
      readableByteChannel = Channels.newChannel(httpConn.getInputStream());
      fileOutputStream = itsThere ? new FileOutputStream(filePath) : new FileOutputStream(filePath,true);
      fileOutputChannel = fileOutputStream.getChannel();
      for (long position = done, size = httpConn.getContentLength(); position < size && !isCancelled(); ) {
        position += fileOutputChannel.transferFrom(readableByteChannel, position, 1 << 16);
      }
      //done
    }catch(Exception e){
      taskState = false;
      e.printStackTrace();

    }finally{
            //close streams conns etc
    }
    return taskState;
  }

}

这是我在下载5-10分钟后得到的错误堆栈跟踪

/*
  javax.net.ssl.SSLException: Connection has been shutdown: javax.net.ssl.SSLException: java.net.SocketException: Connection reset
  at sun.security.ssl.SSLSocketImpl.checkEOF(Unknown Source)
  at sun.security.ssl.AppInputStream.read(Unknown Source)
  at java.io.BufferedInputStream.read1(Unknown Source)
  at java.io.BufferedInputStream.read(Unknown Source)
  at sun.net.www.MeteredStream.read(Unknown Source)
  at java.io.FilterInputStream.read(Unknown Source)
  at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(Unknown Source)
  at java.nio.channels.Channels$ReadableByteChannelImpl.read(Unknown Source)
  at com.objects.DownloadByteChannel.read(DownloadByteChannel.java:117)
  at sun.nio.ch.FileChannelImpl.transferFromArbitraryChannel(Unknown Source)
  at sun.nio.ch.FileChannelImpl.transferFrom(Unknown Source)
  at com.core.DownloadTask.doInBackground(DownloadTask.java:154)
  at com.core.DownloadTask.doInBackground(DownloadTask.java:59)
  at com.util.ZSwingWorker$1.call(ZSwingWorker.java:286)
  at java.util.concurrent.FutureTask.run(Unknown Source)
  at com.util.ZSwingWorker.run(ZSwingWorker.java:325)
  at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
  at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
  at java.lang.Thread.run(Unknown Source)
Caused by: javax.net.ssl.SSLException: java.net.SocketException: Connection reset
  at sun.security.ssl.Alerts.getSSLException(Unknown Source)
  at sun.security.ssl.SSLSocketImpl.fatal(Unknown Source)
  at sun.security.ssl.SSLSocketImpl.fatal(Unknown Source)
  at sun.security.ssl.SSLSocketImpl.handleException(Unknown Source)
  at sun.security.ssl.SSLSocketImpl.handleException(Unknown Source)
  ... 18 more
Caused by: java.net.SocketException: Connection reset
  at java.net.SocketInputStream.read(Unknown Source)
  at java.net.SocketInputStream.read(Unknown Source)
  at sun.security.ssl.InputRecord.readFully(Unknown Source)
  at sun.security.ssl.InputRecord.read(Unknown Source)
  at sun.security.ssl.SSLSocketImpl.readRecord(Unknown Source)
  at sun.security.ssl.SSLSocketImpl.readDataRecord(Unknown Source)
  ... 18 more
*/

用法:

public static void main(String[] args){
  int counter = 1;
  for(String url: urls){  
    new DownloadProcess(url,"\\\\fileshare.server\\xyz"+(counter++)+".txt").execute();
  }      
}

Answer 1

您将必须更改服务器端的连接超时。我一路上挑选了一些链接（如果它们有任何重要性的话）：

修改会话安全设置

延长销售人员会话超时

希望这有帮助，祝你好运并让我知道:)