如何恢复中断下载 - 第2部分

gre*_*egS 21 java android http download

这是我之前的问题的延续,当我不是注册用户时,我发布了这个问题.作为复习,我正在尝试从我的Yahoo!继续下载大文件.下载中断时的网站服务器.我以前认为中断是由于100秒超时限制(因为Yahoo!对用户编写的脚本强制执行该时间限制).但是,当我测量下载中断的时间时,我发现中断时序变化很大(有时下载不间断运行不到100秒,有时甚至长达7分钟).所以我不知道超时的原因,我只是想解决它们.

我尝试了naikus的建议(谢谢),根据http标头字段的转储,看来我的Yahoo! 网站服务器确实识别"范围"属性,该属性应允许下载在中断的偏移处恢复.遗憾的是,虽然恢复连接中的http标头中的字节范围显示正确,但传输的内容始终在文件的开头重新开始.(我的测试文件是一个由50,000个4字节整数组成的数组,从0开始递增.我的下载文件总是在每次发生下载中断的偏移处开始重新计数.)

雅虎还有一些其他的HTTP连接属性请求我应该去做!服务器实际上跳到标头字节范围中指定的文件偏移量? 这是代码及其转储的内容:

         // Setup connection.
         URL url = new URL(strUrl[0]);
         URLConnection connection = url.openConnection();
         downloaded = Integer.parseInt(strUrl[3]);
         if (downloaded > 0) {
             connection.setRequestProperty("Range", "bytes="+downloaded+"-");
             connection.connect();
             fileLength = mDownloadFileLength;
             Log.d("AsyncDownloadFile", 
                 "new download seek: " + downloaded +
                 "; lengthFile: " + fileLength);
         }
         else {
             connection.connect();
             downloaded = 0;
             fileLength = connection.getContentLength();
             mDownloadFileLength = fileLength;
         }
         Map<String, List<String>> map = connection.getHeaderFields();
         Log.d("AsyncDownloadFile", "header fields: " + map.toString());

         // Setup streams and buffers.
         input = new BufferedInputStream(url.openStream(), 8192);
         outFile = new RandomAccessFile(strUrl[1], "rw");
         if (downloaded > 0)  
             outFile.seek(downloaded);
         byte data[] = new byte[1024];

         // Download file.
         for (int count=0, i=0; (count=input.read(data, 0, 1024)) != -1; i++) { 
             outFile.write(data, 0, count);
             downloaded += count; 
             if (downloaded >= fileLength)
                 break;

             // Display progress.
             Log.d("AsyncDownloadFile", "bytes: " + downloaded);
             if ((i%10) == 0)
                 publishProgress((int)(downloaded*100/fileLength));
             if (mFlagDisableAsyncTask) {
                 downloaded = 0;
                 break;
             }
         }

         // Close streams.
         outFile.close();
         input.close();
Run Code Online (Sandbox Code Playgroud)

倾倒:

@ 4:08:24  
D/AsyncDownloadFile( 2372): header fields: {p3p=[policyref="http://info.yahoo.co
m/w3c/p3p.xml", CP="CAO DSP COR CUR ADM DEV TAI PSA PSD IVAi IVDi CONi TELo OTPi
 OUR DELi SAMi OTRi UNRi PUBi IND PHY ONL UNI PUR FIN COM NAV INT DEM CNT STA PO
L HEA PRE LOC GOV"], content-type=[application/zip], connection=[close], last-mo
dified=[Fri, 06 Aug 2010 14:47:50 GMT], content-length=[2000000], age=[0], serve
r=[YTS/1.17.13], accept-ranges=[bytes], date=[Fri, 06 Aug 2010 20:08:33 GMT]}  
D/AsyncDownloadFile( 2372): bytes: 1024  
D/AsyncDownloadFile( 2372): bytes: 1033  
D/AsyncDownloadFile( 2372): bytes: 2057  
D/AsyncDownloadFile( 2372): bytes: 2493  
D/AsyncDownloadFile( 2372): bytes: 3517  
D/AsyncDownloadFile( 2372): bytes: 3953  

.
.
.

@ 4:13:25  
D/AsyncDownloadFile( 2372): bytes: 386473  
D/AsyncDownloadFile( 2372): bytes: 387497  
D/AsyncDownloadFile( 2372): bytes: 387933  
D/AsyncDownloadFile( 2372): new download seek: 387933; lengthFile: 2000000  
D/AsyncDownloadFile( 2372): header fields: {p3p=[policyref="http://info.yahoo.co
m/w3c/p3p.xml", CP="CAO DSP COR CUR ADM DEV TAI PSA PSD IVAi IVDi CONi TELo OTPi
 OUR DELi SAMi OTRi UNRi PUBi IND PHY ONL UNI PUR FIN COM NAV INT DEM CNT STA PO
L HEA PRE LOC GOV"], content-type=[application/zip], connection=[close], last-mo
dified=[Fri, 06 Aug 2010 14:47:50 GMT], content-length=[1612067], age=[0], serve
r=[YTS/1.17.13], accept-ranges=[bytes], date=[Fri, 06 Aug 2010 20:13:29 GMT], co
ntent-range=[bytes 387933-1999999/2000000]}  
D/AsyncDownloadFile( 2372): bytes: 388957  
D/AsyncDownloadFile( 2372): bytes: 389981  
D/AsyncDownloadFile( 2372): bytes: 390409  
D/AsyncDownloadFile( 2372): bytes: 391433  
D/AsyncDownloadFile( 2372): bytes: 391869  
D/AsyncDownloadFile( 2372): bytes: 392893  

.
.
.

@ 4:18:45  
D/AsyncDownloadFile( 2372): bytes: 775413  
D/AsyncDownloadFile( 2372): bytes: 775849  
D/AsyncDownloadFile( 2372): bytes: 776873  
D/AsyncDownloadFile( 2372): bytes: 777309  
D/AsyncDownloadFile( 2372): new download seek: 777309; lengthFile: 2000000  
D/AsyncDownloadFile( 2372): header fields: {p3p=[policyref="http://info.yahoo.co
m/w3c/p3p.xml", CP="CAO DSP COR CUR ADM DEV TAI PSA PSD IVAi IVDi CONi TELo OTPi
 OUR DELi SAMi OTRi UNRi PUBi IND PHY ONL UNI PUR FIN COM NAV INT DEM CNT STA PO
L HEA PRE LOC GOV"], content-type=[application/zip], connection=[close], last-mo
dified=[Fri, 06 Aug 2010 14:47:50 GMT], content-length=[1222691], age=[0], serve
r=[YTS/1.17.13], accept-ranges=[bytes], date=[Fri, 06 Aug 2010 20:18:54 GMT], co
ntent-range=[bytes 777309-1999999/2000000]}  
D/dalvikvm( 2372): GC_FOR_MALLOC freed 11019 objects / 470560 bytes in 155ms  
D/AsyncDownloadFile( 2372): bytes: 778333  
D/AsyncDownloadFile( 2372): bytes: 779357  
D/AsyncDownloadFile( 2372): bytes: 779790  
D/AsyncDownloadFile( 2372): bytes: 780814  
D/AsyncDownloadFile( 2372): bytes: 781250  
D/AsyncDownloadFile( 2372): bytes: 782274  

.
.
.

@ 4:23:45  
D/AsyncDownloadFile( 2372): bytes: 1163334  
D/AsyncDownloadFile( 2372): bytes: 1163770  
D/AsyncDownloadFile( 2372): bytes: 1164794  
D/AsyncDownloadFile( 2372): bytes: 1165230  
D/AsyncDownloadFile( 2372): new download seek: 1165230; lengthFile: 2000000  
D/AsyncDownloadFile( 2372): header fields: {p3p=[policyref="http://info.yahoo.co
m/w3c/p3p.xml", CP="CAO DSP COR CUR ADM DEV TAI PSA PSD IVAi IVDi CONi TELo OTPi
 OUR DELi SAMi OTRi UNRi PUBi IND PHY ONL UNI PUR FIN COM NAV INT DEM CNT STA PO
L HEA PRE LOC GOV"], content-type=[application/zip], connection=[close], last-mo
dified=[Fri, 06 Aug 2010 14:47:50 GMT], content-length=[834770], age=[0], server
=[YTS/1.17.13], accept-ranges=[bytes], date=[Fri, 06 Aug 2010 20:23:47 GMT], con
tent-range=[bytes 1165230-1999999/2000000]}  
D/AsyncDownloadFile( 2372): bytes: 1166246  
D/AsyncDownloadFile( 2372): bytes: 1167270  
D/AsyncDownloadFile( 2372): bytes: 1167706  
D/AsyncDownloadFile( 2372): bytes: 1168730  
D/AsyncDownloadFile( 2372): bytes: 1169754  
D/AsyncDownloadFile( 2372): bytes: 1170778  

.
.
.

@ 4:30:25  
D/AsyncDownloadFile( 2372): bytes: 1551255  
D/AsyncDownloadFile( 2372): bytes: 1551691  
D/AsyncDownloadFile( 2372): bytes: 1552715  
D/AsyncDownloadFile( 2372): bytes: 1553151  
D/AsyncDownloadFile( 2372): new download seek: 1553151; lengthFile: 2000000  
D/AsyncDownloadFile( 2372): header fields: {p3p=[policyref="http://info.yahoo.co
m/w3c/p3p.xml", CP="CAO DSP COR CUR ADM DEV TAI PSA PSD IVAi IVDi CONi TELo OTPi
 OUR DELi SAMi OTRi UNRi PUBi IND PHY ONL UNI PUR FIN COM NAV INT DEM CNT STA PO
L HEA PRE LOC GOV"], content-type=[application/zip], connection=[close], last-mo
dified=[Fri, 06 Aug 2010 14:47:50 GMT], content-length=[446849], age=[0], server
=[YTS/1.17.13], accept-ranges=[bytes], date=[Fri, 06 Aug 2010 20:30:44 GMT], con
tent-range=[bytes 1553151-1999999/2000000]}  
D/AsyncDownloadFile( 2372): bytes: 1554167  
D/AsyncDownloadFile( 2372): bytes: 1554184  
D/AsyncDownloadFile( 2372): bytes: 1555208  
D/AsyncDownloadFile( 2372): bytes: 1555644  
D/AsyncDownloadFile( 2372): bytes: 1556668  
D/AsyncDownloadFile( 2372): bytes: 1557104  

.
.
.

@ 4:37:10  
D/AsyncDownloadFile( 2372): bytes: 1939188  
D/AsyncDownloadFile( 2372): bytes: 1939624  
D/AsyncDownloadFile( 2372): bytes: 1940648  
D/AsyncDownloadFile( 2372): bytes: 1941084  
D/AsyncDownloadFile( 2372): new download seek: 1941084; lengthFile: 2000000  
D/dalvikvm( 2372): GC_FOR_MALLOC freed 13701 objects / 604600 bytes in 128ms
D/AsyncDownloadFile( 2372): header fields: {p3p=[policyref="http://info.yahoo.co
m/w3c/p3p.xml", CP="CAO DSP COR CUR ADM DEV TAI PSA PSD IVAi IVDi CONi TELo OTPi
 OUR DELi SAMi OTRi UNRi PUBi IND PHY ONL UNI PUR FIN COM NAV INT DEM CNT STA PO
L HEA PRE LOC GOV"], content-type=[application/zip], connection=[close], last-mo
dified=[Fri, 06 Aug 2010 14:47:50 GMT], content-length=[58916], age=[0], server=
[YTS/1.17.13], accept-ranges=[bytes], date=[Fri, 06 Aug 2010 20:37:16 GMT], cont
ent-range=[bytes 1941084-1999999/2000000]}  
D/AsyncDownloadFile( 2372): bytes: 1942108  
D/AsyncDownloadFile( 2372): bytes: 1942117  
D/AsyncDownloadFile( 2372): bytes: 1943141  
D/AsyncDownloadFile( 2372): bytes: 1943577  
D/AsyncDownloadFile( 2372): bytes: 1944601  
D/AsyncDownloadFile( 2372): bytes: 1945037  

.
.
.

@ 4:38:30  
D/AsyncDownloadFile( 2372): bytes: 1993217  
D/AsyncDownloadFile( 2372): bytes: 1994241  
D/AsyncDownloadFile( 2372): bytes: 1994677  
D/AsyncDownloadFile( 2372): bytes: 1995701  
D/AsyncDownloadFile( 2372): bytes: 1996137  
D/AsyncDownloadFile( 2372): bytes: 1997161  
D/AsyncDownloadFile( 2372): bytes: 1997597  
D/AsyncDownloadFile( 2372): bytes: 1998621  
D/AsyncDownloadFile( 2372): bytes: 1999057  
D/onPostExecute( 2372): download: unsuccessful  

- - -  
Run Code Online (Sandbox Code Playgroud)

在BalusC的提示之后(谢谢),我修改了连接设置,但雅虎!服务器在每次中断时继续重置为文件的开头.这是更改的代码和生成的转储:

            // Setup connection.
            URL url = new URL(strUrl[0]);
            URLConnection connection = url.openConnection();
            downloaded = Integer.parseInt(strUrl[3]);
            if (downloaded == 0) {
                connection.connect();
                strLastModified = connection.getHeaderField("Last-Modified");
                fileLength = connection.getContentLength();
                mDownloadFileLength = fileLength;
            }
            else {
                connection.setRequestProperty("Range", "bytes=" + downloaded + "-");
                connection.setRequestProperty("If-Range", strLastModified);
                connection.connect();
                fileLength = mDownloadFileLength;
                Log.d("AsyncDownloadFile", 
                        "new download seek: " + downloaded +
                        "; lengthFile: " + fileLength);
            }
            map = connection.getHeaderFields();
            Log.d("AsyncDownloadFile", "header fields: " + map.toString());
Run Code Online (Sandbox Code Playgroud)

倾倒:

@12:36:40 started  
D/AsyncDownloadFile(  413): header fields: {p3p=[policyref="http://info.yahoo.c
m/w3c/p3p.xml", CP="CAO DSP COR CUR ADM DEV TAI PSA PSD IVAi IVDi CONi TELo OTP
 OUR DELi SAMi OTRi UNRi PUBi IND PHY ONL UNI PUR FIN COM NAV INT DEM CNT STA P
L HEA PRE LOC GOV"], content-type=[application/zip], connection=[close], last-m
dified=[Fri, 06 Aug 2010 14:47:50 GMT], content-length=[2000000], age=[0], serv
r=[YTS/1.17.13], accept-ranges=[bytes], date=[Sat, 07 Aug 2010 04:36:56 GMT]}  
D/AsyncDownloadFile(  413): bytes: 1024  
D/AsyncDownloadFile(  413): bytes: 2048  
D/AsyncDownloadFile(  413): bytes: 2476  
D/AsyncDownloadFile(  413): bytes: 3500  
D/AsyncDownloadFile(  413): bytes: 3936  

...

@12:39:20 interrupted  
D/AsyncDownloadFile(  413): bytes: 388068  
D/AsyncDownloadFile(  413): bytes: 389092  
D/AsyncDownloadFile(  413): bytes: 389376  
D/AsyncDownloadFile(  413): new download seek: 389376; lengthFile: 2000000  
D/AsyncDownloadFile(  413): header fields: {p3p=[policyref="http://info.yahoo.co
m/w3c/p3p.xml", CP="CAO DSP COR CUR ADM DEV TAI PSA PSD IVAi IVDi CONi TELo OTPi
 OUR DELi SAMi OTRi UNRi PUBi IND PHY ONL UNI PUR FIN COM NAV INT DEM CNT STA PO
L HEA PRE LOC GOV"], content-type=[application/zip], connection=[close], last-mo
dified=[Fri, 06 Aug 2010 14:47:50 GMT], content-length=[1610624], age=[0], serve
r=[YTS/1.17.13], accept-ranges=[bytes], date=[Sat, 07 Aug 2010 04:39:21 GMT], co
ntent-range=[bytes 389376-1999999/2000000]}  
D/AsyncDownloadFile(  413): bytes: 390400  
D/AsyncDownloadFile(  413): bytes: 390409  
D/AsyncDownloadFile(  413): bytes: 391433  
D/AsyncDownloadFile(  413): bytes: 391869  

...

@12:44:10 interrupted  
D/AsyncDownloadFile(  413): bytes: 775413  
D/AsyncDownloadFile(  413): bytes: 775849  
D/AsyncDownloadFile(  413): bytes: 776873  
D/AsyncDownloadFile(  413): bytes: 777309  
D/AsyncDownloadFile(  413): new download seek: 777309; lengthFile: 2000000  
D/AsyncDownloadFile(  413): header fields: {p3p=[policyref="http://info.yahoo.co
m/w3c/p3p.xml", CP="CAO DSP COR CUR ADM DEV TAI PSA PSD IVAi IVDi CONi TELo OTPi
 OUR DELi SAMi OTRi UNRi PUBi IND PHY ONL UNI PUR FIN COM NAV INT DEM CNT STA PO
L HEA PRE LOC GOV"], content-type=[application/zip], connection=[close], last-mo
dified=[Fri, 06 Aug 2010 14:47:50 GMT], content-length=[1222691], age=[0], serve
r=[YTS/1.17.13], accept-ranges=[bytes], date=[Sat, 07 Aug 2010 04:44:20 GMT], co
ntent-range=[bytes 777309-1999999/2000000]}  
D/dalvikvm(  413): GC_FOR_MALLOC freed 10869 objects / 465664 bytes in 122ms  
D/AsyncDownloadFile(  413): bytes: 778333  
D/AsyncDownloadFile(  413): bytes: 778342  
D/AsyncDownloadFile(  413): bytes: 779366  
D/AsyncDownloadFile(  413): bytes: 779802  

...

@12:49:30 interrupted  
D/AsyncDownloadFile(  413): bytes: 1163782  
D/AsyncDownloadFile(  413): bytes: 1164806  
D/AsyncDownloadFile(  413): bytes: 1165242  
D/AsyncDownloadFile(  413): new download seek: 1165242; lengthFile: 2000000  
D/AsyncDownloadFile(  413): header fields: {p3p=[policyref="http://info.yahoo.co
m/w3c/p3p.xml", CP="CAO DSP COR CUR ADM DEV TAI PSA PSD IVAi IVDi CONi TELo OTPi
 OUR DELi SAMi OTRi UNRi PUBi IND PHY ONL UNI PUR FIN COM NAV INT DEM CNT STA PO
L HEA PRE LOC GOV"], content-type=[application/zip], connection=[close], last-mo
dified=[Fri, 06 Aug 2010 14:47:50 GMT], content-length=[834758], age=[0], server
=[YTS/1.17.13], accept-ranges=[bytes], date=[Sat, 07 Aug 2010 04:49:43 GMT], con
tent-range=[bytes 1165242-1999999/2000000]}  
D/AsyncDownloadFile(  413): bytes: 1166266  
D/AsyncDownloadFile(  413): bytes: 1167290  
D/AsyncDownloadFile(  413): bytes: 1167718  
D/AsyncDownloadFile(  413): bytes: 1168742  

...

@12:55:30 interrupted  
D/AsyncDownloadFile(  413): bytes: 1552722  
D/AsyncDownloadFile(  413): bytes: 1553158  
D/AsyncDownloadFile(  413): bytes: 1554182  
D/AsyncDownloadFile(  413): bytes: 1554618  
D/AsyncDownloadFile(  413): new download seek: 1554618; lengthFile: 2000000  
D/AsyncDownloadFile(  413): header fields: {p3p=[policyref="http://info.yahoo.co
m/w3c/p3p.xml", CP="CAO DSP COR CUR ADM DEV TAI PSA PSD IVAi IVDi CONi TELo OTPi
 OUR DELi SAMi OTRi UNRi PUBi IND PHY ONL UNI PUR FIN COM NAV INT DEM CNT STA PO
L HEA PRE LOC GOV"], content-type=[application/zip], connection=[close], last-mo
dified=[Fri, 06 Aug 2010 14:47:50 GMT], content-length=[445382], age=[0], server
=[YTS/1.17.13], accept-ranges=[bytes], date=[Sat, 07 Aug 2010 04:55:39 GMT], con
tent-range=[bytes 1554618-1999999/2000000]}  
D/AsyncDownloadFile(  413): bytes: 1555642  
D/AsyncDownloadFile(  413): bytes: 1556666  
D/AsyncDownloadFile(  413): bytes: 1557094  
D/AsyncDownloadFile(  413): bytes: 1558118  

...

@12:57:20 interrupted  
D/AsyncDownloadFile(  413): bytes: 1941834  
D/AsyncDownloadFile(  413): bytes: 1942858  
D/AsyncDownloadFile(  413): bytes: 1943882  
D/AsyncDownloadFile(  413): bytes: 1943994  
D/AsyncDownloadFile(  413): new download seek: 1943994; lengthFile: 2000000  
D/AsyncDownloadFile(  413): header fields: {p3p=[policyref="http://info.yahoo.co
m/w3c/p3p.xml", CP="CAO DSP COR CUR ADM DEV TAI PSA PSD IVAi IVDi CONi TELo OTPi
 OUR DELi SAMi OTRi UNRi PUBi IND PHY ONL UNI PUR FIN COM NAV INT DEM CNT STA PO
L HEA PRE LOC GOV"], content-type=[application/zip], connection=[close], last-mo
dified=[Fri, 06 Aug 2010 14:47:50 GMT], content-length=[56006], age=[0], server=
[YTS/1.17.13], accept-ranges=[bytes], date=[Sat, 07 Aug 2010 04:57:15 GMT], cont
ent-range=[bytes 1943994-1999999/2000000]}  
D/dalvikvm(  413): GC_FOR_MALLOC freed 13617 objects / 602200 bytes in 165ms  
D/AsyncDownloadFile(  413): bytes: 1945018  
D/AsyncDownloadFile(  413): bytes: 1946042  
D/AsyncDownloadFile(  413): bytes: 1946470  
D/AsyncDownloadFile(  413): bytes: 1947494  

...

@12:58:10 finished  
D/AsyncDownloadFile(  413): bytes: 1996103  
D/AsyncDownloadFile(  413): bytes: 1997127  
D/AsyncDownloadFile(  413): bytes: 1997563  
D/AsyncDownloadFile(  413): bytes: 1998587  
D/AsyncDownloadFile(  413): bytes: 1999023  
D/onPostExecute(  413): downloaded: unsuccessful  
Run Code Online (Sandbox Code Playgroud)

Bal*_*usC 26

要恢复下载,您不仅需要发送Range请求标头,还需要发送If-Range应包含唯一文件标识符或文件修改时间戳的请求标头.

如果服务器ETag在初始下载时返回响应头,则应If-Range在后续恢复请求的标头中使用它.或者,如果它返回Last-Modified响应头,那么您应该在If-Range请求头中使用它.

查看日志,服务器已发送Last-Modified响应标头.因此,您应该If-Range在简历请求的标题中将其发回.

// Initial download.
String lastModified = connection.getHeaderField("Last-Modified");

// ...

// Resume download.
connection.setRequestProperty("If-Range", lastModified); 
Run Code Online (Sandbox Code Playgroud)

服务器将使用此信息来验证您是否请求完全相同的文件.

  • 我收到了雅虎的回复!技术支持说雅虎!服务器不支持字节范围请求:"Yahoo! Web Hosting不支持Accept-range标头,因为我们使用服务器池并且每个请求可能会到达不同的服务器.您将在响应头中看到connection = [close]这个." (2认同)

小智 5

似乎问题在于呼唤

input = new BufferedInputStream(url.openStream(), 8192);
Run Code Online (Sandbox Code Playgroud)

代替

input = new BufferedInputStream(connection.getInputStream(), 8192);
Run Code Online (Sandbox Code Playgroud)

url.openStream()再次调用openConnection() WITHOUT属性.