HTTP GET检索压缩文件

Question

我已经开发了一个HTTP通信对象,用于通过GET请求下载文件.

这在下载文本文件时工作得很好.但是,下载压缩文件(如zip,gz或tar.gz)似乎会下载该文件,但该文件无效.

在zip的情况下,我得到一个meesage说它试图在文件的开始之前移动指针.在.tar.gz的情况下,消息是file.tar中的数据错误.文件坏了.

在所有情况下,我使用的下载链接都允许从URL进行完整和正确的下载.然而,基于Java代码的下载使文件失效但无效.

代码如下:

public class HTTPCommunicationGet {

    private URIBuilder sendData;
    private URI target;
    private HttpGet getConnection;

    public HTTPCommunicationGet(String url, TreeMap<String, String> components) {
        super(url, components);
    }

    public HTTPCommunicationGet(String url, String queryString) {
        super(url, queryString);
    }

    protected void defineSendData() throws URISyntaxException, IOException {
        this.sendData = new URIBuilder(new URI(this.getUrl()));
        if (this.getComponents() != null && this.getComponents().size() > 0) {
            for (Map.Entry<String, String> component : this.getComponents().entrySet()) {
                this.sendData.setParameter(component.getKey(), component.getValue());
            }
        }
    }

    protected void retrieveRemoteData() throws IOException, MalformedURLException, URISyntaxException, DataMapHTTPGetException {

        this.target = this.sendData.build();
        this.getConnection = new HttpGet(target);
        HttpResponse response = client.execute(this.getConnection);
        if (response.getStatusLine().toString().toUpperCase().contains("200 OK")) {
            this.setResponse(response.getStatusLine().toString(), "Data Retrieved");
            BufferedReader rd = new BufferedReader(new InputStreamReader(response.getEntity().getContent()));
            String line = "";
            while ((line = rd.readLine()) != null) {
                this.remoteData.append(line);
            }
        } else {
            String message = String.format("%s: Provider connection exception; response returned was not 200 OK", this.target.toASCIIString());
            this.setResponse(response.getStatusLine().toString(), message);
            DataMapHTTPGetException ex = new DataMapHTTPGetException(target.toString(), message);
            throw ex;
        }
    }

    public void downloadFiles(String localFile) throws DataMapConnectionException, FileNotFoundException, IOException, URISyntaxException {
        // check that we have remoteData set
        this.defineSendData();
        this.retrieveRemoteData(); // everything is bubbled up to the controller class that is calling this.

        File localMetaFile = new File(localFile);
        switch (this.archiveMetaFile(localMetaFile)) {
            case -1:
                IOException ex = new IOException(String.format("The file %s could not be moved", localFile));
                throw ex;
            //break;
            case 0:
                infoLog.info(String.format("%s: this file did not already exist", localFile));
                break;
            case 1:
                infoLog.info(String.format("%s: this file was found and successfully archived to the processed directory", localFile));
                break;
        }

        BufferedWriter fileWriter = new BufferedWriter(new FileWriter(localFile));
        fileWriter.write(this.remoteData.toString());
        fileWriter.close();
    }
}

正如您所看到的,在初始化对象后通过downloadFiles调用此方法.我已经删除了这个例子不需要的代码,例如archiveMetaFile方法.

关于为什么这不适用于压缩文件的任何指针都非常感谢.

干杯内森

Answer 1

问题可能是您使用的是BufferedReader而不是InputStream.读者用于文本数据并强加字符编码,而InputStreams可以处理原始二进制数据.

尝试切换到一个BufferedInputStream.使用任何Reader类都会破坏二进制数据.