如何以流的形式获取httpclient响应

Question

我正在使用 httpclient 4.5.5 我想获得高达 1 gb 的大文件作为响应。但好像

CloseableHttpResponse response = httpClient.execute(httpGet); HttpEntity entity = response.getEntity();

这将返回整个响应，因此在内存中包含整个响应并不好。有没有办法以流的形式获得响应？

Answer 1

Apache HttpClient 自 4.0 版起（以及 Apache HttpAsyncClient）支持传入和传出 HTTP 消息的完整内容流。用于HttpEntity访问底层内容输入流

CloseableHttpClient client = HttpClients.createDefault();
HttpGet httpGet = new HttpGet("http://myhost/tons-of-stuff");
try (CloseableHttpResponse response1 = client.execute(httpGet)) {
    final HttpEntity entity = response1.getEntity();
    if (entity != null) {
        try (InputStream inputStream = entity.getContent()) {
            // do something useful with the stream
        }
    }
}

Answer 2

您需要 Apache 异步客户端。

HttpAsyncClient 是 Apache HttpClient 的异步版本。Apache HttpClient 在内存中构造整个响应，而使用 HttpAsyncClient，您可以定义一个 Handler（消费者）来在接收数据时处理响应。

https://hc.apache.org/httpcomponents-asyncclient-4.1.x/index.html

这是他们官方示例代码中的示例

package org.apache.http.examples.nio.client;

import java.io.IOException;
import java.nio.CharBuffer;
import java.util.concurrent.Future;

import org.apache.http.HttpResponse;
import org.apache.http.impl.nio.client.CloseableHttpAsyncClient;
import org.apache.http.impl.nio.client.HttpAsyncClients;
import org.apache.http.nio.IOControl;
import org.apache.http.nio.client.methods.AsyncCharConsumer;
import org.apache.http.nio.client.methods.HttpAsyncMethods;
import org.apache.http.protocol.HttpContext;

/**
 * This example demonstrates an asynchronous HTTP request / response exchange with
 * a full content streaming.
 */
public class AsyncClientHttpExchangeStreaming {

    public static void main(final String[] args) throws Exception {
        CloseableHttpAsyncClient httpclient = HttpAsyncClients.createDefault();
        try {
            httpclient.start();
            Future<Boolean> future = httpclient.execute(
                    HttpAsyncMethods.createGet("http://httpbin.org/"),
                    new MyResponseConsumer(), null);
            Boolean result = future.get();
            if (result != null && result.booleanValue()) {
                System.out.println("Request successfully executed");
            } else {
                System.out.println("Request failed");
            }
            System.out.println("Shutting down");
        } finally {
            httpclient.close();
        }
        System.out.println("Done");
    }

    static class MyResponseConsumer extends AsyncCharConsumer<Boolean> {

        @Override
        protected void onResponseReceived(final HttpResponse response) {
        }

        @Override
        protected void onCharReceived(final CharBuffer buf, final IOControl ioctrl) throws IOException {
            while (buf.hasRemaining()) {
                System.out.print(buf.get());
            }
        }

        @Override
        protected void releaseResources() {
        }

        @Override
        protected Boolean buildResult(final HttpContext context) {
            return Boolean.TRUE;
        }

    }

}

Answer 3

使用HttpURLConnection而不是httpClient.

final HttpURLConnection conn = (HttpURLConnection)url.openConnection();
final int bufferSize = 1024 * 1024;
conn.setChunkedStreamingMode(bufferSize);
final OutputStream out = conn.getOutputStream();