如何处理传输编码：使用 .NET Core HttpClient.PostAsync 下载文件时分块

Question

情况

我正在使用 HttpClient（System.Net.Http，版本=4.2.1.0）将包含多部分表单数据的 HTTP 请求 POST 到 Web API。表单数据包括字符串参数 ( ) 和包含在 中的benchmark文件 ( ) 。API 调用返回一个我想要保存到磁盘的 CSV 文件。addressFilestream

响应包含标头Transfer-Encoding: chunked，其中包含的数据responseBytes 包括块标头。我希望HttpClient库删除这些标头，它们是实际内容的元数据。相反，它只是将标题行包含在Content.

问题

处理这些块头的正确方法是什么？

我当然可以自己编写一个方法来处理标头，但我发现很难相信 HttpClient 库还没有在某个地方内置此功能。

代码

using (var client = new HttpClient())
        {
            var content = new MultipartFormDataContent();
            content.Add(new StringContent("Public_AR_Current"), "benchmark");
            content.Add(new ByteArrayContent(stream.ToArray()), "addressFile", "addressFile.csv");

            var response = await client.PostAsync("https://geocoding.geo.census.gov/geocoder/locations/addressbatch", content);

            var responseBytes = await response.Content.ReadAsByteArrayAsync();
            saveResponse(responseBytes);

            var geocodedItems = ParseGeocodeResponse(responseBytes);
            var parsedItems = geocodedItems.Select(gi => gi.ToEpaHandlerUsCensusGeocode());
            return parsedItems;
        }

结果

请注意第一行和后续行 ( 0fe8、0060、0fe8) 上的块标头。

0fe8
0fe8
"AK0000036228","500 HOLLYWOOD DR, ANCHORAGE, AK, 99501","Match","Exact","500 HOLLYWOOD DR, ANCHORAGE, AK, 99501","-149.87424,61.23034","190797469","R"
"AK0000363994","3155 E 18TH CIR, ANCHORAGE, AK, 99508","Match","Non_Exact","3155 E 18TH CIR, ANCHORAGE, AK, 99508","-149.82193,61.20462","190799569","L"
...
0060
28712","N 65 DEG 35 15 W 167 DEG 55 18, WALES, AK, 99734","No_Match"
"AK0000112227","KODIAK ARPR
...
0fe8
T AREA, KODIAK, AK, 99615","No_Match"
"AK0000033902","2130 E DIMOND BLVD, ANCHORAGE, AK, 99515","Match","Non_Exact","2130 W DIMOND BLVD, ANCHORAGE, AK, 99515","-149.91881,61.1375","190795925","L"
"AK0000562769","3100 TONGASS AVE, KETCHIKAN, AK, 99901-5746","No_Match"

预期结果

我希望 HttpClient 库能够删除标头。

"AK0000036228","500 HOLLYWOOD DR, ANCHORAGE, AK, 99501","Match","Exact","500 HOLLYWOOD DR, ANCHORAGE, AK, 99501","-149.87424,61.23034","190797469","R"
"AK0000363994","3155 E 18TH CIR, ANCHORAGE, AK, 99508","Match","Non_Exact","3155 E 18TH CIR, ANCHORAGE, AK, 99508","-149.82193,61.20462","190799569","L"
"AK0000228718","1050 ASPEN ST, FAIRBANKS, AK, 99709-5501","Match","Exact","1050 ASPEN ST, FAIRBANKS, AK, 99709","-147.7731,64.8535","605310042","L"
"AK0000536714","SMITH COVE IN SMITH LAGOON T74S R86E CRM S17 & 20, KASAAN, AK, 99901","No_Match"
"AK0001413822","USS-12403, N BANK WOOD RIVER, ALEKNAGIK, AK, 99555","No_Match"
"AK0000489567","BREAKWATER BTWN WESTERN AVE & TAIT ST, METLAKATLA, AK, 99926","No_Match"

Answer 1

我最终编写了这个扩展方法，它对于我的用例来说表现得足够好。

    public static Task<Stream> ReadAsStreamAsync(this HttpContent content, bool isChunked)
    {
        if (!isChunked)
        {
            return content.ReadAsStreamAsync();
        }
        else
        {
            var task = content.ReadAsStreamAsync()
            .ContinueWith<Stream>((streamTask) =>
            {
                var outputStream = new MemoryStream();
                var buffer = new char[1024 * 1024];
                var stream = streamTask.Result;

                // No using() so that we don't dispose stream.
                var tr = new StreamReader(stream);
                var tw = new StreamWriter(outputStream);

                while (!tr.EndOfStream)
                {
                    var chunkSizeStr = tr.ReadLine().Trim();
                    var chunkSize = int.Parse(chunkSizeStr, System.Globalization.NumberStyles.HexNumber);

                    tr.ReadBlock(buffer, 0, chunkSize);
                    tw.Write(buffer, 0, chunkSize);
                    tr.ReadLine();
                }

                return outputStream;
            });

            return task;
        }


    }