流式HTTP与GZIP由StreamReader缓冲？

Question

苦苦寻找遇到类似问题或任何类似问题的人.

我目前正在使用具有GZip要求的http(json)流,我遇到了从发送数据到reader.ReadLine()读取数据时的延迟.有人向我建议,这可能与解码保留缓冲区中的数据有关？

这就是我目前的情况,它除了延迟之外还可以正常工作.

HttpWebRequest request = (HttpWebRequest)WebRequest.Create(endPoint);
request.Method = "GET";

request.PreAuthenticate = true;
request.Credentials = new NetworkCredential(username, password);

request.AutomaticDecompression = DecompressionMethods.GZip;
request.ContentType = "application/json";
request.Accept = "application/json";
request.Timeout = 30;
request.BeginGetResponse(AsyncCallback, request);

然后在AsyncCallback方法中我有:

HttpWebRequest request = result.AsyncState as HttpWebRequest;

using (HttpWebResponse response = (HttpWebResponse)request.EndGetResponse(result))
using (Stream stream = response.GetResponseStream())
using (StreamReader reader = new StreamReader(stream, Encoding.UTF8))
{

    while (!reader.EndOfStream)
    {
        string line = reader.ReadLine();
        if (string.IsNullOrWhiteSpace(line)) continue;

        Console.WriteLine(line);
    }
}

它只是坐下来reader.Readline()直到收到更多的数据,然后甚至阻止其中一些.还有收到的保持活动的新行,当它决定阅读某些内容时,通常会立即读出这些新行.

我已经测试了运行curl命令并行运行的流,curl命令完全接收和解压缩数据.

任何见解都会很棒.谢谢,

担

编辑 在streamreader上使用缓冲区大小没有运气.

new StreamReader(stream, Encoding.UTF8, true, 1)

编辑 也没有运气更新到.NET 4.5和使用

request.AllowReadStreamBuffering = false;

Answer 1

更新:这似乎在较长时间内具有较高的卷速率问题,并且应该仅在缓冲区影响应用程序功能的小卷上使用.我已经切换回了StreamReader.

所以这就是我最终想出来的.这没有延迟.这不会通过自动GZip解压缩得到缓冲.

using (HttpWebResponse response = (HttpWebResponse)request.EndGetResponse(result))
using (Stream stream = response.GetResponseStream())
using (MemoryStream memory = new MemoryStream())
using (GZipStream gzip = new GZipStream(memory, CompressionMode.Decompress))
{
    byte[] compressedBuffer = new byte[8192];
    byte[] uncompressedBuffer = new byte[8192];
    List<byte> output = new List<byte>();

    while (stream.CanRead)
    {
        int readCount = stream.Read(compressedBuffer, 0, compressedBuffer.Length);

        memory.Write(compressedBuffer.Take(readCount).ToArray(), 0, readCount);
        memory.Position = 0;

        int uncompressedLength = gzip.Read(uncompressedBuffer, 0, uncompressedBuffer.Length);

        output.AddRange(uncompressedBuffer.Take(uncompressedLength));

        if (!output.Contains(0x0A)) continue;

        byte[] bytesToDecode = output.Take(output.LastIndexOf(0x0A) + 1).ToArray();
        string outputString = Encoding.UTF8.GetString(bytesToDecode);
        output.RemoveRange(0, bytesToDecode.Length);

        string[] lines = outputString.Split(new[] { Environment.NewLine }, new StringSplitOptions());
        for (int i = 0; i < (lines.Length - 1); i++)
        {
            Console.WriteLine(lines[i]);
        }

        memory.SetLength(0);
    }
}