从打开的HTTP流中读取数据

Question

我正在尝试使用.NET WebRequest/WebResponse类来访问Twitter流API "http://stream.twitter.com/spritzer.json".

我需要能够打开连接并从打开的连接中逐步读取数据.

目前,当我调用WebRequest.GetResponse方法时,它会一直阻塞,直到整个响应被下载.我知道有一种BeginGetResponse方法,但这只会在后台线程上做同样的事情.在下载仍在进行时,我需要访问响应流.对于这些类,这对我来说似乎不太可能.

Twitter文档中有关于此的具体评论:

"请注意,某些HTTP客户端库仅在服务器关闭连接后才返回响应主体.这些客户端不能用于访问Streaming API.您必须使用将以递增方式返回响应数据的HTTP客户端.最强大的HTTP客户端库将提供此功能.例如,Apache HttpClient将处理此用例."

他们指向Appache HttpClient,但这没有多大帮助,因为我需要使用.NET.

是否可以使用WebRequest/WebResponse,或者我是否必须选择较低级别的网络类？也许有其他库可以让我这样做？

艾伦

Answer 1

我最终使用了TcpClient,工作正常.仍然有兴趣知道这是否可以使用WebRequest/WebResponse.这是我的代码,以防任何人感兴趣:

using (TcpClient client = new TcpClient())
{

string requestString = "GET /spritzer.json HTTP/1.1\r\n";
requestString += "Authorization: " + token + "\r\n";
requestString += "Host: stream.twitter.com\r\n";
requestString += "Connection: keep-alive\r\n";
requestString += "\r\n";

client.Connect("stream.twitter.com", 80);

using (NetworkStream stream = client.GetStream())
{
    // Send the request.
    StreamWriter writer = new StreamWriter(stream);
    writer.Write(requestString);
    writer.Flush();

    // Process the response.
    StreamReader rdr = new StreamReader(stream);

    while (!rdr.EndOfStream)
    {
        Console.WriteLine(rdr.ReadLine());
    }
}
}

Answer 2

BeginGetResponse是您需要的方法.它允许您以递增方式读取响应流:

class Program
{
    static void Main(string[] args)
    {
        WebRequest request = WebRequest.Create("http://stream.twitter.com/spritzer.json");
        request.Credentials = new NetworkCredential("username", "password");
        request.BeginGetResponse(ar => 
        {
            var req = (WebRequest)ar.AsyncState;
            // TODO: Add exception handling: EndGetResponse could throw
            using (var response = req.EndGetResponse(ar))
            using (var reader = new StreamReader(response.GetResponseStream()))
            {
                // This loop goes as long as twitter is streaming
                while (!reader.EndOfStream)
                {
                    Console.WriteLine(reader.ReadLine());
                }
            }
        }, request);

        // Press Enter to stop program
        Console.ReadLine();
    }
}

或者如果您对WebClient感觉更舒服(我个人更喜欢它而不是WebRequest):

using (var client = new WebClient())
{
    client.Credentials = new NetworkCredential("username", "password");
    client.OpenReadCompleted += (sender, e) =>
    {
        using (var reader = new StreamReader(e.Result))
        {
            while (!reader.EndOfStream)
            {
                Console.WriteLine(reader.ReadLine());
            }
        }
    };
    client.OpenReadAsync(new Uri("http://stream.twitter.com/spritzer.json"));
}
Console.ReadLine();

Answer 3

您尝试过WebRequest.BeginGetRequestStream()吗？

或者是这样的：

HttpWebRequest request = (HttpWebRequest)WebRequest.Create (http://www.twitter.com );
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
StreamReader reader = new StreamReader(response.GetResponseStream()); 

string str = reader.ReadLine();
while(str != null)
{
   Console.WriteLine(str);
   str = reader.ReadLine();
}