并行http请求

Question

我有一个应用程序使用backgroundWorker向last.fm网站发出API请求.最初我不知道我需要做多少请求.响应包含总页数,所以我只会在第一次请求后得到它.这是下面的代码.

    private void backgroundWorker_DoWork(object sender, DoWorkEventArgs e)
    {            
        int page = 1;
        int totalpages = 1;

        while (page <= totalpages)
        {
            if (backgroundWorker.CancellationPending)
            {
                e.Cancel = true;
                return;
            }

            //Here is the request part
            string Response = RecentTracksRequest(username, from, page);

            if (Response.Contains("lfm status=\"ok"))
            {
                totalpages = Convert.ToInt32(Regex.Match(Response, @"totalPages=.(\d+)").Groups[1].Value);

                MatchCollection match = Regex.Matches(Response, "<track>((.|\n)*?)</track>");
                foreach (Match m in match)
                    ParseTrack(m.Groups[1].Value);
            }
            else
            {
                MessageBox.Show("Error sending the request.", "Error",
                    MessageBoxButtons.OK, MessageBoxIcon.Error);
                return;
            }

            if (page >= totalpages)
                break;

            if (totalpages == 0)
                break;

            if (page < totalpages)
                page++;
        }

问题是last.fm API非常慢,最多可能需要5秒才能获得响应.对于大量页面,加载将花费很长时间.

我想做并行请求,一次说3个并行请求.可能吗？如果是,我该怎么做？

非常感谢.

Answer 1

你可以利用HttpClient,假设你有网址列表:

var client = new HttpClient();
var tasks = urls.Select(url => client.GetAsync(url).ContinueWith(t =>
            {
                var response = t.Result;
                response.EnsureSuccessStatusCode();

                //Do something more
            }));

如果使用异步方法,则可以等待所有任务完成,如下所示:

var results = await Task.WhenAll(tasks);