HttpClient - 发送一批请求

Question

我想迭代一批请求,使用HttpClient类将它们中的每一个发送到外部API.

  foreach (var MyRequest in RequestsBatch)
  {
            try
            {
                HttpClient httpClient = new HttpClient();
                httpClient.Timeout = TimeSpan.FromMilliseconds(5);
                HttpResponseMessage response = await httpClient.PostAsJsonAsync<string>(string.Format("{0}api/GetResponse", endpoint), myRequest);
                JObject resultResponse = await response.Content.ReadAsAsync<JObject>();
            }
            catch (Exception ex)
            {
                continue;
            }
 }

这里的上下文是我需要设置一个非常小的超时值,所以如果响应花费的时间超过了那个时间,我们只需得到"任务被取消"异常并继续迭代.

现在,在上面的代码中,注释这两行:

                HttpResponseMessage response = await httpClient.PostAsJsonAsync<string>(string.Format("{0}api/GetResponse", endpoint), myRequest);
                resultResponse = await response.Content.ReadAsAsync<JObject>();

迭代结束得非常快.取消注释,然后重试.这需要很多时间.

我想知道用await调用PostAsJsonAsync/ReadAsAsync方法比超时值花费更多时间吗？

根据下面的答案,假设它将创建不同的线程,我们有这个方法:

  public Task<JObject> GetResponse(string endPoint, JObject request, TimeSpan timeout)
    {
        return Task.Run(async () =>
        {
            try
            {
                HttpClient httpClient = new HttpClient();
                httpClient.Timeout = TimeSpan.FromMilliseconds(5);
                HttpResponseMessage response = await httpClient.PostAsJsonAsync<string>(string.Format("{0}api/GetResponse", endPoint), request).WithTimeout<HttpResponseMessage>(timeout);
                JObject resultResponse = await response.Content.ReadAsAsync<JObject>().WithTimeout<JObject>(timeout);
                return resultResponse;
            }
            catch (Exception ex)
            {
                return new JObject() { new JProperty("ControlledException", "Invalid response. ")};
            }
        });
    }

在那里引发异常并且应该非常快地返回JObject异常,但是,如果使用httpClient方法,即使它引发异常也需要花费很多时间.是否存在影响Task的幕后处理,即使返回值是一个简单的异常JObject？

如果是,可以使用哪种方法以非常快的方式向API发送一批请求？

Answer 1

我同意接受的答案,因为加快速度的关键是并行运行请求.但是任何通过使用I/O绑定异步操作而使用Task.Run或者Parallel.ForEach没有获得任何效率的强制其他线程进入混合的解决方案.如果它有任何损害.

您可以轻松地同时运行所有调用,同时让底层异步子系统决定尽可能高效地完成任务所需的线程数.有可能该数字远小于并发调用的数量,因为它们在等待响应时根本不需要任何线程.

此外,接受的答案HttpClient为每个呼叫创建一个新实例.不要那么做 - 坏事可能发生.

以下是已接受答案的修改版本:

var httpClient = new HttpClient {
    Timeout = TimeSpan.FromMilliseconds(5)
};

var taskList = new List<Task<JObject>>();

foreach (var myRequest in RequestsBatch)
{
    // by virtue of not awaiting each call, you've already acheived parallelism
    taskList.Add(GetResponseAsync(endPoint, myRequest));
}

try
{
    // asynchronously wait until all tasks are complete
    await Task.WhenAll(taskList.ToArray());
}
catch (Exception ex)
{
}

async Task<JObject> GetResponseAsync(string endPoint, string myRequest)
{
    // no Task.Run here!
    var response = await httpClient.PostAsJsonAsync<string>(
        string.Format("{0}api/GetResponse", endpoint), 
        myRequest);
    return await response.Content.ReadAsAsync<JObject>();
}

Answer 2

看起来您实际上并没有为每个请求运行单独的线程。尝试这样的事情：

var taskList = new List<Task<JObject>>();

foreach (var myRequest in RequestsBatch)
{
    taskList.Add(GetResponse(endPoint, myRequest));
}

try
{
    Task.WaitAll(taskList.ToArray());
}
catch (Exception ex)
{
}

public Task<JObject> GetResponse(string endPoint, string myRequest)
{
    return Task.Run(() =>
        {
            HttpClient httpClient = new HttpClient();

            HttpResponseMessage response = httpClient.PostAsJsonAsync<string>(
                 string.Format("{0}api/GetResponse", endpoint), 
                 myRequest, 
                 new CancellationTokenSource(TimeSpan.FromMilliseconds(5)).Token);

            JObject resultResponse = response.Content.ReadAsAsync<JObject>();
        });
}