相对于正确配置的同步，异步模型是否真的在吞吐量方面带来了好处？

Question

每个人都知道，异步为您提供了“更好的吞吐量”，“可伸缩性”，并且在资源消耗方面更加高效。在下面进行实验之前，我也曾想过这种（简单的）方法。它基本上表明，如果我们考虑到异步代码的所有开销，并将其与正确配置的同步代码进行比较，则几乎不会产生性能/吞吐量/资源消耗方面的优势。

The question: Does asynchronous code actually perform so much better comparing to the synchronous code with correctly configured thread pool? May be my performance tests are flawed in some dramatic way?

Test setup: Two ASP.NET Web API methods with JMeter trying to call them with 200 threads thread group (30 seconds rump up time).

[HttpGet]
[Route("async")]
public async Task<string> AsyncTest()
{
    await Task.Delay(_delayMs);

    return "ok";
}

[HttpGet]
[Route("sync")]
public string SyncTest()
{
    Thread.Sleep(_delayMs);

    return "ok";
}

Here is response time (log scale). Notice how synchronous code becomes faster when Thread Pool injected enough threads. If we were to set up Thread Pool beforehand (via SetMinThreads) it would outperform async right from the start.

What about resources consumption you would ask. "Thread has big cost in terms of CPU time scheduling, context switching and RAM footprint". Not so fast. Threads scheduling and context switching is efficient. As far as the stack usage goes thread does not instantly consume the RAM, but rather just reserve virtual address space and commit only a tiny fraction which is actually needed.

Let's look at what the data says. Even with bigger amount threads sync version has smaller memory footprint (working set which maps into the physical memory).

UPDATE. I want to post the results of follow-up experiment which should be more representational since avoids some biases of the first one.

First of all, the results of the first experiment are taken using IIS Express, which is basically dev time server, so I needed to move away from that. Also, considering the feedback I've isolated load generation machine from the server (two Azure VMs in the same network). I've also discovered that some IIS threading limits are from hard to impossible to violate and ended up switching to ASP.NET WebAPI self-hosting to eliminate IIS from the variables as well. Note that memory footprints/CPU times are radically different with this test, please do not compare numbers across the different test runs as setups are totally different (hosting, hardware, machines setup). Additionally, when I moved to another machines and another hosting solution the Thread Pool strategy changed (it is dynamic) and injection rate increased.

Settings: Delay 100ms, 200 JMeter "users", 30 sec ramp-up time.

I want to conclude these experiments with the following: Yes, under some specific (more laboratory like) circumstances it's possible to get comparable results for sync vs. async, but in real world cases where workload can not be 100% predictable and workload is uneven we inevitably will hit some kind of threading limits: either server side limits, or Thread Pool grow limits (and bear in mind that thread pool management is automatic mechanism with not always easily predictable properties). Additionally, sync version does have a bigger memory footprint (both working set, and way bigger virtual memory size). As far as CPU consumption is concerned async also wins (CPU time per request metric).

On IIS with default settings the situation is even more dramatic: synchronous version is order(s) of magnitude slower (and smaller throughput) due to quite tight limit on threads count - 20 per CPU.

PS. Do use asynchronous pipelines for IO! [... sigh of relief...]

Answer 1

每个人都知道异步为您提供“更好的吞吐量”、“可扩展性”和更高效的资源消耗。

可扩展性，是的。吞吐量：这取决于。每个异步请求都比等效的同步请求慢，因此您只会在可扩展性发挥作用时看到吞吐量优势（即，请求数多于可用线程数）。

与正确配置线程池的同步代码相比，异步代码实际上表现得更好吗？

好吧，那里的问题是“正确配置的线程池”。您假设的是您可以 1) 预测您的负载，以及 2) 拥有一个足够大的服务器来处理每个请求使用一个线程。对于许多（大多数？）现实世界的生产场景，这两种情况中的一个或两个都不正确。

来自我关于异步 ASP.NET 的文章：

为什么不增加线程池的大小[而不是使用异步]？答案是双重的：与阻塞线程池线程相比，异步代码扩展得更远也更快。

首先，异步代码比同步代码更能扩展。通过更真实的示例代码，ASP.NET 服务器（经过压力测试）的总体可扩展性显示出成倍增加。换句话说，异步服务器可以处理数倍于同步服务器的连续请求数量（两个线程池都达到该硬件的最大值）。然而，这些实验（不是我做的）是在平均 ASP.NET 应用程序的预期“现实基线”上完成的。我不知道相同的结果如何延续到 noop 字符串返回。

其次，异步代码比同步代码扩展得更快。这是很明显的；同步代码可以很好地扩展到线程池线程的数量，但不能比线程注入速率更快。因此，如响应时间图的开头所示，您对突然的重负载做出了非常缓慢的响应。

我认为你所做的工作很有趣；我对内存使用差异（或者更确切地说，缺乏差异）感到特别惊讶。我很想看到你把它写成一篇博文。建议：

• 使用 ASP.NET Core 进行测试。旧的 ASP.NET 只有一个部分异步的管道；ASP.NET Core 对于同步与异步的更“纯”比较是必要的。
• 不要在本地测试；这样做时有很多注意事项。我建议选择 VM 大小（或单实例 Docker 容器或其他）并在云中测试可重复性。
• 除了负载测试外，还可以尝试压力测试。继续增加负载，直到服务器完全不堪重负，看看异步和同步服务器如何响应。

最后提醒一下（也来自我的文章）：

请记住，异步代码不会替换线程池。这不是线程池或异步代码；它是线程池和异步代码。异步代码使您的应用程序能够最佳地利用线程池。它使用现有的线程池并将其变为 11。