任务不是并行运行的

Question

我想并行运行一些事情，但它所做的只是减慢一切。这是一个示例（不是我的实际问题，但它模拟了它）。

如果我只运行一次迭代，则运行时间约为 2 秒，但当我尝试突然运行 10 个并行实例时，每个实例的运行时间约为 20 秒。

await RunParallel();

private async Task RunParallel()
{
    var tasks = Enumerable.Range(0, 1)
        .AsParallel()
        .Select(async x =>
        {
            AppLogger.WriteInfo($"task {x}");
            await DoSomeWork();
        })
        .ToList();

    await Task.WhenAll(tasks);
}

private Task DoSomeWork()
{
    return Task.Run(() =>
    {
        Stopwatch workWatch = Stopwatch.StartNew();

        string input = "Just some random string to hash it";
        for (int i = 0; i < 1000000; i++)
        {
            using (SHA1 sha1 = SHA1.Create())
            {
                var hash = sha1.ComputeHash(Encoding.UTF8.GetBytes(input));
                var sb = new StringBuilder(hash.Length * 2);

                foreach (byte b in hash)
                {
                    // can be "x2" if you want lowercase
                    sb.Append(b.ToString("X2"));
                }

                input = sb.ToString();
            }
        }

        workWatch.Stop();

        AppLogger.WriteInfo($"Work took {workWatch.Elapsed.TotalSeconds}s");
    });
}

如果我将哈希代码替换为

while (workWatch.ElapsedMilliseconds < 10000)
    Thread.SpinWait(1000);

或者Task.Delay它按预期工作。这里发生了什么？

Answer 1

您基本上是在此处分析分配器和垃圾收集器。你有如此多的分配——中间字符串、到处都是新数组……这基本上是 GC 地狱，并且添加更多的工作线程会加剧这种情况，使线程相互竞争而不是一起工作。

解决办法很简单：不要这样做- 请注意分配不是免费的，并且在这样的紧密循环中：可能是一场灾难。但有很多方法可以避免这些分配。

您还使用同步并行 API 来调用任务，然后执行非异步操作。选择一条车道：你的工作是同步的；只需使用Parallel.For.

另外：尽可能通过重用对象来摊销分配。

结果先：

（注意：这里仍然有一些分配，因此非线性堆栈 - 为了保持兼容性，我仍然分配string循环中每个哈希的最终值，所以每个工作线程有 1M 个字符串，40 个字符，所以 80 个字节（加上对象等） - 所以每个工作人员超过 80MiB；不平凡，但与原始分配相比仍然没有什么string；如果我们愿意，这些分配也可以避免）

Running 2 workers...
Work took 0.2348809s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 0.2348424s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Running 4 workers...
Work took 0.2335012s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 0.2368322s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 0.2386911s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 0.2539417s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Running 6 workers...
Work took 0.2652288s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 0.2692273s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 0.2695841s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 0.2697559s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 0.2763701s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 0.2781451s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Running 8 workers...
Work took 0.2808767s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 0.2811291s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 0.2835578s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 0.2965003s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 0.3052418s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 0.3064866s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 0.3156862s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 0.3189329s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Running 10 workers...
Work took 0.3520882s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 0.3547991s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 0.3604287s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 0.3637762s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 0.3688235s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 0.3697951s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 0.3775134s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 0.3854601s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 0.3885827s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 0.3956756s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA

与您的修改版本相比（丢失内容Task并写入最终哈希）：

Running 2 workers...
Work took 1.2838492s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 1.3045196s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Running 4 workers...
Work took 1.543938s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 1.5462838s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 1.5626509s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 1.5651681s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Running 6 workers...
Work took 3.6812531s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 3.7369614s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 3.7422675s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 3.7640644s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 3.7649982s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 3.7662394s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Running 8 workers...
Work took 7.7063103s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 7.7559415s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 7.7966256s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 7.7967291s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 7.7971053s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 7.7995026s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 7.8031556s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 7.8117826s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Running 10 workers...
Work took 6.640342s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 6.8570689s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 7.0663749s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 7.0665148s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 7.0766348s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 7.0905743s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 7.0986777s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 7.1038984s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 7.1090672s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 7.1132156s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA

代码：

Running 2 workers...
Work took 0.2348809s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 0.2348424s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Running 4 workers...
Work took 0.2335012s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 0.2368322s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 0.2386911s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 0.2539417s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Running 6 workers...
Work took 0.2652288s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 0.2692273s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 0.2695841s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 0.2697559s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 0.2763701s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 0.2781451s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Running 8 workers...
Work took 0.2808767s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 0.2811291s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 0.2835578s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 0.2965003s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 0.3052418s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 0.3064866s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 0.3156862s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 0.3189329s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Running 10 workers...
Work took 0.3520882s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 0.3547991s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 0.3604287s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 0.3637762s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 0.3688235s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 0.3697951s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 0.3775134s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 0.3854601s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 0.3885827s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA
Work took 0.3956756s; final hash: 5C1AEFCEF5374768A470C2707AFD2A5AC404CDEA