提高性能异步 Parallel.Foreach

Question

我有一个超过 10k 项的 deviceList，想通过调用另一种方法发送数据。

我尝试使用 Parallel.Foreach 但我不确定这是正确的方法。

我已经在 azure 上发布了这个 webapp，我已经测试了 100 次它工作正常，但是对于 10k 次它有超时问题。我的实现是否需要任何调整，谢谢

private List<Task> taskEventList = new List<Task>();
public async Task ProcessStart()
{
    string messageData = "{\"name\":\"DemoData\",\"no\":\"111\"}";
    RegistryManager registryManager;

    Parallel.ForEach(deviceList, async (device) =>
    {
        // get details for each device and use key to send message
        device = await registryManager.GetDeviceAsync(device.DeviceId);
        SendMessages(device.DeviceId, device.Key, messageData);
    });

    if (taskEventList.Count > 0)
    {
        await Task.WhenAll(taskEventList);
    }
}

private void SendMessages(string deviceId, string Key, string messageData)
{
    DeviceClient deviceClient = DeviceClient.Create(hostName, new DeviceAuthenticationWithRegistrySymmetricKey(deviceId, deviceKey), Microsoft.Azure.Devices.Client.TransportType.Mqtt);
    //created separate Task
    var taskEvents = Task.Run(() => ProcessMessages(deviceId, string messageData));
    taskEventList.Add(taskEvents);
}

private async Task ProcessMessages(string deviceId, string messageData)
{
    var startTime = DateTime.UtcNow;
    while (DateTime.UtcNow - startTime < TimeSpan.FromMinutes(15))
    {
        await deviceClient.SendEventAsync(messageData);
    }
}

Answer 1

至少肯定存在竞争条件。Parallel是只用于同步代码，而不是异步。

据我所知，您不需要Parallelor Task.Run（它们都是 ASP.NET 服务的反模式）：

public async Task ProcessStart()
{
  string messageData = "{\"name\":\"DemoData\",\"no\":\"111\"}";
  RegistryManager registryManager;

  var tasks = deviceList.Select(async device =>
  {
    // get details for each device and use key to send message
    device = await registryManager.GetDeviceAsync(device.DeviceId);
    await SendMessagesAsync(device.DeviceId, device.Key, messageData);
  }).ToList();

  await Task.WhenAll(tasks);
}

private async Task SendMessagesAsync(string deviceId, string Key, string messageData)
{
  DeviceClient deviceClient = DeviceClient.Create(hostName, new DeviceAuthenticationWithRegistrySymmetricKey(deviceId, deviceKey), Microsoft.Azure.Devices.Client.TransportType.Mqtt);
  await ProcessMessagesAsync(deviceId, string messageData);
}

private async Task ProcessMessagesAsync(string deviceId, string messageData)
{
  var startTime = DateTime.UtcNow;
  while (DateTime.UtcNow - startTime < TimeSpan.FromMinutes(15))
  {
    await deviceClient.SendEventAsync(messageData);
  }
}

对于 10k，它有超时问题。

15 分钟对于 HTTP 请求来说是很长的时间。我认为值得后退一步，看看是否有更好的方法来构建整个系统。