通过多线程下载文件

Question

我想下载一个具有 6 个线程的文件以加快进程，因此这是计算每个线程的文件大小的部分代码：

string url = "http://somefile.mp3";
List<FileDownloader> filewonloadersList = new List<FileDownloader>();
System.Net.WebRequest req = System.Net.HttpWebRequest.Create(url);
var response = req.GetResponse();
req.Method = "HEAD";
System.Net.WebResponse resp = req.GetResponse();
int responseLength = int.Parse(resp.Headers.Get("Content-Length"));
int parts = 6;
var eachSize = responseLength / parts;
var lastPartSize = eachSize + (responseLength % parts);
for (int i = 0; i < parts - 1; i++)
{
    filewonloadersList.Add(new FileDownloader(url, i * eachSize, eachSize));
}
filewonloadersList.Add(new FileDownloader(url, (parts - 1) * eachSize, lastPartSize));
var threads = new List<Thread>();
foreach (var item in filewonloadersList)
{
    var newThread = new Thread(DoDownload);
    threads.Add(newThread);
    newThread.Start(item);
}

这是函数体DoDownload：

public static void DoDownload(object data)
{
    retry:
        try
        {
            var downloader = data as FileDownloader;
            HttpWebRequest req = (HttpWebRequest)WebRequest.Create(downloader.Url);
            if (downloader.Start > 0)
            {
                req.AddRange(downloader.Start, downloader.Start + downloader.Count - 1);
            }
            else
            {
                req.AddRange(downloader.Start, downloader.Start + downloader.Count - 1);
            }
            var response = req.GetResponse();
            using (var reponseStream = response.GetResponseStream())
            {
                using (var fs = new FileStream($"temp_{downloader.Start}.sth", FileMode.OpenOrCreate))
                {
                    var buffer = new byte[1024];
                    int bytesRead = 0;
                    do
                    {
                        bytesRead = reponseStream.Read(buffer, 0, 1024);
                        fs.Write(buffer, 0, bytesRead);
                        fs.Flush();
                    } while (bytesRead > 0);
                    fs.Close();
                }
            }
        }
        catch (WebException e)
        {
            if (e.Status == WebExceptionStatus.Timeout || e.Status == WebExceptionStatus.KeepAliveFailure)
                goto retry;
        }
    }

另外，这是FileDownloader定义：

public class FileDownloader
{
    public int Start;
    public int Count;
    public string PathTemp;
    public string Url;
    public FileDownloader(string url, int start, int count)
    {
        Url = url;
        Start = start;
        Count = count;
    }
}

一切都按照我的预期进行，文件的长度正是应有的长度。此外，在我合并下载文件的部分后，它可以正常工作。问题出在螺纹部分。我预计会同时下载 6 个文件，但它们是一个一个下载的，例如，当第一部分完成时，将下载第二部分。我应该如何解决这个问题？

更新

根据建议，我已将功能更改为async：

public async Task DoDownload(object data)
        {
            retry:
            try
            {
                var downloader = data as FileDownloader;
                HttpWebRequest req = (HttpWebRequest)WebRequest.Create(downloader.Url);
                req.AddRange(downloader.Start, downloader.Start + downloader.Count - 1);
                var response = await req.GetResponseAsync();
                using (var reponseStream = response.GetResponseStream())
                {
                    using (var fs = new FileStream($"temp_{downloader.Start}.sth", FileMode.OpenOrCreate))
                    {
                        var buffer = new byte[1024];
                        int bytesRead = 0;
                        do
                        {
                            //reponseStream.Seek(downloader.Start, SeekOrigin.Current);
                            bytesRead = await reponseStream.ReadAsync(buffer, 0, 1024);
                            await fs.WriteAsync(buffer, 0, bytesRead);
                            await fs.FlushAsync();
                        } while (bytesRead > 0);
                        fs.Close();
                    }
                }
            }
            catch (WebException e)
            {
                if (e.Status == WebExceptionStatus.Timeout || e.Status == WebExceptionStatus.KeepAliveFailure)
                    goto retry;
            }
        }

以及foreach填充部件的循环：

foreach (var item in filewonloadersList)
            {
                Task.WhenAll(DoDownload(item));
            }

但结果是一样的！同时仅下载文件的一部分。

Answer 1

基于此链接尝试并行运行多个HTTP请求，但受到Windows（注册表）的限制。简而言之，这是ServicePoint 的问题。它为 HTTP 连接提供连接管理。ServicePoint 对象允许的默认最大并发连接数为 2。

我只需要在代码中添加一行漂亮的代码：

System.Net.ServicePointManager.DefaultConnectionLimit = 1000;

它就像一个魅力！同时存在多个文件，并且async和Thread的工作方式没有区别。至少他们都达到了我想要的结果。