use*_*682 7 .net memory concurrency multithreading scalability
关于大内存分配对.Net运行时的可伸缩性的影响,我遇到了一些奇怪的事情.在我的测试应用程序中,我在一个紧密的循环中创建了许多字符串,持续了固定的循环次数,并吐出了每秒循环迭代的速率.当我在几个线程中运行这个循环时,出现了奇怪的现象 - 看起来速率并没有线性增加.创建大字符串时问题会变得更严重.
让我告诉你结果.我的机器是一个运行Windows Server 2008 R1,32位的8GB 8核盒子.它有两个4核Intel Xeon 1.83ghz(E5320)处理器."工作"进行的是一组交替调用的ToUpper(),并ToLower()在一个字符串.我为一个线程,两个线程等运行测试 - 最多.下表中的列是:
第一个例子从一个线程开始,然后是两个线程,最终用八个线程运行测试.每个线程创建10,000个字符串,每个字符串1024个字符:
Creating 10000 strings per thread, 1024 chars each, using up to 8 threads GCMode = Server Rate Linear Rate % Variance Threads -------------------------------------------------------- 322.58 322.58 0.00 % 1 689.66 645.16 -6.90 % 2 882.35 967.74 8.82 % 3 1081.08 1290.32 16.22 % 4 1388.89 1612.90 13.89 % 5 1666.67 1935.48 13.89 % 6 2000.00 2258.07 11.43 % 7 2051.28 2580.65 20.51 % 8 Done.
在第二个例子中,我将每个字符串的字符数增加到32,000.
Creating 10000 strings per thread, 32000 chars each, using up to 8 threads GCMode = Server Rate Linear Rate % Variance Threads -------------------------------------------------------- 14.10 14.10 0.00 % 1 24.36 28.21 13.64 % 2 33.15 42.31 21.66 % 3 40.98 56.42 27.36 % 4 48.08 70.52 31.83 % 5 61.35 84.63 27.51 % 6 72.61 98.73 26.45 % 7 67.85 112.84 39.86 % 8 Done.
注意方差与线性速率的差异; 在第二个表中,实际费率比线性费率低39%.
我的问题是:为什么这个应用程序不能线性扩展？
我最初认为这可能是由于虚假共享,但正如您在源代码中看到的那样,我不会共享任何集合,而且字符串非常大.可能存在的唯一重叠是在一个字符串的开头和另一个字符串的结尾.
我正在使用gcServer enabled = true,以便每个核心获得自己的堆和垃圾收集器线程.
我不认为我分配的对象被发送到大对象堆,因为它们大于85000字节.
我认为由于实际的MSDN,字符串值可能在引擎盖下共享,所以我尝试编译实习禁用.这产生了比上面显示的更糟的结果
我尝试使用小型和大型整数数组的相同示例,其中我遍历每个元素并更改值.它产生类似的结果,跟随更大的分配表现更差的趋势.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading;
using System.Diagnostics;
using System.Runtime;
using System.Runtime.CompilerServices;
namespace StackOverflowExample
{
public class Program
{
private static int columnWidth = 14;
static void Main(string[] args)
{
int loopCount, maxThreads, stringLength;
loopCount = maxThreads = stringLength = 0;
try
{
loopCount = args.Length != 0 ? Int32.Parse(args[0]) : 1000;
maxThreads = args.Length != 0 ? Int32.Parse(args[1]) : 4;
stringLength = args.Length != 0 ? Int32.Parse(args[2]) : 1024;
}
catch
{
Console.WriteLine("Usage: StackOverFlowExample.exe [loopCount] [maxThreads] [stringLength]");
System.Environment.Exit(2);
}
float rate;
float linearRate = 0;
Stopwatch stopwatch;
Console.WriteLine("Creating {0} strings per thread, {1} chars each, using up to {2} threads", loopCount, stringLength, maxThreads);
Console.WriteLine("GCMode = {0}", GCSettings.IsServerGC ? "Server" : "Workstation");
Console.WriteLine();
PrintRow("Rate", "Linear Rate", "% Variance", "Threads"); ;
PrintRow(4, "".PadRight(columnWidth, '-'));
for (int runCount = 1; runCount <= maxThreads; runCount++)
{
// Create the workers
Worker[] workers = new Worker[runCount];
workers.Length.Range().ForEach(index => workers[index] = new Worker());
// Start timing and kick off the threads
stopwatch = Stopwatch.StartNew();
workers.ForEach(w => new Thread(
new ThreadStart(
() => w.DoWork(loopCount, stringLength)
)
).Start());
// Wait until all threads are complete
WaitHandle.WaitAll(
workers.Select(p => p.Complete).ToArray());
stopwatch.Stop();
// Print the results
rate = (float)loopCount * runCount / stopwatch.ElapsedMilliseconds;
if (runCount == 1) { linearRate = rate; }
PrintRow(String.Format("{0:#0.00}", rate),
String.Format("{0:#0.00}", linearRate * runCount),
String.Format("{0:#0.00} %", (1 - rate / (linearRate * runCount)) * 100),
runCount.ToString());
}
Console.WriteLine("Done.");
}
private static void PrintRow(params string[] columns)
{
columns.ForEach(c => Console.Write(c.PadRight(columnWidth)));
Console.WriteLine();
}
private static void PrintRow(int repeatCount, string column)
{
for (int counter = 0; counter < repeatCount; counter++)
{
Console.Write(column.PadRight(columnWidth));
}
Console.WriteLine();
}
}
public class Worker
{
public ManualResetEvent Complete { get; private set; }
public Worker()
{
Complete = new ManualResetEvent(false);
}
public void DoWork(int loopCount, int stringLength)
{
// Build the string
string theString = "".PadRight(stringLength, 'a');
for (int counter = 0; counter < loopCount; counter++)
{
if (counter % 2 == 0) { theString.ToUpper(); }
else { theString.ToLower(); }
}
Complete.Set();
}
}
public static class HandyExtensions
{
public static IEnumerable<int> Range(this int max)
{
for (int counter = 0; counter < max; counter++)
{
yield return counter;
}
}
public static void ForEach<T>(this IEnumerable<T> items, Action<T> action)
{
foreach(T item in items)
{
action(item);
}
}
}
}
<?xml version="1.0" encoding="utf-8" ?>
<configuration>
<runtime>
<gcServer enabled="true"/>
</runtime>
</configuration>
要在您的盒子上运行StackOverflowExample.exe,请使用以下命令行参数调用它:
StackOverFlowExample.exe [loopCount] [maxThreads] [stringLength]
loopCount:每个线程操纵字符串的次数.maxThreads:要前进的线程数.stringLength:填充字符串的字符数.