内存缓存.Net 4.0性能测试:惊人的结果

Question

此性能测试是错误的还是系统缓存正在以卓越的性能运行？

这是我的结果:
[13]交互次数100000:63毫秒
[14]交互次数100000:139毫秒
[12]交互次数100000:47毫秒
[15]交互次数100000:44毫秒
测试结束.

硬件:x86 Family 6 Model 23 Stepping GenuineIntel~2992 Mhz 3.327 MB,5.1.2600 Service Pack 3

using System;
using System.Collections.Generic;
using System.Runtime.Caching;
using System.Diagnostics;
using System.Threading;

namespace CacheNet40
{
    public class CacheTest
    {
        private ObjectCache cache;

        public CacheTest()
        {
            cache = MemoryCache.Default;
        }

        public void AddItem(CacheItem item, double span)
        {
            CacheItemPolicy cp = new CacheItemPolicy();
            cp.SlidingExpiration.Add(TimeSpan.FromMinutes(span));

            cache.Add(item, cp);
        }
        public Object GetItem(string key)
        {
            return cache.Get(key);
        }
    }

    class Program
    {        
        private static CacheTest Cache = new CacheTest();
        private static string allowedChars = "abcdefghijkmnopqrstuvwxyzABCDEFGHJKLMNOPQRSTUVWXYZ0123456789!@$?_-";
        private static int counter = 0;
        private static readonly object locker = new object();

        static string CreateRandomString(int passwordLength, int idx)
        {            
            char[] chars = new char[passwordLength];
            Random rd = new Random((int)DateTime.Now.Ticks + idx);

            for (int i = 0; i < passwordLength; i++)
            {
                chars[i] = allowedChars[rd.Next(0, allowedChars.Length)];
            }
            return new string(chars);
        }

        private static void CacheAccessTes()
        {
            int span = 5;
            string key;
            string data;
            int itens = 1000;
            int interactions = 100000;
            int cont = 0;
            int index = 0;
            List<string> keys = new List<string>();

            lock (locker)
            {
                counter++;
            }

            cont = itens;

            //populates it with data in the cache
            do
            {                
                key = CreateRandomString(127, Thread.CurrentThread.ManagedThreadId + cont);
                keys.Add(key);

                data = CreateRandomString(156000, Thread.CurrentThread.ManagedThreadId + cont + 1);

                CacheItem ci = new CacheItem(key, data);
                Cache.AddItem(ci, span);

                cont--;
            }
            while (cont > 0);

            cont = interactions;
            index = 0;

            //test readings
            Stopwatch stopWatch = new Stopwatch();
            stopWatch.Start();            
            do
            {
                Object ci = Cache.GetItem(keys[index]);

                ci = null;
                index++;
                if (index == itens)
                {
                    index = 0;
                }

                cont--;
            }
            while (cont > 0);
            stopWatch.Stop();

            lock (locker)
            {
                counter--;
            }

            string outstring = String.Format("[{0}] number of interactions {1} : {2} milliseconds", Thread.CurrentThread.ManagedThreadId, interactions, stopWatch.ElapsedMilliseconds );
            Console.WriteLine(outstring);
        }

        static void Main(string[] args)
        {
            for (int threads = 0; threads < 4; threads++)
            {
                Thread thread = new Thread(new ThreadStart(CacheAccessTes));
                thread.Start();
            }

            Thread.Sleep(1000);

            while (true)
            {
                lock (locker)
                {
                    if (counter == 0) break;
                }
                Thread.Sleep(100);
            }

            Console.WriteLine("End of test.");
            Console.ReadLine();
        }
    }
}

Answer 1

我刚刚在网上搜索了有关MemoryCache性能的信息，并偶然发现了这个 SO 问题。我问自己为什么没有使用合适的基准测试库，所以我非常懒惰（所有优秀的程序员都应该这样做：-）并使用了令人难以置信的BenchmarkDotNet库来检查如何好（或没有）这个类的行为。

先上结果

BenchmarkDotNet=v0.11.5, OS=Windows 10.0.17134.706 (1803/April2018Update/Redstone4)
Intel Core i5-8250U CPU 1.60GHz (Kaby Lake R), 1 CPU, 8 logical and 4 physical cores
Frequency=1757813 Hz, Resolution=568.8887 ns, Timer=TSC
  [Host]     : .NET Framework 4.7.2 (CLR 4.0.30319.42000), 64bit RyuJIT-v4.7.3394.0
  DefaultJob : .NET Framework 4.7.2 (CLR 4.0.30319.42000), 64bit RyuJIT-v4.7.3394.0



|                         Method |     N |      Mean |     Error |    StdDev |
|------------------------------- |------ |----------:|----------:|----------:|
|            FindDadosEmpInCache | 30000 | 231.40 ns | 0.4435 ns | 0.3703 ns |
|               FindDataAtTheEnd | 30000 | 429.90 ns | 1.1490 ns | 1.0186 ns |
|           FindDataInDictionary | 30000 |  24.09 ns | 0.2244 ns | 0.2099 ns |
| FindDataInConcurrentDictionary | 30000 |  29.66 ns | 0.0990 ns | 0.0926 ns |
|              FindDataInHashset | 30000 |  16.25 ns | 0.0077 ns | 0.0065 ns |

现在一些解释...

我最感兴趣的是看看与具有数千个条目的MemoryCache散列列表 ( Dictionary, Hashset...)相比有多快，以及在如此“长”的列表上进行最坏情况的线性搜索。所以我添加了一些额外的测试，并意识到虽然MemoryCache没有简单列表或并发列表那么快，但速度仍然处于纳秒级。在 30,000 长的缓存项目列表中检索项目甚至不需要一毫秒。

公平地说，MemoryCache它比那些简单的列表要多得多，因为它必须控制并发性、项目过期/驱逐等。我相信它对于各种工作负载来说都足够快，但是如果您不需要它的附加功能，例如驱逐策略，你最好坚持使用更简单的哈希列表实现。

另一方面，由于它比哈希查找“慢”一个数量级，因此可能有改进的空间。我猜设计师认为它已经足够好了，我有什么资格反对 DOTNET 工程师？:-)

这是基准程序的源代码：

using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;
using System;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.Linq;
using System.Runtime.Caching;

namespace TestListPerformance
{
    class Program
    {
        static void Main(string[] args) {
            var summary = BenchmarkRunner.Run<BenchmarkMemoryCache>();
        }
    }

    public class BenchmarkMemoryCache
    {
        [Params(30000)]
        public int N { get; set; }
        public string FindStr;
        private IList<DadosEmp> data;
        private Dictionary<string, DadosEmp> dict;
        private ConcurrentDictionary<string, DadosEmp> concurrentDict;
        private HashSet<DadosEmp> hashset;
        private DadosEmp last;

        [GlobalSetup]
        public void BuildData() {
            FindStr = N.ToString();
            data = new List<DadosEmp>(N);
            dict = new Dictionary<string, DadosEmp>(N);
            concurrentDict = new ConcurrentDictionary<string, DadosEmp>();
            hashset = new HashSet<DadosEmp>();
            for (int i = 0; i <= N; i++) {
                DadosEmp d;
                data.Add(d = new DadosEmp {
                    Identificacao = i,
                    Pis = i * 100,
                    NumCartao = i * 1000,
                    Nome = "Nome " + i.ToString(),
                });
                MemoryCache.Default.Add(i.ToString(), d, 
                    new CacheItemPolicy { SlidingExpiration = TimeSpan.FromMinutes(30) });
                dict.Add(i.ToString(), d);
                concurrentDict.TryAdd(i.ToString(), d);
                hashset.Add(d);
                last = d;
            }
        }
        [Benchmark]
        public DadosEmp FindDadosEmpInCache() {
            var f = (DadosEmp)MemoryCache.Default.Get(FindStr);
            return f;
        }
        [Benchmark]
        public DadosEmp FindDataAtTheEnd() {
            var f = data.FirstOrDefault(e => e.NumCartao == N || e.Pis == N || e.Identificacao == N);
            return f;
        }
        [Benchmark]
        public DadosEmp FindDataInDictionary() {
            var f = dict[FindStr];
            return f;
        }
        [Benchmark]
        public DadosEmp FindDataInConcurrentDictionary() {
            var f = concurrentDict[FindStr];
            return f;
        }
        [Benchmark]
        public bool FindDataInHashset() {
            return hashset.Contains(last);
        }

    }

    public class DadosEmp : IEquatable<DadosEmp> 
    {
        public const string BIO_EXCLUSAO = "xbio";

        public DadosEmp() {
            Biometrias = new List<string>();
        }
        public long Identificacao { get; set; }
        public long Pis { get; set; }
        public long NumCartao { get; set; }
        public string Nome { get; set; }
        public int VersaoBio { get; set; }
        public string Unidade { get; set; }
        public IList<string> Biometrias { get; set; }
        public string Biometria { get; set; } 
        public bool ExcluirBiometria { get { return Biometria == BIO_EXCLUSAO; } }
        public DateTime DataEnvioRep { get; set; } 
        public string SenhaTeclado { get; set; }
        public bool ExigeAutorizacaoSaida { get; set; }
        public bool BioRepPendente { get; set; }
        public override bool Equals(object obj) {
            DadosEmp e = obj as DadosEmp;
            if (ReferenceEquals(e, null))
                return false;
            return Equals(e);
        }
        public bool Equals(DadosEmp e) {
            if (ReferenceEquals(e, null))
                return false;
            return e.Pis == this.Pis;
        }
        public override int GetHashCode() {
            return Pis.GetHashCode();
        }
        public override string ToString() {
            return string.Format("{0} ({1} - {2})", Nome, Pis, Identificacao);
        }
    }
}

Answer 2

看起来不错.虽然低于一秒的时间不是很可靠; 你可能遇到了垃圾收集,你的电脑可能会做一些其他事情,第一次JIT编译等.

所以增加计数.这也应该使每个线程的结果更加接近.

我上周做的一些测试使得每秒800万次迭代(不是很多,但仍然是)单线程.所以是的,PC现在很快;-)

Answer 3

问题是StopWatch类不能在多核机器上使用!(我假设你有一个多核CPU)当一个线程从一个核心移动到另一个核心时,BIOS处理该计数器的方式(甚至一个线程应用程序跳转核心!).

编辑:
退房 - http://msdn.microsoft.com/en-us/library/windows/desktop/ms644904(v=vs.85).aspx - 特别是备注部分.还有一个stackoverflow帖子 - 多核和线程感知.Net秒表？. 结束编辑

我已经搜索了从高到低的最佳方法来衡量应用程序性能,而我提出的最可靠的方法是DateTime.UtcNow.获取开始和结束时间,然后区分它们.你必须循环你的代码足以超越低精度,但我遇到的其他方法没有提供更可靠的准确性.