Azure Redis缓存-多个错误TimeoutException：执行GET {key}的超时

Question

我们将应用程序部署到Azure。它正在使用Azure Redis缓存，并且我们遇到了很多超时。即：

[TimeoutException: Timeout performing GET textobjectDetails__23290_TextObject, inst: 1, mgr: Inactive, queue: 5, qu=0, qs=5, qc=0, wr=0/0, in=56864/0]

[TimeoutException: Timeout performing GET featured_series_CachedSeries, inst: 1, mgr: Inactive, queue: 4, qu=0, qs=4, qc=0, wr=0/0, in=44470/0]

[TimeoutException: Timeout performing GET SeriesByFranchiseId_1_CachedSeries, inst: 1, mgr: Inactive, queue: 3, qu=0, qs=3, qc=0, wr=0/0, in=11252/0]

[TimeoutException: Timeout performing GET media_silo-1-1-0_Media, inst: 1, mgr: Inactive, queue: 3, qu=0, qs=3, qc=0, wr=0/0, in=15188/0]

[TimeoutException: Timeout performing GET textobjectDetails__3092_TextObject, inst: 3, mgr: Inactive, queue: 7, qu=0, qs=7, qc=0, wr=0/0, in=65536/0]

[TimeoutException: Timeout performing GET textobjectbytype_104__TextObject, inst: 11, mgr: Inactive, queue: 9, qu=0, qs=9, qc=0, wr=0/0, in=65536/0]

[TimeoutException: Timeout performing GET groupnews_2_14_TextObject, inst: 1, mgr: Inactive, queue: 7, qu=0, qs=7, qc=0, wr=0/0, in=65536/0]

[TimeoutException: Timeout performing GET archived_news_by_group_13586_1_TextObject, inst: 2, mgr: Inactive, queue: 7, qu=0, qs=7, qc=0, wr=0/0, in=65536/0]

[TimeoutException: Timeout performing GET textobjectDetails__24404_TextObject, inst: 11, mgr: Inactive, queue: 12, qu=0, qs=12, qc=0, wr=0/0, in=65536/0]

[TimeoutException: Timeout performing GET standings_232_lcds_fixtures, inst: 2, mgr: Inactive, queue: 11, qu=0, qs=11, qc=0, wr=0/0, in=65536/0]

[TimeoutException: Timeout performing GET player_name1099_Player, inst: 4, mgr: Inactive, queue: 11, qu=0, qs=11, qc=0, wr=0/0, in=65536/0]

[TimeoutException: Timeout performing GET groupnews_1_6_TextObject, inst: 4, mgr: Inactive, queue: 9, qu=0, qs=9, qc=0, wr=0/0, in=65536/0]

[TimeoutException: Timeout performing GET archivednewscount__20789_TextObject, inst: 2, mgr: Inactive, queue: 11, qu=0, qs=11, qc=0, wr=0/0, in=65536/0]

[TimeoutException: Timeout performing GET media_id3648_Media, inst: 1, mgr: Inactive, queue: 10, qu=0, qs=10, qc=0, wr=0/0, in=65536/0]

它们的异常内容都是相同的：

StackExchange.Redis.ConnectionMultiplexer.ExecuteSyncImpl[T](Message message, ResultProcessor`1 processor, ServerEndPoint server):509
    StackExchange.Redis.RedisBase.ExecuteSync[T](Message message, ResultProcessor`1 processor, ServerEndPoint server):25
    StackExchange.Redis.RedisDatabase.StringGet(RedisKey key, CommandFlags flags):16
    AB.SiteCaching.Providers.RedisDataSource+<>c__DisplayClasse`1.<RetrieveCacheObject>b__b():0
    Microsoft.Practices.TransientFaultHandling.RetryPolicy.ExecuteAction[TResult](Func`1 func):115
    AB.SiteCaching.Providers.RedisDataSource.RetrieveCacheObject[T](String fullCacheKey):56
    AB.SiteCaching.Providers.RedisDataSource.RetrieveCached[T](String key, Func`1 onNotCached, TimeSpan timeOut):61
    DataAccess.Data.Caching.CachedSeries.GetSeriesByFranchiseId(Int32 franchiseId):64
    Shared.Services.SeriesService.LatestYearByFranchiseId(Int32 franchiseId):0
    AllBlacksdotcom.Controllers.FixturesController._MostRecentYearOfFixtures(Int32 franchiseId):0
    (unknown).lambda_method(Closure , ControllerBase , Object[] ):-1
    System.Web.Mvc.ActionMethodDispatcher.Execute(ControllerBase controller, Object[] parameters):0
    System.Web.Mvc.ReflectedActionDescriptor.Execute(ControllerContext controllerContext, IDictionary`2 parameters):87
    System.Web.Mvc.ControllerActionInvoker.InvokeActionMethod(ControllerContext controllerContext, ActionDescriptor actionDescriptor, IDictionary`2 parameters):0
    System.Web.Mvc.Async.AsyncControllerActionInvoker.<BeginInvokeSynchronousActionMethod>b__39(IAsyncResult asyncResult, ActionInvocation innerInvokeState):0
    System.Web.Mvc.Async.AsyncResultWrapper+WrappedAsyncResult`2.CallEndDelegate(IAsyncResult asyncResult):0
    System.Web.Mvc.Async.AsyncResultWrapper+WrappedAsyncResultBase`1.End():41
    System.Web.Mvc.Async.AsyncControllerActionInvoker.EndInvokeActionMethod(IAsyncResult asyncResult):0
    System.Web.Mvc.Async.AsyncControllerActionInvoker+AsyncInvocationWithFilters.<InvokeActionMethodFilterAsynchronouslyRecursive>b__3d():20
    System.Web.Mvc.Async.AsyncControllerActionInvoker+AsyncInvocationWithFilters+<>c__DisplayClass46.<InvokeActionMethodFilterAsynchronouslyRecursive>b__3f():134
    System.Web.Mvc.Async.AsyncControllerActionInvoker+<>c__DisplayClass33.<BeginInvokeActionMethodWithFilters>b__32(IAsyncResult asyncResult):0
    System.Web.Mvc.Async.AsyncResultWrapper+WrappedAsyncResult`1.CallEndDelegate(IAsyncResult asyncResult):0
    System.Web.Mvc.Async.AsyncResultWrapper+WrappedAsyncResultBase`1.End():41
    System.Web.Mvc.Async.AsyncControllerActionInvoker.EndInvokeActionMethodWithFilters(IAsyncResult asyncResult):0
    System.Web.Mvc.Async.AsyncControllerActionInvoker+<>c__DisplayClass21+<>c__DisplayClass2b.<BeginInvokeAction>b__1c():0
    System.Web.Mvc.Async.AsyncControllerActionInvoker+<>c__DisplayClass21.<BeginInvokeAction>b__1e(IAsyncResult asyncResult):65
    System.Web.Mvc.Async.AsyncResultWrapper+WrappedAsyncResult`1.CallEndDelegate(IAsyncResult asyncResult):0
    System.Web.Mvc.Async.AsyncResultWrapper+WrappedAsyncResultBase`1.End():41
    System.Web.Mvc.Async.AsyncControllerActionInvoker.EndInvokeAction(IAsyncResult asyncResult):0
    System.Web.Mvc.Controller.<BeginExecuteCore>b__1d(IAsyncResult asyncResult, ExecuteCoreState innerState):0
    System.Web.Mvc.Async.AsyncResultWrapper+WrappedAsyncVoid`1.CallEndDelegate(IAsyncResult asyncResult):0
    System.Web.Mvc.Async.AsyncResultWrapper+WrappedAsyncResultBase`1.End():41
    System.Web.Mvc.Controller.EndExecuteCore(IAsyncResult asyncResult):0
    System.Web.Mvc.Controller.<BeginExecute>b__15(IAsyncResult asyncResult, Controller controller):0
    System.Web.Mvc.Async.AsyncResultWrapper+WrappedAsyncVoid`1.CallEndDelegate(IAsyncResult asyncResult):0
    System.Web.Mvc.Async.AsyncResultWrapper+WrappedAsyncResultBase`1.End():41
    System.Web.Mvc.Controller.EndExecute(IAsyncResult asyncResult):0
    System.Web.Mvc.Controller.System.Web.Mvc.Async.IAsyncController.EndExecute(IAsyncResult asyncResult):0
    System.Web.Mvc.MvcHandler.<BeginProcessRequest>b__5(IAsyncResult asyncResult, ProcessRequestState innerState):0
    System.Web.Mvc.Async.AsyncResultWrapper+WrappedAsyncVoid`1.CallEndDelegate(IAsyncResult asyncResult):0
    System.Web.Mvc.Async.AsyncResultWrapper+WrappedAsyncResultBase`1.End():41
    System.Web.Mvc.MvcHandler.EndProcessRequest(IAsyncResult asyncResult):0
    System.Web.Mvc.MvcHandler.System.Web.IHttpAsyncHandler.EndProcessRequest(IAsyncResult result):0
    System.Web.Mvc.HttpHandlerUtil+ServerExecuteHttpHandlerAsyncWrapper+<>c__DisplayClassa.<EndProcessRequest>b__9():0
    System.Web.Mvc.HttpHandlerUtil+ServerExecuteHttpHandlerWrapper+<>c__DisplayClass4.<Wrap>b__3():0
    System.Web.Mvc.HttpHandlerUtil+ServerExecuteHttpHandlerWrapper.Wrap[TResult](Func`1 func):0
    System.Web.Mvc.HttpHandlerUtil+ServerExecuteHttpHandlerWrapper.Wrap(Action action):25
    System.Web.Mvc.HttpHandlerUtil+ServerExecuteHttpHandlerAsyncWrapper.EndProcessRequest(IAsyncResult result):32
    System.Web.HttpServerUtility.ExecuteInternal(IHttpHandler handler, TextWriter writer, Boolean preserveForm, Boolean setPreviousPage, VirtualPath path, VirtualPath filePath, String physPath, Exception error, String queryStringOverride):772

请查看我们的超时设置：

retryTimeoutInMilliseconds = "5000" 
connectionTimeoutInMilliseconds = "5000" 
operationTimeoutInMilliseconds = "1000" 

如何处理这些超时？增大operationTimeoutInMilliseconds会成功吗？我还阅读了有关g-zip压缩的信息，该压缩有助于减少从Redis读取数据所需的时间。

与Redis相关的所有nuget软件包都在其最新版本上。我们正在使用C1版本的Azure Redis缓存（将增加到C2帮助吗？）。

Answer 1

有几点改善了我们的情况：

Protobuf-net代替BinaryFormatter

我建议使用protobuf-net，因为它会减小要存储在缓存中的值的大小。

public interface ICacheDataSerializer
    {
        byte[] Serialize(object o);
        T Deserialize<T>(byte[] stream);
    }

public class ProtobufNetSerializer : ICacheDataSerializer
    {
        public byte[] Serialize(object o)
        {
            using (var memoryStream = new MemoryStream())
            {
                Serializer.Serialize(memoryStream, o);

                return memoryStream.ToArray();
            }
        }

        public T Deserialize<T>(byte[] stream)
        {
            var memoryStream = new MemoryStream(stream);

            return Serializer.Deserialize<T>(memoryStream);
        }
    }

实施重试策略

实现此RedisCacheTransientErrorDetectionStrategy来处理超时问题。

using Microsoft.Practices.TransientFaultHandling;

public class RedisCacheTransientErrorDetectionStrategy : ITransientErrorDetectionStrategy
    {
        /// <summary>
        /// Custom Redis Transient Error Detenction Strategy must have been implemented to satisfy Redis exceptions.
        /// </summary>
        /// <param name="ex"></param>
        /// <returns></returns>
        public bool IsTransient(Exception ex)
        {
            if (ex == null) return false;

            if (ex is TimeoutException) return true;

            if (ex is RedisServerException) return true;

            if (ex is RedisException) return true;

            if (ex.InnerException != null)
            {
                return IsTransient(ex.InnerException);
            }

            return false;
        }
    }

像这样实例化：

private readonly RetryPolicy _retryPolicy;

// CODE
var retryStrategy = new FixedInterval(3, TimeSpan.FromSeconds(2));
            _retryPolicy = new RetryPolicy<RedisCacheTransientErrorDetectionStrategy>(retryStrategy);

像这样使用：

var cachedString = _retryPolicy.ExecuteAction(() => dataCache.StringGet(fullCacheKey));

查看您的代码以最小化缓存调用和您存储在缓存中的值。通过更有效地存储值，我减少了很多错误。

如果没有帮助。移至更高的缓存（我们最终使用C3而不是C1）。