在内存中使用没有加载库的LinqToSql

Question

我有一个问题.如果我正在使用LinqToSql,我的程序将我的数据库加载到内存中.小例子:

//pageNumber = 1; pageSize = 100;                
                var result =
                    (
                    from a in db.Stats.AsEnumerable()
                    where (DictionaryFilter(a, sourceDictionary) && DateFilter(a, beginTime, endTime) && ErrorFilter(a, WarnLevel))
                    select a
                    );
                var size = result.Count(); // size = 1007
                var resultList = result.Skip((pageNumber-1)*pageSize).Take(pageSize).ToList();
                return resultList;

DictionaryFilter,DateFilter和ErrorFilter是过滤我的数据库的函数.在此之后我的程序使用~250Mb的Ram.如果我不使用:

var size = result.Count(); 

我的程序使用~120MB Ram.在使用此代码之前,我的程序使用~35MB Ram.

如何使用count并将函数加载到内存中的所有数据库？

static bool DateFilter(Stat table, DateTime begin, DateTime end)
{

    if ((table.RecordTime >= begin.ToFileTime()) && (table.RecordTime <= end.ToFileTime()))
    {
        return true;
    }
    return false;
}
static bool ErrorFilter(Stat table, bool[] WarnLevel)
{
    if (WarnLevel[table.WarnLevel]) return true;
    else return false;
}
static bool DictionaryFilter(Stat table, Dictionary<GetSourcesNameResult, bool> sourceDictionary)
{
    foreach (var word in sourceDictionary)
    {
        if (table.SourceName == word.Key.SourceName)
        {
            return word.Value;
        }
    }
    //
    return false;
}

Answer 1

简单:不要使用.AsEnumerable().这意味着"切换到LINQ到对象".在此之前,db.Stats是IQueryable<T>,这是一个组合的 API,并会尽你所期望的.

但是,这意味着你不能使用像DictionaryFilter和那样的C#方法DateFilter,而必须用ExpressionAPI 来组合.如果你能说明他们的所作所为,我可以进一步提出建议.

通过编辑,可以调整过滤,例如:

static IQueryable<Stat> ErrorFilter(IQueryable<Stat> source, bool[] WarnLevel) {
    // extract the enabled indices (match to values)
    int[] levels = WarnLevel.Select((val, index) => new { val, index })
                            .Where(pair => pair.val)
                            .Select(pair => pair.index).ToArray();

    switch(levels.Length)
    {
        case 0:
            return source.Where(x => false);
        case 1:
            int level = levels[0];
            return source.Where(x => x.WarnLevel == level);
        case 2:
            int level0 = levels[0], level1 = levels[1];
            return source.Where(
                  x => x.WarnLevel == level0 || x.WarnLevel == level1);
        default:
            return source.Where(x => levels.Contains(x.WarnLevel));
    }
}

日期过滤器更简单:

static IQueryable<Stat> DateFilter(IQueryable<Stat> source,
      DateTime begin, DateTime end)
{
    var from = begin.ToFileTime(), to = end.ToFileTime();
    return source.Where(table => table.RecordTime >= from
        && table.RecordTime <= to);
}

字典有点像水平:

static IQueryable<Stat> DictionaryFilter(IQueryable<Stat> source,
    Dictionary<GetSourcesNameResult, bool> sourceDictionary)
{
    var words = (from word in sourceDictionary
                 where word.Value
                 select word.Key.SourceName).ToArray();

    switch (words.Length)
    {
        case 0:
            return source.Where(x => false);
        case 1:
            string word = words[0];
            return source.Where(x => x.SourceName == word);
        case 2:
            string word0 = words[0], word1 = words[1];
            return source.Where(
                  x => x.SourceName == word0 || x.SourceName == word1);
        default:
            return source.Where(x => words.Contains(x.SourceName));
    }
}

和:

IQueryable<Stat> result = db.Stats;
result = ErrorFilter(result, WarnLevel);
result = DateFiter(result, beginTime, endTime);
result = DictionaryFilter(result, sourceDictionary);
// etc - note we're *composing* a filter here

var size = result.Count(); // size = 1007
var resultList = result.Skip((pageNumber-1)*pageSize).Take(pageSize).ToList();
return resultList;

关键是我们现在正在使用IQueryable<T>和Expression排他.