Django - 复杂的查询

Question

假设我有两个模型:

class Profile(models.Model):
    #some fields here

class Ratings(models.Model):
    profile = models.ForeignKey(profile)
    category = models.IntegerField()
    points = models.IntegerField()

假设遵循MySQL表"评级":

profile    |    category    |    points
   1                1               10
   1                1               4
   1                2               10
   1                3               0
   1                4               10
   1                4               10
   1                4               10
   1                5               0

我的POST数据以及其他字段值中包含以下值:

category_1_avg_val = 7
category_2_avg_val = 5
category_3_avg_val = 5
category_4_avg_val = 7
category_5_avg_val = 9

我想过滤具有针对高于或等于所需值的类别计算的平均评级的配置文件.

一些过滤器最初应用为:

q1 = [('associated_with', search_for),
      ('profile_type__slug__exact', profile_type),
      ('gender__in', gender),
      ('rank__in', rank),
      ('styles__style__in', styles),
      ('age__gte', age_from),
      ('age__lte', age_to)]
q1_list = [Q(x) for x in q1 if x[1]]

q2 = [('user__first_name__icontains', search_term),
      ('user__last_name__icontains', search_term),
      ('profile_type__name__icontains', search_term),
      ('styles__style__icontains', search_term),
      ('rank__icontains', search_term)]
q2_list = [Q(x) for x in q2 if x[1]]

if q1_list:
    objects = Profile.objects.filter(
        reduce(operator.and_, q1_list))

if q2_list:
    if objects:
        objects = objects.filter(
            reduce(operator.or_, q2_list))
    else:
        objects = Profile.objects.filter(
            reduce(operator.or_, q2_list))

if order_by_ranking_level == 'desc':
    objects = objects.order_by('-ranking_level').distinct()
else:
    objects = objects.order_by('ranking_level').distinct()

现在我想过滤其(平均点数)(按类别分组)> =(后期类别的平均值)的配置文件

我试图一个接一个地这样做

objects = objects.filter(
    ratings__category=1) \
    .annotate(avg_points=Avg('ratings__points'))\
    .filter(avg_points__gte=category_1_avg_val)


objects = objects.filter(
    ratings__category=2) \
    .annotate(avg_points=Avg('ratings__points'))\
    .filter(avg_points__gte=category_2_avg_val)

但我认为这是错误的.请帮帮我.如果return是一个很棒的查询集.

编辑 使用hynekcer我发布的答案我提出了略有不同的解决方案,因为我已经查询了需要根据评级过滤的配置文件.

def check_ratings_avg(pr, rtd):
    ok = True
    qr = Ratings.objects.filter(profile__id=pr.id) \
        .values('category')\
        .annotate(points_avg=Avg('points'))
    qr = {i['category']:i['points_avg'] for i in qr}

    for cat in rtd:
        val = rtd[cat]
        if qr[cat] >= val:
            pass
        else:
            ok = False
            break
    return ok


rtd = {1: category_1_avg_val, 2: category_2_avg_val, 3: category_3_avg_val,
       4: category_4_avg_val, 5: category_5_avg_val}
objects = [i for i in objects if check_ratings_avg(i, rtd)]

Answer 1

您的复杂查询需要原则中的子查询.可能的解决方案是

• 一个子查询的书面"额外"的查询集方法或原始的SQL查询.它不是DRY,它不受某些db后端的支持,例如某些版本的MySQL,但是从Django 1.1开始,子查询的使用方式有限.
• 将中间结果保存到数据库中的临时表中.这在Django并不好.
• 在Python中循环仿真外部查询.最好的通用解决方案.Python中的第一个查询聚合的数据库数据循环可以足够快地聚合和过滤数据.

A)Python模拟的子查询

from django.db.models import Q, Avg
from itertools import groupby
from myapp.models import Profile, Ratings

def iterator_filtered_by_average(dictionary):
    qr = Ratings.objects.values('profile', 'category', 'points').order_by(
            'profile', 'category').annotate(points_avg=Avg('points'))
    f = Q()
    for k, v in dictionary.iteritems():
        f |= Q(category=k, points_avg__gte=v)
    for profile, grp in groupby(qr.filter(f).values('profile')):
        if len(list(grp)) == len(dictionary):
            yield profile

#example
FILTER_DATA = {1:category_1_avg_val, 2:category_2_avg_val, 3:category_3_avg_val,
               4:category_4_avg_val, 5:category_5_avg_val}
for row in iterator_filtered_by_average(FILTER_DATA):
    print row

这是原始问题的简单解决方案,无需后续的额外要求.

B)子查询的解决方案:
对于更详细的问题版本是必要的,因为如果初始过滤器基于某个类型的字段,ManyToManyField并且因为它包含一个distinct子句:

# objects:  QuerySet that you get from your initial filters. Not yet executed.
if rtd:
    # Method `as_nested_sql` removes the `order_by` clase, unlike `as_sql`
    subquery3 = objects.values('id').query \
            .get_compiler(connection=connection).as_nested_sql()
    subquery2 = ("""SELECT profile_id, category, avg(points) AS points_avg
          FROM myapp_ratings
          WHERE profile_id in
          ( %s
          ) GROUP BY profile_id, category
            """ % subquery3[0], subquery3[1]
    )
    where_sql = ' OR '.join(
            'category = %d AND points_avg >= %%s' % cat for cat in rtd.keys()
    )
    subquery = (
        """SELECT profile_id
        FROM
        ( %s
        ) subquery2
        WHERE %s
        GROUP BY profile_id
        HAVING count(*) = %s
        """ % (subquery2[0], where_sql, len(rtd)),
        subquery2[1] + tuple(rtd.values())
    )
    assert order_by_ranking_level in ('asc', 'desc')
    mainquery = ("""SELECT myapp_profile.* FROM myapp_profile
      INNER JOIN
      ( %s
      ) subquery ON subquery.profile_id=myapp_profile.id
      ORDER BY ranking_level %s"""
        % (subquery[0], order_by_ranking_level), subquery[1]
    )
    objects = Profile.objects.raw(mainquery[0], params=mainquery[1])
return objects

请更换所有字符串myapp通过name_of_your_application.

此代码生成的SQL示例

SELECT myapp_profile.* FROM myapp_profile
  INNER JOIN
  ( SELECT profile_id
    FROM
    ( SELECT profile_id, category, avg(points) AS points_avg
      FROM myapp_ratings
      WHERE profile_id IN
      ( SELECT U0.`id` FROM `myapp_profile` U0 WHERE U0.`ranking_level` >= 4
      ) GROUP BY profile_id, category
    ) subquery2
    WHERE category = 1 AND points_avg >= 7 OR category = 2 AND points_avg >= 5
       OR category = 3 AND points_avg >= 5 OR category = 4 AND points_avg >= 7
       OR category = 5 AND points_avg >= 9
    GROUP BY profile_id
    HAVING count(*) = 5
  ) subquery ON subquery.profile_id=myapp_profile.id
  ORDER BY ranking_level asc

(这个SQL是为了更好的可读性,手动解析,并且%s参数替换了字符串,但是出于安全原因,数据库引擎会接收未解析的参数.)

您的问题是由于Django生成的子查询支持不多.只有更复杂查询文档中的示例才会创建子查询.(例如,aggregate之后annotate或count之后annotate或aggregate之后distinct,但annotate之后distinct或之后annotate)复杂的嵌套聚合被简化为一个意外的查询.

所有其他为第一个查询过滤的每个对象执行新的单独SQL查询的解决方案都不鼓励生产,尽管它们对于测试任何更好的解决方案的结果非常有用.