为什么我的聚合超过了最大文档大小？

Question

我有一个名为的集合roles和另一个名为的集合subjects.每个主题都有一个名为的字段role_id,其中包含该主题角色的ID.我正在尝试使用此查询计算每个角色的主题数:

db.roles.aggregate([
    {"$lookup" : { 
        "from": "subjects", 
        "localField": "_id",
        "foreignField": "role_id",
        "as": "subject_matches"
    } },
    {"$project": { "_id": 1, "total_matches": {"$size": "$subject_matches"} } },
    {"$out": "testing"}
], {allowDiskUse: true} )

当我这样做时,我收到以下错误:

assert: command failed: {
    "ok" : 0,
    "errmsg" : "Total size of documents in subjects matching { role_id: { $eq: ObjectId('55421027b2fb3e916f0001e9') } } exceeds maximum document size",
    "code" : 4568
} : aggregate failed
_getErrorWithCode@src/mongo/shell/utils.js:25:13
doassert@src/mongo/shell/assert.js:13:14
assert.commandWorked@src/mongo/shell/assert.js:267:5
DBCollection.prototype.aggregate@src/mongo/shell/collection.js:1312:5
@(shell):1:1

我不知道为什么我得到这个 - 根据Mongo文档,结果大小限制仅适用于返回的文档,并且在管道处理期间可以超出此大小限制(https://docs.mongodb.com/manual/core/aggregation-pipeline-limits /).由于我的返回文档只包含_id和total_matches字段,似乎我不应该收到此错误.

当我在我的集​​合的一小部分上执行此操作时,它可以工作,结果如下所示:

`{ "_id" : ObjectId("55421026b2fb3e916f000001"), "total_matches" : 208 }
{ "_id" : ObjectId("55421026b2fb3e916f000002"), "total_matches" : 2 }
{ "_id" : ObjectId("55421026b2fb3e916f000003"), "total_matches" : 11 }
{ "_id" : ObjectId("55421026b2fb3e916f000004"), "total_matches" : 0 }
{ "_id" : ObjectId("55421026b2fb3e916f000005"), "total_matches" : 87 }`

对此为何的任何想法？

Answer 1

我最终解决了这个问题,首先将subject_id计数从主题集合中获取到一个新的role_counts集合中:

db.subjects.aggregate([
    {"$group": {
        "_id": "$role_id", 
        "count": {"$sum": 1},
    }},
    {"$out": "role_counts"}
])

然后从角色到role_counts进行查找:

db.roles.aggregate([    
    {"$lookup" : { 
        "from": "role_counts", 
        "localField": "_id",
        "foreignField": "_id",
        "as": "role_matches"
        }
    },
    {"$unwind": {"path": "$role_matches", "preserveNullAndEmptyArrays": true}},
    {"$project": {
        "count": "$role_matches.count", "_id": 1
    }}
])

这避免了导致文档大小限制问题的长数组.

感谢你的帮助!