Node.JS Mongo DB驱动程序没有拆分批量插入？

Question

我正在尝试collection.insert使用Node.JS的标准Mongo DB驱动程序通过一次调用插入大约100k文档:

var MongoClient = require('mongodb').MongoClient;

MongoClient.connect('mongodb://localhost/testdb', function(err, db) {
    var collection = db.collection('testcollection');
    var docs = [];

    var doc = {
        str: 'Lorem ipsum dolor sit amet, consectetur adipiscing elit. Etiam sit amet urna consequat quam pharetra sagittis vitae at nulla. Suspendisse non felis sollicitudin, condimentum urna eu, congue massa. Nam arcu dui, sodales eget auctor nec, ullamcorper in turpis. Praesent sit amet purus mi. Mauris egestas sapien magna, a mattis tellus luctus et. Suspendisse potenti. Nam posuere neque at vulputate ornare. Nunc mollis lorem est, at porttitor augue sodales sed. Ut dui sapien, fermentum eu laoreet sed, sodales et augue. Aliquam erat volutpat.'
    };

    for (var i = 0; i < 100000; i++) {
        docs[i] = doc;
    }

    collection.insert(docs, function(err) {
        throw err;
    });
});

但是,我收到以下错误:

/var/node/testproject/node_modules/mongodb/lib/mongodb/connection/base.js:242
        throw message;
              ^
Error: Document exceeds maximum allowed bson size of 16777216 bytes
    at InsertCommand.toBinary (/var/node/testproject/node_modules/mongodb/lib/mongodb/commands/insert_command.js:86:11)
    at Connection.write (/var/node/testproject/node_modules/mongodb/lib/mongodb/connection/connection.js:230:42)
    at __executeInsertCommand (/var/node/testproject/node_modules/mongodb/lib/mongodb/db.js:1857:14)
    at Db._executeInsertCommand (/var/node/testproject/node_modules/mongodb/lib/mongodb/db.js:1930:5)
    at insertAll (/var/node/testproject/node_modules/mongodb/lib/mongodb/collection/core.js:205:13)
    at Collection.insert (/var/node/testproject/node_modules/mongodb/lib/mongodb/collection/core.js:35:3)
    at /var/node/testproject/dbtest.js:15:16
    at /var/node/testproject/node_modules/mongodb/lib/mongodb/mongo_client.js:431:11
    at process._tickCallback (node.js:664:11)

由于单个文档明显小于16 MB并且给定堆栈跟踪,因此驱动程序似乎不会自动拆分命令.我如何解决这个问题,最好不要自己编码？

Answer 1

我问这些问题是为了澄清你在做什么,并且怀疑文件阵列确实超过了64MB.

您似乎期望这是每个文档的限制,但您没想到的是您的整个请求实际上是BSON文档.

这是mongodb的有线协议的一部分,因此批量请求受到相同的限制,因为您的整个提交的大小不能超过16MB.

如果您查看2.6 文档中的插入和更新操作的runCommand页面,则会明确说明.

所以从本质上讲,这不是一个错误.您需要分解大批量请求,并将它们保持在16MB大小之下.