mongoimport在导入JSON文件时卡在同一点

Question

OS: ubuntu 14.04 LTS
Mongo version: rs0:PRIMARY> db.version()
    3.0.7
Storage engine: wiredTiger.

通过以下命令导入JSON文件(每行13GB,1个JSON文档):

$ mongoimport --db InfDB --collection SFTest --file InfMapRed.json

这个命令曾经在2.6中使用以前的存储引擎工作正常但现在没有超过0.2%.该程序不断反复打印下面的行.该集合通过.count()显示1000条记录.

    2015-10-24T06:11:41.799+0000    connected to: localhost
    2015-10-24T06:11:44.788+0000    [........................] InfDB.SFTest       20.5 MB/13.0 GB (0.2%)
    2015-10-24T06:11:47.788+0000    [........................] InfDB.SFTest       20.5 MB/13.0 GB (0.2%)
    2015-10-24T06:11:50.788+0000    [........................] InfDB.SFTest       20.5 MB/13.0 GB (0.2%)
    2015-10-24T06:11:53.788+0000    [........................] InfDB.SFTest       20.5 MB/13.0 GB (0.2%)
    2015-10-24T06:11:56.788+0000    [........................] InfDB.SFTest       20.5 MB/13.0 GB (0.2%)
    2015-10-24T06:11:59.788+0000    [........................] InfDB.SFTest       20.5 MB/13.0 GB (0.2%)
    2015-10-24T06:12:02.788+0000    [........................] InfDB.SFTest       20.5 MB/13.0 GB (0.2%)
    2015-10-24T06:12:05.788+0000    [........................] InfDB.SFTest       20.5 MB/13.0 GB (0.2%)
    2015-10-24T06:12:08.788+0000    [........................] InfDB.SFTest       20.5 MB/13.0 GB (0.2%)
    2015-10-24T06:12:11.788+0000    [........................] InfDB.SFTest       20.5 MB/13.0 GB (0.2%)
    2015-10-24T06:12:14.788+0000    [........................] InfDB.SFTest       20.5 MB/13.0 GB (0.2%)
    2015-10-24T06:12:17.788+0000    [........................] InfDB.SFTest       20.5 MB/13.0 GB (0.2%)
    2015-10-24T06:12:20.788+0000    [........................] InfDB.SFTest       20.5 MB/13.0 GB (0.2%)
    2015-10-24T06:12:23.788+0000    [........................] InfDB.SFTest       20.5 MB/13.0 GB (0.2%)
....

写了一个简单的python脚本来读取文件并顺利插入文档,并且工作正常.

Answer 1

使用较小的批量大小解决了这个问题。

mongoimport --db InfDB --collection SFTest --文件 InfMapRed.json --batchSize 100

这在导入大型文档时很有用，默认批量大小为 10000。