如何在Django中执行批量插入？

Question

在mysql中,您可以在一个查询中向表中插入多行,其中n> 0:

INSERT INTO tbl_name (a,b,c) VALUES(1,2,3),(4,5,6),(7,8,9), ..., (n-2, n-1, n);

有没有办法用Django查询集方法实现上述目的？这是一个例子:

values = [(1, 2, 3), (4, 5, 6), ...]

for value in values:
    SomeModel.objects.create(first=value[0], second=value[1], third=value[2])

我相信以上是为for循环的每次迭代调用插入查询.我正在寻找单个查询,这在Django中可能吗？

Answer 1

这些答案已经过时了.bulk_create已被带入Django 1.4:

https://docs.djangoproject.com/en/dev/ref/models/querysets/#bulk-create

Answer 2

我最近自己寻找这样的东西(受QuerySet.update()启发,我想你也是如此).据我所知,当前的生产框架中没有批量创建(截至今天的1.1.1).我们最终为需要批量创建的模型创建了一个自定义管理器,并在该管理器上创建了一个函数,以使用VALUES参数序列构建适当的SQL语句.

喜欢的东西(道歉,如果这不起作用......希望我已经从我们的代码中运行了这个):

from django.db import models, connection

class MyManager(models.Manager):

    def create_in_bulk(self, values):
        base_sql = "INSERT INTO tbl_name (a,b,c) VALUES "
        values_sql = []
        values_data = []

        for value_list in values:
            placeholders = ['%s' for i in range(len(value_list))]
            values_sql.append("(%s)" % ','.join(placeholders))
            values_data.extend(value_list)

        sql = '%s%s' % (base_sql, ', '.join(values_sql))

        curs = connection.cursor()
        curs.execute(sql, values_data)

class MyObject(models.Model):
    # model definition as usual... assume:
    foo = models.CharField(max_length=128)

    # custom manager
    objects = MyManager()

MyObject.objects.create_in_bulk( [('hello',), ('bye',), ('c', )] )

这种方法确实存在特定于特定数据库的风险.在我们的例子中,我们希望函数返回刚刚创建的ID,因此我们在函数中有一个特定于postgres的查询,以从表示对象的表的主键序列生成必需数量的ID.也就是说,它在测试中表现得更好,而不是迭代数据并发出单独的QuerySet.create()语句.

Answer 3

这是批量插入仍然通过Django的ORM的方法(因此保留了ORM提供的许多好处).这种方法包括继承InsertQuery类,以及创建一个自定义管理器,用于准备插入数据库的模型实例,与Django的save()方法使用的方式非常相似.下面的BatchInsertQuery类的大多数代码都是直接来自InsertQuery类,只添加或修改了几个关键行.要使用batch_insert方法,请传入要插入数据库的一组模型实例.这种方法可以释放视图中的代码,而不必担心将模型实例转换为有效的SQL值.manager类与BatchInsertQuery类一起处理它.

from django.db import models, connection
from django.db.models.sql import InsertQuery

class BatchInsertQuery( InsertQuery ):

    ####################################################################

    def as_sql(self):
        """
        Constructs a SQL statement for inserting all of the model instances
        into the database.

        Differences from base class method:        

        - The VALUES clause is constructed differently to account for the
        grouping of the values (actually, placeholders) into
        parenthetically-enclosed groups. I.e., VALUES (a,b,c),(d,e,f)
        """
        qn = self.connection.ops.quote_name
        opts = self.model._meta
        result = ['INSERT INTO %s' % qn(opts.db_table)]
        result.append('(%s)' % ', '.join([qn(c) for c in self.columns]))
        result.append( 'VALUES %s' % ', '.join( '(%s)' % ', '.join( 
            values_group ) for values_group in self.values ) ) # This line is different
        params = self.params
        if self.return_id and self.connection.features.can_return_id_from_insert:
            col = "%s.%s" % (qn(opts.db_table), qn(opts.pk.column))
            r_fmt, r_params = self.connection.ops.return_insert_id()
            result.append(r_fmt % col)
            params = params + r_params
        return ' '.join(result), params

    ####################################################################

    def insert_values( self, insert_values ):
        """
        Adds the insert values to the instance. Can be called multiple times
        for multiple instances of the same model class.

        Differences from base class method:

        -Clears self.columns so that self.columns won't be duplicated for each
        set of inserted_values.        
        -appends the insert_values to self.values instead of extends so that
        the values (actually the placeholders) remain grouped separately for
        the VALUES clause of the SQL statement. I.e., VALUES (a,b,c),(d,e,f)
        -Removes inapplicable code
        """
        self.columns = [] # This line is new

        placeholders, values = [], []
        for field, val in insert_values:
            placeholders.append('%s')

            self.columns.append(field.column)
            values.append(val)

        self.params += tuple(values)
        self.values.append( placeholders ) # This line is different

########################################################################

class ManagerEx( models.Manager ):
    """
    Extended model manager class.
    """
    def batch_insert( self, *instances ):
        """
        Issues a batch INSERT using the specified model instances.
        """
        cls = instances[0].__class__
        query = BatchInsertQuery( cls, connection )
        for instance in instances:

             values = [ (f, f.get_db_prep_save( f.pre_save( instance, True ) ) ) \
                 for f in cls._meta.local_fields ]
            query.insert_values( values )

        return query.execute_sql()

########################################################################

class MyModel( models.Model ):
    myfield = models.CharField(max_length=255)
    objects = ManagerEx()

########################################################################

# USAGE:
object1 = MyModel(myfield="foo")
object2 = MyModel(myfield="bar") 
object3 = MyModel(myfield="bam")
MyModels.objects.batch_insert(object1,object2,object3)