加速sql INSERT

Question

我有以下方法将数百万行数据插入表中(我使用SQL 2008),它似乎很慢,有没有办法加速INSERT？

这是代码片段 - 我使用MS企业库

        public void InsertHistoricData(List<DataRow> dataRowList)
        {
            string sql = string.Format( @"INSERT INTO [MyTable] ([Date],[Open],[High],[Low],[Close],[Volumn])
                VALUES( @DateVal, @OpenVal, @High, @Low, @CloseVal, @Volumn )");

            DbCommand dbCommand = VictoriaDB.GetSqlStringCommand( sql );
            DB.AddInParameter(dbCommand, "DateVal", DbType.Date);
            DB.AddInParameter(dbCommand, "OpenVal", DbType.Currency);
            DB.AddInParameter(dbCommand, "High", DbType.Currency );
            DB.AddInParameter(dbCommand, "Low", DbType.Currency);
            DB.AddInParameter(dbCommand, "CloseVal", DbType.Currency);
            DB.AddInParameter(dbCommand, "Volumn", DbType.Int32);

            foreach (NasdaqHistoricDataRow dataRow in dataRowList)
            {
                DB.SetParameterValue( dbCommand, "DateVal", dataRow.Date );
                DB.SetParameterValue( dbCommand, "OpenVal", dataRow.Open );
                DB.SetParameterValue( dbCommand, "High", dataRow.High );
                DB.SetParameterValue( dbCommand, "Low", dataRow.Low );
                DB.SetParameterValue( dbCommand, "CloseVal", dataRow.Close );
                DB.SetParameterValue( dbCommand, "Volumn", dataRow.Volumn );

                DB.ExecuteNonQuery( dbCommand );
            }
        }

Answer 1

请考虑使用批量插入.

SqlBulkCopy允许您使用来自其他源的数据有效地批量加载SQL Server表.SqlBulkCopy类可用于仅将数据写入SQL Server表.但是,数据源不仅限于SQL Server; 只要数据可以加载到DataTable实例或使用IDataReader实例读取,就可以使用任何数据源.对于此示例,该文件将包含大约1000条记录,但此代码可以处理大量数据.

此示例首先创建一个DataTable并用数据填充它.这保存在内存中.

DataTable dt = new DataTable();
string line = null;
bool firstRow = true;

using (StreamReader sr = File.OpenText(@"c:\temp\table1.csv"))
{  
      while ((line = sr.ReadLine()) != null)
      {
            string[] data = line.Split(',');
            if (data.Length > 0)
            {
                  if (firstRow)
                  {
                        foreach (var item in data)
                        {
                              dt.Columns.Add(new DataColumn());
                        }
                        firstRow = false;
                   }

                   DataRow row = dt.NewRow();
                   row.ItemArray = data;
                   dt.Rows.Add(row);
             }
      }
}

然后我们DataTable一次性推送到服务器.

using (SqlConnection cn = new SqlConnection(ConfigurationManager.ConnectionStrings["ConsoleApplication3.Properties.Settings.daasConnectionString"].ConnectionString))
{
      cn.Open();
      using (SqlBulkCopy copy = new SqlBulkCopy(cn))
      {
            copy.ColumnMappings.Add(0, 0);
            copy.ColumnMappings.Add(1, 1);
            copy.ColumnMappings.Add(2, 2);
            copy.ColumnMappings.Add(3, 3);
            copy.ColumnMappings.Add(4, 4);
            copy.DestinationTableName = "Censis";
            copy.WriteToServer(dt);
      }
}