重构 - System.OutOfMemoryException

Question

在尝试生成一些HTML报告时,我收到了'System.OutOfMemoryException'.

我如何重新考虑这个因素,以便将其缓冲到文件中,而不是将其全部读取到内存中,然后写入文件.

在数据表中有超过2000条记录,并且它已经在2000行内存不足.

DetailsUpdateTemplate包含多行html代码段.我假设我正在创建一个很大的字符串.

我正在使用C#,.NET 3.5

    internal static String SaveARSUpdateHTML(DataTable table, string fileName)
    {
        int recordCount = table.Rows.Count;

        Dictionary<String, object> templateCols = new Dictionary<string, object>();

        templateCols["Track"] = table.TableName;
        templateCols["ProdDate"] = DateTime.Now.ToShortDateString();
        templateCols["ProdTime"] = DateTime.Now.ToShortTimeString();
        templateCols["TotalRecords"] = recordCount;

        String detailOutput = String.Empty;
        for (int i = 0; i < table.Rows.Count; i++)
        {
            int ResultID = i + 1;
            DataRow row = table.Rows[i];
            String ReportDetails = DetailsUpdateTemplate;
            ReportDetails = ReportDetails.Replace(String.Format("{{{0}}}", "ResultID"), ResultID.ToString());
            foreach (DataColumn column in table.Columns)
            {
                String value = row[column.ColumnName].ToString();
                if (column.ColumnName.Equals("TF"))
                {
                    String display = value.Equals("no", StringComparison.CurrentCultureIgnoreCase) ? "none" : "block";
                    ReportDetails = ReportDetails.Replace(String.Format("{{{0}}}", "SuppressNewAddr"), display);
                }

                ReportDetails = ReportDetails.Replace(String.Format("{{{0}}}", column.ColumnName), value);
            } 

            detailOutput += ReportDetails;
        }

        templateCols["ReportDetails"] = detailOutput;

        String masterOut = MasterUpdateTemplate;
        foreach (KeyValuePair<string, object> pair in templateCols)
        {

            masterOut = masterOut.Replace(String.Format("{{{0}}}", pair.Key), pair.Value.ToString());
        }

        String outputFile = String.Format("{0}.htm", fileName);
        using (StreamWriter sw = new StreamWriter(outputFile))
        {
            sw.Write(masterOut);
        }

        return outputFile;
    }

Answer 1

连接字符串超过几次时使用StringBuilders.

特别是这里:

detailOutput += ReportDetails;

还使用DataReader.DataReader将返回支持IDataRecord的记录,IDataRecord具有与DataRow类似的接口.

当我在ASP.NET中使用DataTables和DataSets时,你正在使用大约相同的行数来处理OOM,其中工作进程的内存有限,然后才会因为过多的内存使用而获得回收.这就是为什么我一直在积极地切换到DataReaders.

更新:

DataReader解决方案看起来像......(TableReaders存在,但它不会在内存保护方面为你买任何东西,你只需要一个与DataReaders更相似的接口)

internal static String SaveARSUpdateHTML(DbDataReader myReader, string fileName)
    {

if (myReader.HasRows)
  while (myReader.Read())
  { 
      object something = myReader["TF"];
  }

else
  Console.WriteLine("No rows returned.");

myReader.Close();