优化C#文件IO

Question

场景 - 150MB文本文件,它是旧电子邮件帐户的导出收件箱.需要解析并从特定用户中提取电子邮件并将其写入新的单个文件.我有适用的代码,它只是顽强的慢.

我正在使用标记字符串来搜索从原始文件开始/结束副本的位置.

这是主要功能:

 StreamReader sr = new StreamReader("c:\\Thunderbird_Inbox.txt");
        string working = string.Empty;
        string mystring = string.Empty;
        while (!sr.EndOfStream)
        {
            while ((mystring = sr.ReadLine()) != null)
            {
                if (mystring == strBeginMarker)
                {
                    writeLog(mystring);

                    //read the next line
                    working = sr.ReadLine();

                        while( !(working.StartsWith(strEndMarker)))
                        {
                            writeLog(working);
                            working = sr.ReadLine();

                        }
                  }
            }

        }
        this.Text = "DONE!!";
        sr.Close();

将所选消息写入新文件的函数:

  public void writeLog(string sMessage)
    {
            fw = new System.IO.StreamWriter(path, true);
            fw.WriteLine(sMessage);
            fw.Flush();
            fw.Close();
    }

同样,这个过程也有效.我得到了一个很好的输出文件,它只需要很长时间,我确信有办法让它更快.

Answer 1

最大的优化是更改writeLog方法,在此操作开始时打开文件一次,多次写入,然后在结束时关闭它.

现在,你在每次迭代时打开和关闭文件,这肯定会减慢速度.

请尝试以下方法:

// Open this once at the beginning!
using(fw = new System.IO.StreamWriter(path, true))
{
    using(StreamReader sr = new StreamReader("c:\\Thunderbird_Inbox.txt"))
    {
        string working;
        string mystring;
        while ((mystring = sr.ReadLine()) != null)
        {
           if (mystring == strBeginMarker)
           {
                writeLog(mystring);

                //read the next line
                working = sr.ReadLine();

                while( !(working.StartsWith(strEndMarker)))
                {
                    fw.WriteLine(working);
                    working = sr.ReadLine();
                }
            }
        }
    }
}
this.Text = "DONE!!";