读取文件的最后30,000行

Question

如果有一个csv文件,其数据会不时增加.现在我需要做的是阅读最后30,000行.

代码:

string[] lines = File.ReadAllLines(Filename).Where(r => r.ToString() != "").ToArray();

 int count = lines.Count();

 int loopCount = count > 30000 ? count - 30000 : 0;

  for (int i = loopCount; i < lines.Count(); i++)
  {
      string[] columns = lines[i].Split(',');
      orderList.Add(columns[2]);
  }

它工作正常,但问题是

File.ReadAllLines(Filename)

阅读导致性能不足的完整文件.我想要它只读取最后30,000行迭代整个文件.

PS:我正在使用.Net 3.5..Net 3.5中不存在Files.ReadLines()

Answer 1

您可以使用File.ReadLines()方法而不是使用File.ReadAllLines()

来自 MSDN：File.ReadLines()

The ReadLines and ReadAllLines methods differ as follows:
When you use ReadLines, you can start enumerating the collection of strings before the whole collection is returned; when you use ReadAllLines, you must wait for the whole array of strings be returned before you can access the array.

Therefore, when you are working with very large files, ReadLines can be more efficient.

Solution 1 :

        string[] lines = File.ReadAllLines(FileName).Where(r => r.ToString() != "").ToArray();

        int count = lines.Count();
        List<String> orderList = new List<String>();
        int loopCount = count > 30000 ? 30000 : 0;

        for (int i = count-1; i > loopCount; i--)
        {
            string[] columns = lines[i].Split(',');
            orderList.Add(columns[2]);
        }

Solution 2: if you are using .NET Framework 3.5 as you said in comments below , you can not use File.ReadLines() method as it is avaialble since .NET 4.0 .

You can use StreamReader as below:

        List<string> lines = new List<string>();
        List<String> orderList = new List<String>();
        String line;
        int count=0;
        using (StreamReader reader = new StreamReader("c:\\Bethlehem-Deployment.txt"))
        {
            while ((line = reader.ReadLine()) != null)
            {
                lines.Add(line);
                count++;
            }
        }

        int loopCount = (count > 30000) ? 30000 : 0;

        for (int i = count-1; i > loopCount; i--)
        {
            string[] columns = lines[i].Split(',');
            orderList.Add(columns[0]);
        }