需要加速转换时间戳的C#应用​​程序

Question

我很难让一个小应用程序更快地工作.我不是开发人员,我花了一些时间让这个工作原样.任何人都可以提供任何建议或替代代码来加快这个过程,它需要大约1小时来处理10米的输入文件.

下面列出了代码,下面是输入文件的示例.

4401,imei:0000000000,2012-09-01 12:12:12.9999

using System;
using System.Globalization;
using System.IO;

class Sample 
{
    public static void Main(string[] args) 
    {
if (args.Length == 0)
                {
                    return;
                }


                using (FileStream stream = File.Open(args[0], FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
                {

                    using (StreamReader streamReader = new StreamReader(stream))
                    {
                        System.Text.StringBuilder builder = new System.Text.StringBuilder();
                        while (!streamReader.EndOfStream)
                        {
                            var line = streamReader.ReadLine();
                            var values = line.Split(',');
                            DateTime dt = new DateTime();
                            DateTime.TryParse(values[2], out dt);
                            values[2] = Convert.ToString(dt.Ticks);

                            string[] output = new string[values.Length];
                            bool firstColumn = true;
                            for (int index = 0; index < values.Length; index++)
                            {
                                if (!firstColumn)
                                    builder.Append(',');
                                builder.Append(values[index]);
                                firstColumn = false;

                            }
                            File.WriteAllText(args[1], builder.AppendLine().ToString());

                        }
                    }
                }
            }
}

Answer 1

最大的性能影响是,每次读取一行时,整个文件(到目前为止处理)都会写回磁盘.要快速获胜,请尝试将StringBuilder移出循环:

            System.Text.StringBuilder builder = new System.Text.StringBuilder();

            using (FileStream stream = File.Open(args[0], FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
            {
                using (StreamReader streamReader = new StreamReader(stream))
                {
                    while (!streamReader.EndOfStream)
                    {
                        var line = streamReader.ReadLine();
                        var values = line.Split(',');
                        DateTime dt = new DateTime();
                        DateTime.TryParse(values[2], out dt);
                        values[2] = Convert.ToString(dt.Ticks);

                        string[] output = new string[values.Length];
                        bool firstColumn = true;
                        for (int index = 0; index < values.Length; index++)
                        {
                            if (!firstColumn)
                                builder.Append(',');
                            builder.Append(values[index]);
                            firstColumn = false;
                        }
                        builder.AppendLine();
                    }
                }
            }

            File.WriteAllText(args[1], builder.ToString());

如果要重构,请进一步更改逗号分隔逻辑:

            System.Text.StringBuilder builder = new System.Text.StringBuilder();

            using (FileStream stream = File.Open(args[0], FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
            {
                using (StreamReader streamReader = new StreamReader(stream))
                {
                    while (!streamReader.EndOfStream)
                    {
                        var line = streamReader.ReadLine();
                        var values = line.Split(',');
                        DateTime dt = new DateTime();
                        DateTime.TryParse(values[2], out dt);
                        values[2] = Convert.ToString(dt.Ticks);

                        builder.AppendLine(string.Join(",", values));
                    }
                }
            }

            File.WriteAllText(args[1], builder.ToString());

编辑:为避免内存使用,请删除Stringbuilder并使用另一个FileStream写入磁盘.您提出的解决方案(使用a List)仍将使用大量内存,并且可能会破坏较大的文件:

        using (FileStream input = File.Open(args[0], FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
        using (FileStream output = File.Create(args[1]))
        {
            using (StreamReader streamReader = new StreamReader(input))
            using (StreamWriter streamWriter = new StreamWriter(output))
            {
                while (!streamReader.EndOfStream)
                {
                    var line = streamReader.ReadLine();
                    var values = line.Split(',');
                    DateTime dt = new DateTime();
                    DateTime.TryParse(values[2], out dt);
                    values[2] = Convert.ToString(dt.Ticks);

                    streamWriter.WriteLine(string.Join(",", values));
                }
            }
        }