写入内存映射文件比非内存映射文件慢

Question

我正在尝试使用内存映射文件来编写具有较高IO需求的应用程序。在此应用程序中，我收到的数据突发量比磁盘能够支持的速度快。为了避免在应用程序中缓冲逻辑，我考虑过使用内存映射文件。有了这种文件，我只需在映射到文件的内存中写（比磁盘可以支持的速度更快），然后操作系统最终会将这些数据刷新到磁盘。因此，操作系统正在为我做缓冲。

经过实验后，我发现使用内存映射文件可以更快地写入内存，但是刷新磁盘的速度比普通文件要慢。这就是导致我得出这个结论的原因。这是一段代码，可以尽可能快地将其写入非内存映射文件中：

    private static void WriteNonMemoryMappedFile(long fileSize, byte[] bufferToWrite)
    {
        Console.WriteLine(" ==> Non memory mapped file");

        string normalFileName = Path.Combine(Path.GetTempPath(), "MemoryMappedFileWriteTest-NonMmf.bin");
        if (File.Exists(normalFileName))
        {
            File.Delete(normalFileName);
        }

        var stopWatch = Stopwatch.StartNew();
        using (var file = File.OpenWrite(normalFileName))
        {
            var numberOfPages = fileSize/bufferToWrite.Length;

            for (int page = 0; page < numberOfPages; page++)
            {
                file.Write(bufferToWrite, 0, bufferToWrite.Length);
            }
        }

        Console.WriteLine("Non-memory mapped file is now closed after {0} seconds ({1} MB/s)", stopWatch.Elapsed.TotalSeconds, GetSpeed(fileSize, stopWatch));
    }

此代码导致：

==> Non memory mapped file
Non-memory mapped file is now closed after 10.5918587 seconds (966.687541390441 MB/s)

如您所见，我的磁盘非常快。这将是我对内存映射文件的基准。

现在，我尝试使用不安全的代码将相同的数据写入内存映射文件（因为这是我打算在应用程序中执行的操作）：

    [DllImport("msvcrt.dll", EntryPoint = "memcpy", CallingConvention = CallingConvention.Cdecl, SetLastError = false)]
    public static extern IntPtr memcpy(IntPtr dest, IntPtr src, UIntPtr count);

    private static unsafe void WriteMemoryMappedFileWithUnsafeCode(long fileSize, byte[] bufferToWrite)
    {
        Console.WriteLine(" ==> Memory mapped file with unsafe code");

        string fileName = Path.Combine(Path.GetTempPath(), "MemoryMappedFileWriteTest-MmfUnsafeCode.bin");
        if (File.Exists(fileName))
        {
            File.Delete(fileName);
        }

        string mapName = Guid.NewGuid().ToString();

        var stopWatch = Stopwatch.StartNew();
        using (var memoryMappedFile = MemoryMappedFile.CreateFromFile(fileName, FileMode.Create, mapName, fileSize, MemoryMappedFileAccess.ReadWrite))
        using (var view = memoryMappedFile.CreateViewAccessor(0, fileSize, MemoryMappedFileAccess.Write))
        {
            unsafe
            {
                fixed (byte* pageToWritePointer = bufferToWrite)
                {
                    byte* pointer = null;
                    try
                    {
                        view.SafeMemoryMappedViewHandle.AcquirePointer(ref pointer);

                        var writePointer = pointer;

                        var numberOfPages = fileSize/bufferToWrite.Length;

                        for (int page = 0; page < numberOfPages; page++)
                        {
                            memcpy((IntPtr) writePointer, (IntPtr) pageToWritePointer, (UIntPtr) bufferToWrite.Length);
                            writePointer += bufferToWrite.Length;
                        }
                    }
                    finally
                    {
                        if (pointer != null)
                            view.SafeMemoryMappedViewHandle.ReleasePointer();
                    }
                }
            }

            Console.WriteLine("All bytes written in MMF after {0} seconds ({1} MB/s). Will now close MMF. This may be long since some content may not have been flushed to disk yet.", stopWatch.Elapsed.TotalSeconds, GetSpeed(fileSize, stopWatch));
        }

        Console.WriteLine("File is now closed after {0} seconds ({1} MB/s)", stopWatch.Elapsed.TotalSeconds, GetSpeed(fileSize, stopWatch));
    }

然后我得到这个：

==> Memory mapped file with unsafe code
All bytes written in MMF after 6.5442406 seconds (1564.73302033172 MB/s). Will now close MMF. This may be long since some content may not have been flushed to disk yet.
File is now closed after 18.8873186 seconds (542.162704287661 MB/s)

如您所见，这要慢得多。它写入约56％的非内存映射文件。

然后我尝试了另一件事。我尝试使用ViewStreamAccessor而不是不安全的代码：

    private static unsafe void WriteMemoryMappedFileWithViewStream(long fileSize, byte[] bufferToWrite)
    {
        Console.WriteLine(" ==> Memory mapped file with view stream");
        string fileName = Path.Combine(Path.GetTempPath(), "MemoryMappedFileWriteTest-MmfViewStream.bin");
        if (File.Exists(fileName))
        {
            File.Delete(fileName);
        }

        string mapName = Guid.NewGuid().ToString();

        var stopWatch = Stopwatch.StartNew();
        using (var memoryMappedFile = MemoryMappedFile.CreateFromFile(fileName, FileMode.Create, mapName, fileSize, MemoryMappedFileAccess.ReadWrite))
        using (var viewStream = memoryMappedFile.CreateViewStream(0, fileSize, MemoryMappedFileAccess.Write))
        {
            var numberOfPages = fileSize / bufferToWrite.Length;

            for (int page = 0; page < numberOfPages; page++)
            {
                viewStream.Write(bufferToWrite, 0, bufferToWrite.Length);
            }                

            Console.WriteLine("All bytes written in MMF after {0} seconds ({1} MB/s). Will now close MMF. This may be long since some content may not have been flushed to disk yet.", stopWatch.Elapsed.TotalSeconds, GetSpeed(fileSize, stopWatch));
        }

        Console.WriteLine("File is now closed after {0} seconds ({1} MB/s)", stopWatch.Elapsed.TotalSeconds, GetSpeed(fileSize, stopWatch));
    }

然后我得到这个：

==> Memory mapped file with view stream
All bytes written in MMF after 4.6713875 seconds (2192.06548076352 MB/s). Will now close MMF. This may be long since some content may not have been flushed to disk yet.
File is now closed after 16.8921666 seconds (606.198141569359 MB/s)

再一次，这比非内存映射文件要慢得多。

那么，有谁知道写时如何使内存映射文件和非内存映射文件一样快？

顺便说一下，这是我测试程序的其余部分：

    static void Main(string[] args)
    {
        var bufferToWrite = Enumerable.Range(0, Environment.SystemPageSize * 256).Select(i => (byte)i).ToArray();
        long fileSize = 10 * 1024 * 1024 * 1024L; // 2 GB

        WriteNonMemoryMappedFile(fileSize, bufferToWrite);
        WriteMemoryMappedFileWithUnsafeCode(fileSize, bufferToWrite);
        WriteMemoryMappedFileWithViewStream(fileSize, bufferToWrite);
    }

    private static double GetSpeed(long fileSize, Stopwatch stopwatch)
    {
        var mb = fileSize / 1024.0 / 1024.0;
        var mbPerSec = mb / stopwatch.Elapsed.TotalSeconds;
        return mbPerSec;
    }

编辑1：

根据usr的建议，我尝试使用SequenctialScan选项。不幸的是，它没有任何影响。这是我所做的更改：

        using (var file = new FileStream(fileName, FileMode.CreateNew, FileAccess.ReadWrite, FileShare.None, 4096, FileOptions.SequentialScan))
        using (var memoryMappedFile = MemoryMappedFile.CreateFromFile(file, mapName, fileSize, MemoryMappedFileAccess.ReadWrite, null, HandleInheritability.None, leaveOpen: false))

Answer 1

来自SDK 文档：

未映射视图中的已修改页面不会写入磁盘，直到其共享计数达到零，或者换句话说，直到它们从共享页面的所有进程的工作集中取消映射或修剪。即使这样，修改后的页面也会“延迟”写入磁盘；也就是说，修改可能会缓存在内存中并稍后写入磁盘。为了最大限度地减少断电或系统崩溃时数据丢失的风险，应用程序应使用 FlushViewOfFile 函数显式刷新修改的页面。

.NET 程序员认真对待最后一句话，您调用的MemoryMappedViewStream.Dispose() 方法实际上调用了 FlushViewOfFile()。这需要时间，您会在个人资料结果中看到这一点。从技术上讲，可以绕过此调用，不调用 Dispose() 并让终结器关闭视图句柄。

FileStream 不会对文件执行等效操作 (FlushFileBuffers)，因此您可以充分利用从文件系统缓存到磁盘的延迟写入的优势。在 Dispose() 调用之后很久发生，您的程序无法观察到。