如何创建和写入内存映射文件？

Question

编者注:此代码示例来自1.0之前的Rust版本,它使用的代码在Rust 1.0中不存在.一些答案已经更新,以回答更新版本的Rust的核心问题.

我正在尝试使用创建内存映射文件std::os::MemoryMap.目前的方法如下:

use std::os;
use std::ptr;
use std::old_io as io;
use std::os::unix::prelude::AsRawFd;
use std::os::MapOption;

let path = Path::new("test.mmap");

let f = match io::File::open_mode(&path, io::Open, io::ReadWrite) {
    Ok(f) => f,
    Err(err) => panic!("Could not open file: {}", err),
};

let mmap_opts = &[
    MapOption::MapReadable,
    MapOption::MapWritable,
    MapOption::MapFd(f.as_raw_fd())
];

let mmap = match os::MemoryMap::new(1024*1024, mmap_opts) {
    Ok(mmap) => {
        println!("Successfully created the mmap: {}", mmap.len());
        mmap
    }
    Err(err) => panic!("Could not read the mmap: {}", err),
};

unsafe {
   let data = mmap.data();

    if data.is_null() {
        panic!("Could not access data from memory mapped file")
    }

    let src = "Hello!";
    ptr::copy_memory(data, src.as_ptr(), src.as_bytes().len());
}

这个程序失败了

Process didn't exit successfully: `target/mmap` (status=4)

在ptr::copy_memory对数据进行呼叫或任何其他操作时.

• 我无法写入(或读取)数据的原因是什么MemoryMap？
• MemoryMap在Rust中使用的正确方法是什么？

Answer 1

真正的答案是使用提供此功能的箱子,理想情况是以跨平台的方式.

extern crate memmap;

use std::{
    fs::OpenOptions,
    io::{Seek, SeekFrom, Write},
};

const SIZE: u64 = 1024 * 1024;

fn main() {
    let src = "Hello!";

    let mut f = OpenOptions::new()
        .read(true)
        .write(true)
        .create(true)
        .open("test.mmap")
        .expect("Unable to open file");

    // Allocate space in the file first
    f.seek(SeekFrom::Start(SIZE)).unwrap();
    f.write_all(&[0]).unwrap();
    f.seek(SeekFrom::Start(0)).unwrap();

    let mut data = unsafe {
        memmap::MmapOptions::new()
            .map_mut(&f)
            .expect("Could not access data from memory mapped file")
    };

    data[..src.len()].copy_from_slice(src.as_bytes());
}

请注意,此代码仍可能导致未定义的行为.由于切片由文件支持,因此文件的内容(以及切片)可能会从Rust程序的外部更改,从而打破了unsafe块应该保留的不变量.程序员需要确保文件在地图生命周期内不会发生变化.不幸的是,箱子本身并没有提供太多帮助来防止这种情况发生,甚至没有任何文件警告用户.

如果您希望使用较低级别的系统调用,则缺少两个主要部分:

1. mmap 不会自己分配任何空间,因此您需要在文件中设置一些空间.没有这个,我Illegal instruction: 4在macOS上运行时会得到.

2. MemoryMap(默认情况下)是私有的,因此您需要将映射标记为公共,以便将更改写回文件(我假设您希望保存写入).如果没有这个,代码就会运行,但文件永远不会改变.

这是一个适合我的版本:

extern crate libc;

use std::{
    fs::OpenOptions,
    io::{Seek, SeekFrom, Write},
    os::unix::prelude::AsRawFd,
    ptr,
};

fn main() {
    let src = "Hello!";

    let size = 1024 * 1024;

    let mut f = OpenOptions::new()
        .read(true)
        .write(true)
        .create(true)
        .open("test.mmap")
        .expect("Unable to open file");

    // Allocate space in the file first
    f.seek(SeekFrom::Start(size as u64)).unwrap();
    f.write_all(&[0]).unwrap();
    f.seek(SeekFrom::Start(0)).unwrap();

    // This refers to the `File` but doesn't use lifetimes to indicate
    // that. This is very dangerous, and you need to be careful.
    unsafe {
        let data = libc::mmap(
            /* addr: */ ptr::null_mut(),
            /* len: */ size,
            /* prot: */ libc::PROT_READ | libc::PROT_WRITE,
            // Then make the mapping *public* so it is written back to the file
            /* flags: */ libc::MAP_SHARED,
            /* fd: */ f.as_raw_fd(),
            /* offset: */ 0,
        ) as *mut u8;

        if data.is_null() {
            panic!("Could not access data from memory mapped file")
        }

        ptr::copy_nonoverlapping(src.as_ptr(), data, src.len());
    }
}

Answer 2

最新版本:

use std::ptr;
use std::fs;
use std::io::{Write, SeekFrom, Seek};
use std::os::unix::prelude::AsRawFd;
use mmap::{MemoryMap, MapOption};

// from crates.io
extern crate mmap;
extern crate libc;

fn main() {
    let size: usize = 1024*1024;

    let mut f = fs::OpenOptions::new().read(true)
                                      .write(true)
                                      .create(true)
                                      .open("test.mmap")
                                      .unwrap();

    // Allocate space in the file first
    f.seek(SeekFrom::Start(size as u64)).unwrap();
    f.write_all(&[0]).unwrap();
    f.seek(SeekFrom::Start(0)).unwrap();

    let mmap_opts = &[
        // Then make the mapping *public* so it is written back to the file
        MapOption::MapNonStandardFlags(libc::consts::os::posix88::MAP_SHARED),
        MapOption::MapReadable,
        MapOption::MapWritable,
        MapOption::MapFd(f.as_raw_fd()),
    ];

    let mmap = MemoryMap::new(size, mmap_opts).unwrap();

    let data = mmap.data();

    if data.is_null() {
        panic!("Could not access data from memory mapped file")
    }

    let src = "Hello!";
    let src_data = src.as_bytes();

    unsafe {
        ptr::copy(src_data.as_ptr(), data, src_data.len());
    }
}