android mmap因内存不足而失败

Question

我到处寻找答案,但我想我已经达到了我能找到的极限.我的问题似乎与此有点相关:升级到Lollipop后,32位设备上的Android NDK mmap调用被破坏但没有提供答案.

我的问题是我尝试通过mmap调用从文件中存储映射457232384字节.在Android 5.1.1的两个不同的设备(三星Galaxy Note 3和OnePlus One,每个3GB RAM)上,该调用失败,错误12"内存不足".实际上,当我尝试分配超过300MB的内存时,调用失败.313524224字节(299MB)工作,314572800(300MB)不会.

事情是,同样的调用适用于第四台仍在Android 4.4.2上的设备.更奇怪的是,这个调用适用于带有SDK 21(Android 5.0)的Android ARM模拟器.不用说,可以毫无问题地加载相同数量的数据(不是mmap').

dmesg向我报告:

<3>[ 1137.488411] [0:Thread-298: 4267] arch_get_unmapped_area (TASK_SIZE - len < addr) len=457232384 task size=3204448256 pid=4267 do_align=0 addr=3034054656 mmap_base=3069939712

尝试映射文件的函数(来自openfst)如下:

MappedFile* MappedFile::Map(istream* s, const FstReadOptions &opts,
                        size_t size) {
  size_t pos = s->tellg();

  if (opts.mode == FstReadOptions::MAP && pos >= 0 &&
      pos % kArchAlignment == 0) {
    int fd = open(opts.source.c_str(), O_RDONLY);
    if (fd != -1) {
      int pagesize = getpagesize();
      off_t offset = pos % pagesize;
      off_t upsize = size + offset;
      void *map = mmap(NULL, upsize, PROT_READ, MAP_SHARED, fd, pos - offset);
      char *data = reinterpret_cast<char*>(map);
      if (close(fd) == 0 && map != MAP_FAILED) {
        MemoryRegion region;
        region.mmap = map;
        region.size = upsize;
        region.data = reinterpret_cast<void*>(data + offset);
        MappedFile *mmf = new MappedFile(region);
        s->seekg(pos + size, ios::beg);
        if (s) {
          VLOG(1) << "mmap'ed region of " << size << " at offset " << pos
                  << " from " << opts.source.c_str() << " to addr " << map;
          return mmf;
        }
        delete mmf;
      } else {
        LOG(INFO) << "Mapping of file failed: " << strerror(errno);
      }
    }
  }
  // If all else fails resort to reading from file into allocated buffer.
  if (opts.mode != FstReadOptions::READ) {
    LOG(WARNING) << "File mapping at offset " << pos << " of file "
                 << opts.source << " could not be honored, reading instead.";
  }
  MappedFile* mf = Allocate(size);
  if (!s->read(reinterpret_cast<char*>(mf->mutable_data()), size)) {
    delete mf;
    return NULL;
  }
  return mf;
}

每次从mmap返回都是MAP_FAILED.

有人建议我在哪里可以解决我的问题吗？谢谢!

编辑:

这是臭名昭着的mmap调用之后/ proc/self/maps的内容:http://pastebin.com/1864jZC2

一点差距分析:

Gap between 00000000 and 12c00000 (diff = 314572800 bytes, 300 MB)
Gap between 42c00000 and 55281000 (diff = 308809728 bytes, 294.50390625 MB)
Gap between 67e80000 and 67ea4000 (diff = 147456 bytes, 0.140625 MB)
Gap between 7778b000 and 77800000 (diff = 479232 bytes, 0.45703125 MB)
Gap between 77a80000 and 77a82000 (diff = 8192 bytes, 0.0078125 MB)
Gap between 77c00000 and 77c04000 (diff = 16384 bytes, 0.015625 MB)
Gap between 78080000 and 780b7000 (diff = 225280 bytes, 0.21484375 MB)
Gap between 79ac1000 and 79ac2000 (diff = 4096 bytes, 0.00390625 MB)
Gap between 7db70000 and 7db71000 (diff = 4096 bytes, 0.00390625 MB)
Gap between 7e000000 and 7e001000 (diff = 4096 bytes, 0.00390625 MB)
Gap between 7e0fe000 and 7e0ff000 (diff = 4096 bytes, 0.00390625 MB)
Gap between 7e145000 and 7e146000 (diff = 4096 bytes, 0.00390625 MB)
Gap between b6fb9000 and be6ff000 (diff = 125067264 bytes, 119.2734375 MB)
Gap between beeff000 and ffff0000 (diff = 1091506176 bytes, 1040.94140625 MB)

编辑:

在@fadden的回答评论中对我有用的解决方案.

TL; DR:设置dalvik.vm.heapsize为512m.

Answer 1

转储失败/proc/self/maps后的副本mmap()(只需从代码中打开文件并将内容复制到临时文件).由于ASLR,您可能在获取大的连续虚拟地址范围时遇到问题.

Android中的一些Zip文件处理代码mmap()用于映射整个文件,并在内存中对其进行操作.有一天,有人创建了一个1GB的Zip文件,无法打开它.虽然进程的虚拟地址空间有足够的可用页面,但没有足够的连续页面来创建单个线性映射.(IIRC,解决方案mmap()只是中央目录.)

该maps输出会告诉你你的进程的地址空间是什么样子.对于ASLR /碎片而言,300MB似乎有点低,但它是一个开始寻找的好地方,并且可能解释了不一致的行为.