如何从java中使用FileChannel映射的内存中取消映射文件？

Question

我正在使用文件("sample.txt")映射到内存FileChannel.map(),然后使用关闭通道fc.close().在此之后,当我使用FileOutputStream写入文件时,我收到以下错误:

java.io.FileNotFoundException:sample.txt(无法在打开用户映射部分的文件上形成请求的操作)

File f = new File("sample.txt");
RandomAccessFile raf = new RandomAccessFile(f,"rw");
FileChannel fc = raf.getChannel();
MappedByteBuffer mbf = fc.map(FileChannel.MapMode.READ_ONLY, 0, fc.size());
fc.close();
raf.close();

FileOutputStream fos = new FileOutputStream(f);
fos.write(str.getBytes());
fos.close();

我认为这可能是由于文件仍然映射到内存,即使我关闭后FileChannel.我对吗？.如果是这样,我如何从内存中"取消映射"文件？(我在API中找不到任何方法).谢谢.

编辑:看起来像(添加一个unmap方法)作为RFE提交给sun一段时间了:http : //bugs.sun.com/view_bug.do?bug_id=4724038

Answer 1

可以使用以下静态方法:

public static void unmap(MappedByteBuffer buffer)
{
   sun.misc.Cleaner cleaner = ((DirectBuffer) buffer).cleaner();
   cleaner.clean();
}

但这是不安全的解决方案,原因如下:
1)如果有人在unmap之后使用MappedByteBuffer导致失败
2)它依赖于MappedByteBuffer实现细节

Answer 2

[WinXP,SunJDK1.6]我从filechannel获取了一个映射的ByteBuffer.在阅读了SO帖后,终于设法通过反射呼叫清洁工而没有任何太阳.*包进口.不再存在文件锁定.

编辑添加了JDK9 +代码(Luke Hutchison).

private static void closeDirectBuffer(ByteBuffer cb) {
    if (cb==null || !cb.isDirect()) return;
    // we could use this type cast and call functions without reflection code,
    // but static import from sun.* package is risky for non-SUN virtual machine.
    //try { ((sun.nio.ch.DirectBuffer)cb).cleaner().clean(); } catch (Exception ex) { }

    // JavaSpecVer: 1.6, 1.7, 1.8, 9, 10
    boolean isOldJDK = System.getProperty("java.specification.version","99").startsWith("1.");  
    try {
        if (isOldJDK) {
            Method cleaner = cb.getClass().getMethod("cleaner");
            cleaner.setAccessible(true);
            Method clean = Class.forName("sun.misc.Cleaner").getMethod("clean");
            clean.setAccessible(true);
            clean.invoke(cleaner.invoke(cb));
        } else {
            Class unsafeClass;
            try {
                unsafeClass = Class.forName("sun.misc.Unsafe");
            } catch(Exception ex) {
                // jdk.internal.misc.Unsafe doesn't yet have an invokeCleaner() method,
                // but that method should be added if sun.misc.Unsafe is removed.
                unsafeClass = Class.forName("jdk.internal.misc.Unsafe");
            }
            Method clean = unsafeClass.getMethod("invokeCleaner", ByteBuffer.class);
            clean.setAccessible(true);
            Field theUnsafeField = unsafeClass.getDeclaredField("theUnsafe");
            theUnsafeField.setAccessible(true);
            Object theUnsafe = theUnsafeField.get(null);
            clean.invoke(theUnsafe, cb);
        }
    } catch(Exception ex) { }
    cb = null;
}

想法来自这些帖子.
*如何从java中使用FileChannel映射的内存中取消映射文件？
*强制释放本机内存直接ByteBuffer使用sun.misc.Unsafe分配的示例？
*https://github.com/elasticsearch/elasticsearch/blob/master/src/main/java/org/apache/lucene/store/bytebuffer/ByteBufferAllocator.java#L40

Answer 3

来自MappedByteBufferjavadoc:

映射的字节缓冲区及其表示的文件映射在缓冲区本身被垃圾收集之前保持有效.

试着打电话System.gc()？即使这只是对VM的建议.

Answer 4

sun.misc.Cleaner javadoc说:

通用的基于幻像参考的清洁剂.清洁剂是最终确定的轻量级和更强大的替代品.它们是轻量级的,因为它们不是由VM创建的,因此不需要创建JNI上行调用,并且因为它们的清理代码是由引用处理程序线程而不是终结器线程直接调用的.它们更强大,因为它们使用幻像引用,最弱的参考对象类型,从而避免了最终确定的令人讨厌的排序问题.清理器跟踪引用对象并封装任意清理代码.在GC检测到清洁器的指示对象已变为幻像可达之后的一段时间,引用处理程序线程将运行清除程序.清洁工也可以直接调用; 它们是线程安全的,并确保它们最多运行一次.清洁工不是最终确定的替代品.只有在清理代码非常简单和直接时才应该使用它们.非平凡的清洁工是不可取的,因为它们有可能阻塞参考处理程序线程并延迟进一步的清理和最终化.

如果你的缓冲区总大小很小,运行System.gc()是可以接受的解决方案,但如果我映射了几千兆字节的文件,我会尝试这样实现:

((DirectBuffer) buffer).cleaner().clean()

但!确保在清洁后没有访问该缓冲区,或者最终会:

Java运行时环境检测到致命错误:pc = 0x0000000002bcf700处的EXCEPTION_ACCESS_VIOLATION(0xc0000005),pid = 7592,tid = 10184 JRE版本:Java(TM)SE运行时环境(8.0_40-b25)(版本1.8.0_40- b25)Java VM:Java HotSpot(TM)64位服务器VM(25.40-b25混合模式windows-amd64压缩oops)有问题的帧:J 85 C2 java.nio.DirectByteBuffer.get(I)B(16字节)@ 0x0000000002bcf700 [0x0000000002bcf6c0 + 0x40]无法写入核心转储.默认情况下,在客户端版本的Windows上未启用小型转储.具有更多信息的错误报告文件保存为:C:\ Users\?????\Programs\testApp\hs_err_pid7592.log编译方法(c2)42392 85 4 java.堆中的nio.DirectByteBuffer :: get(16字节)总计[0x0000000002bcf590,0x0000000002bcf828] = 664重定位[0x0000000002bcf6b0,
[0x0000000002bcf760,0x0000000002bcf778] = 24个糟糕
[0x0000000002bcf778,0x0000000002bcf780] = 8元数据
[0x0000000002bcf780,0x0000000002bcf798] = 24米范围数据
[0x0000000002bcf798,0x0000000002bcf7e0] = 72米范围件
[0x0000000002bcf7e0,0x0000000002bcf820] = 64依赖性
[0x0000000002bcf820,0x0000000002bcf828] = 8

祝好运!

Answer 5

其他答案中涵盖的方法((DirectBuffer) byteBuffer).cleaner().clean()在 JDK 9+ 上不起作用（即使以反射形式），而不显示警告An illegal reflective access operation has occurred。这将在未来的 JDK 版本中完全停止工作。幸运的是，sun.misc.Unsafe.invokeCleaner(ByteBuffer)可以为您进行完全相同的调用，而不会出现警告：（来自 OpenJDK 11 源代码）：

public void invokeCleaner(java.nio.ByteBuffer directBuffer) {
    if (!directBuffer.isDirect())
        throw new IllegalArgumentException("buffer is non-direct");

    DirectBuffer db = (DirectBuffer)directBuffer;
    if (db.attachment() != null)
        throw new IllegalArgumentException("duplicate or slice");

    Cleaner cleaner = db.cleaner();
    if (cleaner != null) {
        cleaner.clean();
    }
}

作为一个sun.misc类，它会在某个时候被删除。有趣的是，除此之外的所有调用都sun.misc.Unsafe直接代理到jdk.internal.misc.Unsafe. 我不知道为什么它invokeCleaner(ByteBuffer)不以与所有其他方法相同的方式进行代理——它可能被省略了，因为DirectByteBuffer从 JDK 15 开始将有一种直接释放内存引用（包括实例）的新方法。

我编写了以下代码，能够在 JDK 7/8 以及 JDK 9+ 上清理/关闭/取消映射 DirectByteBuffer/MappedByteBuffer 实例，并且不会给出反射警告：

private static boolean PRE_JAVA_9 = 
        System.getProperty("java.specification.version","9").startsWith("1.");

private static Method cleanMethod;
private static Method attachmentMethod;
private static Object theUnsafe;

static void getCleanMethodPrivileged() {
    if (PRE_JAVA_9) {
        try {
            cleanMethod = Class.forName("sun.misc.Cleaner").getMethod("clean");
            cleanMethod.setAccessible(true);
            final Class<?> directByteBufferClass =
                    Class.forName("sun.nio.ch.DirectBuffer");
            attachmentMethod = directByteBufferClass.getMethod("attachment");
            attachmentMethod.setAccessible(true);
        } catch (final Exception ex) {
        }
    } else {
        try {
            Class<?> unsafeClass;
            try {
                unsafeClass = Class.forName("sun.misc.Unsafe");
            } catch (Exception e) {
                // jdk.internal.misc.Unsafe doesn't yet have invokeCleaner(),
                // but that method should be added if sun.misc.Unsafe is removed.
                unsafeClass = Class.forName("jdk.internal.misc.Unsafe");
            }
            cleanMethod = unsafeClass.getMethod("invokeCleaner", ByteBuffer.class);
            cleanMethod.setAccessible(true);
            final Field theUnsafeField = unsafeClass.getDeclaredField("theUnsafe");
            theUnsafeField.setAccessible(true);
            theUnsafe = theUnsafeField.get(null);
        } catch (final Exception ex) {
        }
    }
}

static {
    AccessController.doPrivileged(new PrivilegedAction<Object>() {
        @Override
        public Object run() {
            getCleanMethodPrivileged();
            return null;
        }
    });
}

private static boolean closeDirectByteBufferPrivileged(
            final ByteBuffer byteBuffer, final LogNode log) {
    try {
        if (cleanMethod == null) {
            if (log != null) {
                log.log("Could not unmap ByteBuffer, cleanMethod == null");
            }
            return false;
        }
        if (PRE_JAVA_9) {
            if (attachmentMethod == null) {
                if (log != null) {
                    log.log("Could not unmap ByteBuffer, attachmentMethod == null");
                }
                return false;
            }
            // Make sure duplicates and slices are not cleaned, since this can result in
            // duplicate attempts to clean the same buffer, which trigger a crash with:
            // "A fatal error has been detected by the Java Runtime Environment:
            // EXCEPTION_ACCESS_VIOLATION"
            // See: /sf/answers/2211506321/
            if (attachmentMethod.invoke(byteBuffer) != null) {
                // Buffer is a duplicate or slice
                return false;
            }
            // Invoke ((DirectBuffer) byteBuffer).cleaner().clean()
            final Method cleaner = byteBuffer.getClass().getMethod("cleaner");
            cleaner.setAccessible(true);
            cleanMethod.invoke(cleaner.invoke(byteBuffer));
            return true;
        } else {
            if (theUnsafe == null) {
                if (log != null) {
                    log.log("Could not unmap ByteBuffer, theUnsafe == null");
                }
                return false;
            }
            // In JDK9+, calling the above code gives a reflection warning on stderr,
            // need to call Unsafe.theUnsafe.invokeCleaner(byteBuffer) , which makes
            // the same call, but does not print the reflection warning.
            try {
                cleanMethod.invoke(theUnsafe, byteBuffer);
                return true;
            } catch (final IllegalArgumentException e) {
                // Buffer is a duplicate or slice
                return false;
            }
        }
    } catch (final Exception e) {
        if (log != null) {
            log.log("Could not unmap ByteBuffer: " + e);
        }
        return false;
    }
}

/**
 * Close a {@code DirectByteBuffer} -- in particular, will unmap a
 * {@link MappedByteBuffer}.
 * 
 * @param byteBuffer
 *            The {@link ByteBuffer} to close/unmap.
 * @param log
 *            The log.
 * @return True if the byteBuffer was closed/unmapped (or if the ByteBuffer
 *            was null or non-direct).
 */
public static boolean closeDirectByteBuffer(final ByteBuffer byteBuffer,
            final Log log) {
    if (byteBuffer != null && byteBuffer.isDirect()) {
        return AccessController.doPrivileged(new PrivilegedAction<Boolean>() {
            @Override
            public Boolean run() {
                return closeDirectByteBufferPrivileged(byteBuffer, log);
            }
        });
    } else {
        // Nothing to unmap
        return false;
    }
}

请注意，您需要将 添加requires jdk.unsupported到 JDK 9+ 上的模块化运行时中的模块描述符（使用 时需要Unsafe）。

您的罐子可能还需要RuntimePermission("accessClassInPackage.sun.misc")、、RuntimePermission("accessClassInPackage.jdk.internal.misc")和ReflectPermission("suppressAccessChecks")。

ClassGraph 中实现了更完整的垃圾收集 aMappedByteBuffer或 a的方法DirectByteBuffer（我是作者）——入口点是closeDirectByteBuffer()末尾的方法FileUtils：

https://github.com/classgraph/classgraph/blob/latest/src/main/java/nonapi/io/github/classgraph/utils/FileUtils.java#L543

编写此代码是为了使用反射，因为所使用的 Java API（包括Unsafe）将在不久的将来消失。

请注意，JDK 16+ 中还有一个额外问题：

除非您使用Narcissus或JVM-Driver库来规避强封装，否则此代码在 JDK 16+ 中无法开箱即用。这是因为MappedByteBuffer.clean()是私有方法，并且 JDK 16 强制执行强封装。ClassGraph 通过在运行时通过反射调用 Narcissus 或 JVM 驱动程序来抽象出对私有封装方法的访问：

https://github.com/classgraph/classgraph/blob/latest/src/main/java/nonapi/io/github/classgraph/reflection/ReflectionUtils.java

DirectByteBuffer警告：如果您在清理（释放）后尝试访问它，则会导致虚拟机崩溃。

还有其他安全注意事项，在此错误报告的最后评论中讨论：

https://bugs.java.com/bugdatabase/view_bug.do?bug_id=JDK-4724038