获取文件哈希性能/优化

Question

我正在尝试尽快获取文件的哈希值.我有一个程序,可以在几个文件到几十万个文件之间的任何地方散列大量数据(100GB +),包括随机文件大小(从几KB到每个文件的5GB +).

该程序必须支持所有Java支持的算法(MD2,MD5,SHA-1,SHA-256,SHA-384,SHA-512).

目前我使用:

/**
 * Gets Hash of file.
 * 
 * @param file String path + filename of file to get hash.
 * @param hashAlgo Hash algorithm to use. <br/>
 *     Supported algorithms are: <br/>
 *     MD2, MD5 <br/>
 *     SHA-1 <br/>
 *     SHA-256, SHA-384, SHA-512
 * @return String value of hash. (Variable length dependent on hash algorithm used)
 * @throws IOException If file is invalid.
 * @throws HashTypeException If no supported or valid hash algorithm was found.
 */
public String getHash(String file, String hashAlgo) throws IOException, HashTypeException {
    StringBuffer hexString = null;
    try {
        MessageDigest md = MessageDigest.getInstance(validateHashType(hashAlgo));
        FileInputStream fis = new FileInputStream(file);

        byte[] dataBytes = new byte[1024];

        int nread = 0;
        while ((nread = fis.read(dataBytes)) != -1) {
            md.update(dataBytes, 0, nread);
        }
        fis.close();
        byte[] mdbytes = md.digest();

        hexString = new StringBuffer();
        for (int i = 0; i < mdbytes.length; i++) {
            hexString.append(Integer.toHexString((0xFF & mdbytes[i])));
        }

        return hexString.toString();

    } catch (NoSuchAlgorithmException | HashTypeException e) {
        throw new HashTypeException("Unsuppored Hash Algorithm.", e);
    }
}

是否有更优化的方式来获取文件哈希？我正在寻找极致的表现,我不确定我是否已经采用了最佳方式.

Answer 1

我看到了许多潜在的性能改进.一个是用StringBuilder而不是StringBuffer; 它是源兼容的,但性能更高,因为它不同步.第二(更重要的)将是使用FileChannel和java.nio代替的API FileInputStream-或者至少,包裹FileInputStream在一个BufferedInputStream以优化I/O.