如何只使用一次传递创建文件的多个哈希？

Question

如何从文件中获取MD5,SHA和其他哈希值,但只进行一次传递？我有100mb文件,所以我讨厌多次处理这些100MB文件.

Answer 1

也许这样的事情？

>>> import hashlib
>>> hashes = (hashlib.md5(), hashlib.sha1())
>>> f = open('some_file', 'r')
>>> for line in f:
...     for hash in hashes:
...         hash.update(line)
... 
>>> for hash in hashes:
...     print hash.name, hash.hexdigest()

或者循环遍历f.read(1024)或类似的东西以获得固定长度的块

Answer 2

这是使用'建议的修改后@???u的答案.@Jason S

from __future__ import with_statement
from hashlib import md5, sha1

filename = 'hash_one-pass.py'

hashes = md5(), sha1()
chunksize = max(4096, max(h.block_size for h in hashes))
with open(filename, 'rb') as f:
    while True:
        chunk = f.read(chunksize)
        if not chunk:
            break
        for h in hashes:
            h.update(chunk)

for h in hashes:
    print h.name, h.hexdigest()