UnicodeDecodeError:'utf-8'编解码器无法解码位置65534-65535中的字节:意外的数据结束

Question

我想用简单的AES加密来加密文件,这是我的python3源代码.

import os, random, struct
from Crypto.Cipher import AES

def encrypt_file(key, in_filename, out_filename=None, chunksize=64*1024):
    if not out_filename:
        out_filename = in_filename + '.enc'
    iv = os.urandom(16)
    encryptor = AES.new(key, AES.MODE_CBC, iv)
    filesize = os.path.getsize(in_filename)
    with open(in_filename, 'rb') as infile:
        with open(out_filename, 'wb') as outfile:
            outfile.write(struct.pack('<Q', filesize))
            outfile.write(iv)
            while True:
                chunk = infile.read(chunksize)
                if len(chunk) == 0:
                    break
                elif len(chunk) % 16 != 0:
                    chunk += ' ' * (16 - len(chunk) % 16)
                outfile.write(encryptor.encrypt(chunk.decode('UTF-8','strict')))

它适用于某些文件,遇到某些文件的错误信息,如下所示:

encrypt_file("qwertyqwertyqwer",'/ tmp/test1',out_filename = None,chunksize = 64*1024)

没有错误信息,工作正常.

encrypt_file("qwertyqwertyqwer",'/ tmp/test2',out_filename = None,chunksize = 64*1024)

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 17, in encrypt_file
UnicodeDecodeError: 'utf-8' codec can't decode bytes in position 65534-65535: unexpected end of data

如何修复我的encrypt_file函数？

就像t.m.adam说,修复

outfile.write(encryptor.encrypt(chunk.decode('UTF-8','strict')))

如

outfile.write(encryptor.encrypt(chunk))

试试一些文件.

encrypt_file("qwertyqwertyqwer",'/tmp/test' , out_filename=None, chunksize=64*1024)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 16, in encrypt_file
TypeError: can't concat bytes to str

Answer 1

您的代码的主要问题是您正在使用字符串.AES使用二进制数据,如果您使用的是PyCryptodome,则此代码会引发TypeError:

Object type <class 'str'> cannot be passed to C code

Pycrypto接受字符串,但在内部将它们编码为字节,因此将字节解码为字符串是没有意义的,因为它将被编码回字节.此外,它使用ASCII编码(使用PyCrypto v2.6.1,Python v2.7测试),因此,此代码例如:

encryptor.encrypt(u'?' * 16)

会引发UnicodeEncodeError:

File "C:\Python27\lib\site-packages\Crypto\Cipher\blockalgo.py", line 244, in encrypt
    return self._cipher.encrypt(plaintext)
UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-15

加密或解密数据时应始终使用字节.然后你可以将明文解码为字符串,如果它是文本.

下一个问题是你的填充方法.它会产生一个字符串,所以当你尝试将它应用于明文时,你会得到一个TypeError,它应该是字节.如果你填充字节,你可以解决这个问题,

chunk += b' ' * (16 - len(chunk) % 16)

但最好使用PKCS7填充(目前你使用零填充,但是使用空格而不是零字节).

PyCryptodome提供了填充功能,但似乎你正在使用PyCrypto.在这种情况下,您可以实现PKCS7填充,或者更好地复制PyCryptodome的填充功能.

try:
    from Crypto.Util.Padding import pad, unpad
except ImportError:
    from Crypto.Util.py3compat import bchr, bord

    def pad(data_to_pad, block_size):
        padding_len = block_size-len(data_to_pad)%block_size
        padding = bchr(padding_len)*padding_len
        return data_to_pad + padding

    def unpad(padded_data, block_size):
        pdata_len = len(padded_data)
        if pdata_len % block_size:
            raise ValueError("Input data is not padded")
        padding_len = bord(padded_data[-1])
        if padding_len<1 or padding_len>min(block_size, pdata_len):
            raise ValueError("Padding is incorrect.")
        if padded_data[-padding_len:]!=bchr(padding_len)*padding_len:
            raise ValueError("PKCS#7 padding is incorrect.")
        return padded_data[:-padding_len]

的pad和unpad功能进行了从复制Crypto.Util.Padding和修改,以只使用PKCS7填充.请注意,使用PKCS7填充时,填充最后一个块很重要,即使它的大小是块大小的倍数,否则您将无法正确地取消填充.

将这些更改应用于该encrypt_file功能,

def encrypt_file(key, in_filename, out_filename=None, chunksize=64*1024):
    if not out_filename:
        out_filename = in_filename + '.enc'
    iv = os.urandom(16)
    encryptor = AES.new(key, AES.MODE_CBC, iv)
    filesize = os.path.getsize(in_filename)
    with open(in_filename, 'rb') as infile:
        with open(out_filename, 'wb') as outfile:
            outfile.write(struct.pack('<Q', filesize))
            outfile.write(iv)
            pos = 0
            while pos < filesize:
                chunk = infile.read(chunksize)
                pos += len(chunk)
                if pos == filesize:
                    chunk = pad(chunk, AES.block_size)
                outfile.write(encryptor.encrypt(chunk))

和匹配decrypt_file功能,

def decrypt_file(key, in_filename, out_filename=None, chunksize=64*1024):
    if not out_filename:
        out_filename = in_filename + '.dec'
    with open(in_filename, 'rb') as infile:
        filesize = struct.unpack('<Q', infile.read(8))[0]
        iv = infile.read(16)
        encryptor = AES.new(key, AES.MODE_CBC, iv)
        with open(out_filename, 'wb') as outfile:
            encrypted_filesize = os.path.getsize(in_filename)
            pos = 8 + 16 # the filesize and IV.
            while pos < encrypted_filesize:
                chunk = infile.read(chunksize)
                pos += len(chunk)
                chunk = encryptor.decrypt(chunk)
                if pos == encrypted_filesize:
                    chunk = unpad(chunk, AES.block_size)
                outfile.write(chunk)

此代码与Python2/Python3兼容,它应该与PyCryptodome或PyCrypto一起使用.

但是,如果您正在使用PyCrypto,我建议您更新到PyCryptodome.PyCryptodome是PyCrypto的一个分支,它暴露了相同的API(所以你不必太多地改变你的代码),还有一些额外的功能:填充函数,Authenticated Encryption算法,KDF等.另一方面,PyCrypto不是正在维护,并且,某些版本遭受基于堆的缓冲区溢出漏洞:CVE-2013-7459.