UnicodeEncodeError:'latin-1'编解码器无法编码字符

Question

当我尝试将外来字符插入数据库时​​,可能导致此错误的原因是什么？

>>UnicodeEncodeError: 'latin-1' codec can't encode character u'\u201c' in position 0: ordinal not in range(256)

我该如何解决？

谢谢!

Answer 1

我在使用Python MySQLdb模块时遇到了同样的问题.由于MySQL将允许您在文本字段中存储您想要的任何二进制数据,而不管字符集如何,我在这里找到了我的解决方案:

使用UTF8和Python MySQLdb

编辑:从上面的URL引用以满足第一条评论中的请求...

"UnicodeEncodeError:'latin-1'编解码器无法对字符进行编码......"

这是因为MySQLdb通常会尝试将everythin编码为latin-1.这可以通过在您建立连接后立即执行以下命令来解决:

db.set_character_set('utf8')
dbc.execute('SET NAMES utf8;')
dbc.execute('SET CHARACTER SET utf8;')
dbc.execute('SET character_set_connection=utf8;')

"db"是结果MySQLdb.connect(),"dbc"是结果 db.cursor().

Answer 2

字符U + 201C左双引号在Latin-1(ISO-8859-1)编码中不存在.

这是目前在代码页1252(西欧).这是一种特定于Windows的编码,它基于ISO-8859-1,但将额外的字符放入0x80-0x9F范围内.代码页1252经常与ISO-8859-1混淆,这是一个令人烦恼但现在标准的Web浏览器行为,如果您将页面作为ISO-8859-1提供,浏览器将把它们视为cp1252.但是,它们实际上是两种截然不同的编码:

>>> u'He said \u201CHello\u201D'.encode('iso-8859-1')
UnicodeEncodeError
>>> u'He said \u201CHello\u201D'.encode('cp1252')
'He said \x93Hello\x94'

如果仅将数据库用作字节存储,则可以使用cp1252对“Windows Western代码页中存在的其他字符进行编码.但是cp1252中不存在的其他Unicode字符也会导致错误.

您可以encode(..., 'ignore')通过删除字符来抑制错误,但实际上在本世纪,您应该在数据库和页面中使用UTF-8.此编码允许使用任何字符.您还应该理想地告诉MySQL您正在使用UTF-8字符串(通过设置数据库连接和字符串列上的排序规则),因此它可以获得不区分大小写的比较和排序.

Answer 3

我希望你的数据库至少是UTF-8.然后,yourstring.encode('utf-8')在尝试将其放入数据库之前,您需要运行.

Answer 4

最好的解决方案是

1. 将mysql的charset设置为'utf-8'

2. 喜欢这个评论(添加use_unicode=True和charset="utf8")

   db = MySQLdb.connect(host ="localhost",user ="root",passwd ="",db ="testdb",use_unicode = True,charset ="utf8") - KyungHoon Kim 3月13日在17:04

细节见:

class Connection(_mysql.connection):

    """MySQL Database Connection Object"""

    default_cursor = cursors.Cursor

    def __init__(self, *args, **kwargs):
        """

        Create a connection to the database. It is strongly recommended
        that you only use keyword parameters. Consult the MySQL C API
        documentation for more information.

        host
          string, host to connect

        user
          string, user to connect as

        passwd
          string, password to use

        db
          string, database to use

        port
          integer, TCP/IP port to connect to

        unix_socket
          string, location of unix_socket to use

        conv
          conversion dictionary, see MySQLdb.converters

        connect_timeout
          number of seconds to wait before the connection attempt
          fails.

        compress
          if set, compression is enabled

        named_pipe
          if set, a named pipe is used to connect (Windows only)

        init_command
          command which is run once the connection is created

        read_default_file
          file from which default client values are read

        read_default_group
          configuration group to use from the default file

        cursorclass
          class object, used to create cursors (keyword only)

        use_unicode
          If True, text-like columns are returned as unicode objects
          using the connection's character set.  Otherwise, text-like
          columns are returned as strings.  columns are returned as
          normal strings. Unicode objects will always be encoded to
          the connection's character set regardless of this setting.

        charset
          If supplied, the connection character set will be changed
          to this character set (MySQL-4.1 and newer). This implies
          use_unicode=True.

        sql_mode
          If supplied, the session SQL mode will be changed to this
          setting (MySQL-4.1 and newer). For more details and legal
          values, see the MySQL documentation.

        client_flag
          integer, flags to use or 0
          (see MySQL docs or constants/CLIENTS.py)

        ssl
          dictionary or mapping, contains SSL connection parameters;
          see the MySQL documentation for more details
          (mysql_ssl_set()).  If this is set, and the client does not
          support SSL, NotSupportedError will be raised.

        local_infile
          integer, non-zero enables LOAD LOCAL INFILE; zero disables

        autocommit
          If False (default), autocommit is disabled.
          If True, autocommit is enabled.
          If None, autocommit isn't set and server default is used.

        There are a number of undocumented, non-standard methods. See the
        documentation for the MySQL C API for some hints on what they do.

        """

Answer 5

使用下面的代码片段将文本从拉丁语转换为英语

\n\n

import unicodedata\ndef strip_accents(text):\n    return "".join(char for char in\n                   unicodedata.normalize(\'NFKD\', text)\n                   if unicodedata.category(char) != \'Mn\')\n\nstrip_accents(\'\xc3\xa1\xc3\xa9\xc3\xad\xc3\xb1\xc3\xb3\xc3\xba\xc3\xbc\')\n

\n\n

输出：

\n\n

\n

'艾伊努'

\n

\n