python - 正则表达式和unicode的问题

Question

嗨我在python中有问题.我试着通过一个例子来解释我的问题.

我有这个字符串:

>>> string = 'ÐÑÒÓÔÕÖ×ØÙÚÛÜÝÞßàáâãäåæçèéêëìíîïðñòóôõö÷øùúûüýþÿÀÁÂÃ'
>>> print string
ÐÑÒÓÔÕÖ×ØÙÚÛÜÝÞßàáâãäåæçèéêëìíîïðñòóôõö÷øùúûüýþÿÀÁÂÃ

我希望,例如,用""替换不同于Ñ,Ã,ï的字符串

我试过了:

>>> rePat = re.compile('[^ÑÃï]',re.UNICODE)
>>> print rePat.sub("",string)
?Ñ?????????????????????????????ï???????????????????Ã

我得到了这个 .我认为这是因为python中的这种类型的字符由向量中的两个位置表示:例如\ xc3\x91 =Ñ.为此,当我进行regolar表达时,所有的\ xc3都不会被替换.我怎么能做这种类型的子?????

谢谢佛朗哥

Answer 1

您需要确保您的字符串是unicode字符串,而不是普通字符串(普通字符串就像字节数组).

例:

>>> string = 'ÐÑÒÓÔÕÖ×ØÙÚÛÜÝÞßàáâãäåæçèéêëìíîïðñòóôõö÷øùúûüýþÿÀÁÂÃ'
>>> type(string)
<type 'str'>

# do this instead:
# (note the u in front of the ', this marks the character sequence as a unicode literal)
>>> string = u'\xd0\xd1\xd2\xd3\xd4\xd5\xd6\xd7\xd8\xd9\xda\xdb\xdc\xdd\xde\xdf\xe0\xe1\xe2\xe3\xe4\xe5\xe6\xe7\xe8\xe9\xea\xeb\xec\xed\xee\xef\xf0\xf1\xf2\xf3\xf4\xf5\xf6\xf7\xf8\xf9\xfa\xfb\xfc\xfd\xfe\xff\xc0\xc1\xc2\xc3'
# or:
>>> string = 'ÐÑÒÓÔÕÖ×ØÙÚÛÜÝÞßàáâãäåæçèéêëìíîïðñòóôõö÷øùúûüýþÿÀÁÂÃ'.decode('utf-8')
# ... but be aware that the latter will only work if the terminal (or source file) has utf-8 encoding
# ... it is a best practice to use the \xNN form in unicode literals, as in the first example

>>> type(string)
<type 'unicode'>
>>> print string
ÐÑÒÓÔÕÖ×ØÙÚÛÜÝÞßàáâãäåæçèéêëìíîïðñòóôõö÷øùúûüýþÿÀÁÂÃ

>>> rePat = re.compile(u'[^\xc3\x91\xc3\x83\xc3\xaf]',re.UNICODE)
>>> print rePat.sub("", string)
Ã

从文件string = open('filename.txt').read()读取时,读取字节序列.

要获取unicode内容,请执行以下操作:string = unicode(open('filename.txt').read(), 'encoding').或者:string = open('filename.txt').read().decode('encoding').

该编解码器模块可以解码上即时的Unicode流(如文件).

谷歌搜索python unicode.Python unicode处理起初可能有点难以掌握,阅读它是值得的.

我遵循这条规则:"软件应该只在内部使用Unicode字符串,在输出时转换为特定的编码." (来自http://www.amk.ca/python/howto/unicode)

我还建议:http://www.joelonsoftware.com/articles/Unicode.html