Python unicode.splitlines()以非EOL字符触发

Question

Triyng在Python 2.7中做到这一点:

>>> s = u"some\u2028text"
>>> s
u'some\u2028text'
>>> l = s.splitlines(True)
>>> l
[u'some\u2028', u'text']

\u2028是从左到右嵌入字符,而不是\r或\n,因此不应拆分该行.有错误还是只是我的误会？

Answer 1

\u2028是LINE SEPARATOR,从左到右嵌入是\u202A:

>>> import unicodedata

>>> unicodedata.name(u'\u2028')
'LINE SEPARATOR'

>>> unicodedata.name(u'\u202A')
'LEFT-TO-RIGHT EMBEDDING'

考虑换行的代码点列表很容易(虽然不容易找到)在python源代码中看到(python 2.7,我的评论):

/* Returns 1 for Unicode characters having the line break
 * property 'BK', 'CR', 'LF' or 'NL' or having bidirectional
 * type 'B', 0 otherwise.
 */
int _PyUnicode_IsLinebreak(register const Py_UNICODE ch)
{
    switch (ch) {
    // Basic Latin
    case 0x000A:    // LINE FEED
    case 0x000B:    // VERTICAL TABULATION
    case 0x000C:    // FORM FEED
    case 0x000D:    // CARRIAGE RETURN
    case 0x001C:    // FILE SEPARATOR
    case 0x001D:    // GROUP SEPARATOR
    case 0x001E:    // RECORD SEPARATOR

    // Latin-1 Supplement
    case 0x0085:    // NEXT LINE

    // General punctuation
    case 0x2028:    // LINE SEPARATOR
    case 0x2029:    // PARAGRAPH SEPARATOR
        return 1;
    }
    return 0;
}