使用PHP转换所有类型的智能引号

Question

我正在尝试在处理文本时将所有类型的智能引号转换为常规引号.但是,我编译的以下功能似乎仍然缺乏支持和正确的设计.

有谁知道如何正确地转换所有引用字符？

function convert_smart_quotes($string)
{
    $quotes = array(
        "\xC2\xAB"   => '"', // « (U+00AB) in UTF-8
        "\xC2\xBB"   => '"', // » (U+00BB) in UTF-8
        "\xE2\x80\x98" => "'", // ‘ (U+2018) in UTF-8
        "\xE2\x80\x99" => "'", // ’ (U+2019) in UTF-8
        "\xE2\x80\x9A" => "'", // ‚ (U+201A) in UTF-8
        "\xE2\x80\x9B" => "'", // ? (U+201B) in UTF-8
        "\xE2\x80\x9C" => '"', // “ (U+201C) in UTF-8
        "\xE2\x80\x9D" => '"', // ” (U+201D) in UTF-8
        "\xE2\x80\x9E" => '"', // „ (U+201E) in UTF-8
        "\xE2\x80\x9F" => '"', // ? (U+201F) in UTF-8
        "\xE2\x80\xB9" => "'", // ‹ (U+2039) in UTF-8
        "\xE2\x80\xBA" => "'", // › (U+203A) in UTF-8
    );
    $string = strtr($string, $quotes);

    // Version 2
    $search = array(
        chr(145),
        chr(146),
        chr(147),
        chr(148),
        chr(151)
    );
    $replace = array("'","'",'"','"',' - ');
    $string = str_replace($search, $replace, $string);

    // Version 3
    $string = str_replace(
        array('‘','’','“','”'),
        array("'", "'", '"', '"'),
        $string
    );

    // Version 4
    $search = array(
        '‘', 
        '’', 
        '“', 
        '”', 
        '—',
        '–',
    );
    $replace = array("'","'",'"','"',' - ', '-');
    $string = str_replace($search, $replace, $string);

    return $string;
}

注意:这个问题是关于完整的报价范围的完整查询,包括此处询问的"微软"报价这是一个"重复",就像询问所有轮胎尺寸是要求汽车轮胎的"重复"一样尺寸.

Answer 1

你需要这样的东西(假设输入UTF-8,忽略CJK(中文,日文,韩文)):

$chr_map = array(
   // Windows codepage 1252
   "\xC2\x82" => "'", // U+0082?U+201A single low-9 quotation mark
   "\xC2\x84" => '"', // U+0084?U+201E double low-9 quotation mark
   "\xC2\x8B" => "'", // U+008B?U+2039 single left-pointing angle quotation mark
   "\xC2\x91" => "'", // U+0091?U+2018 left single quotation mark
   "\xC2\x92" => "'", // U+0092?U+2019 right single quotation mark
   "\xC2\x93" => '"', // U+0093?U+201C left double quotation mark
   "\xC2\x94" => '"', // U+0094?U+201D right double quotation mark
   "\xC2\x9B" => "'", // U+009B?U+203A single right-pointing angle quotation mark

   // Regular Unicode     // U+0022 quotation mark (")
                          // U+0027 apostrophe     (')
   "\xC2\xAB"     => '"', // U+00AB left-pointing double angle quotation mark
   "\xC2\xBB"     => '"', // U+00BB right-pointing double angle quotation mark
   "\xE2\x80\x98" => "'", // U+2018 left single quotation mark
   "\xE2\x80\x99" => "'", // U+2019 right single quotation mark
   "\xE2\x80\x9A" => "'", // U+201A single low-9 quotation mark
   "\xE2\x80\x9B" => "'", // U+201B single high-reversed-9 quotation mark
   "\xE2\x80\x9C" => '"', // U+201C left double quotation mark
   "\xE2\x80\x9D" => '"', // U+201D right double quotation mark
   "\xE2\x80\x9E" => '"', // U+201E double low-9 quotation mark
   "\xE2\x80\x9F" => '"', // U+201F double high-reversed-9 quotation mark
   "\xE2\x80\xB9" => "'", // U+2039 single left-pointing angle quotation mark
   "\xE2\x80\xBA" => "'", // U+203A single right-pointing angle quotation mark
);
$chr = array_keys  ($chr_map); // but: for efficiency you should
$rpl = array_values($chr_map); // pre-calculate these two arrays
$str = str_replace($chr, $rpl, html_entity_decode($str, ENT_QUOTES, "UTF-8"));

背景是:

每个Unicode字符都属于一个"常规类别",其中可以包含引号字符的字符如下:

• Ps "标点符号,打开"
• Pe "标点符号,关闭"
• Pi "标点符号,初始引用(根据用法可能表现为Ps或Pe)"
• Pf "标点符号,最终报价(根据使用情况,可能表现为Ps或Pe)"
• Po "标点符号,其他"

(这些页面很方便检查你没有错过任何东西 - 还有一个类别索引)

在启用Unicode的正则表达式中匹配这些类别有时很有用.

此外,Unicode字符具有"属性",您感兴趣的是"属性"Quotation_Mark.不幸的是,这些在正则表达式中无法访问.

在维基百科中,您可以找到具有该Quotation_Mark属性的字符组.最终引用是unicode.org上的PropList.txt,但这是一个ASCII文本文件.

如果您还需要翻译CJK字符,您只需获取他们的代码点,决定他们的翻译,并找到他们的UTF-8编码,例如,通过在fileformat.info中查找(例如,对于U + 301E:http ://www.fileformat.info/info/unicode/char/301e/index.htm).

关于Windows代码页1252:Unicode定义前256个代码点以表示与ISO-8859-1完全相同的字符,但ISO-8859-1经常与Windows代码页1252混淆,因此所有浏览器都呈现范围0x80-0x9F,这在ISO-8859-1中是"空的"(更确切地说:它包含控制字符),就好像它是Windows代码页1252. 维基百科页面中的表列出了Unicode等价物.

注意:strtr()通常比慢str_replace().根据您的输入和PHP版本计算时间.如果它足够快,你可以直接使用像我这样的地图$chr_map.

如果你不确定你的输入是UTF-8编码,并且愿意假设如果不是,那么它是ISO-8859-1或Windows代码页1252,那么你可以在其他任何事情之前做到这一点:

if ( !preg_match('/^\\X*$/u', $str)) {
   $str = utf8_encode($str);
}

警告:此正则表达式在极少数情况下无法检测到非UTF-8编码.例如:"Gruß…"/*CP-1252*/=="Gru\xDF\x85"看起来像这个正则表达式的UTF-8(U + 07C5是N'ko数字5).这个正则表达式可以略微增强,但不幸的是,可以证明对编码检测问题没有完全万无一失的解决方案.

如果要将源自Windows代码页1252的范围0x80-0x9F规范化为常规Unicode代码点,则可以执行此操作(并删除$chr_map上面的第一部分):

$normalization_map = array(
   "\xC2\x80" => "\xE2\x82\xAC", // U+20AC Euro sign
   "\xC2\x82" => "\xE2\x80\x9A", // U+201A single low-9 quotation mark
   "\xC2\x83" => "\xC6\x92",     // U+0192 latin small letter f with hook
   "\xC2\x84" => "\xE2\x80\x9E", // U+201E double low-9 quotation mark
   "\xC2\x85" => "\xE2\x80\xA6", // U+2026 horizontal ellipsis
   "\xC2\x86" => "\xE2\x80\xA0", // U+2020 dagger
   "\xC2\x87" => "\xE2\x80\xA1", // U+2021 double dagger
   "\xC2\x88" => "\xCB\x86",     // U+02C6 modifier letter circumflex accent
   "\xC2\x89" => "\xE2\x80\xB0", // U+2030 per mille sign
   "\xC2\x8A" => "\xC5\xA0",     // U+0160 latin capital letter s with caron
   "\xC2\x8B" => "\xE2\x80\xB9", // U+2039 single left-pointing angle quotation mark
   "\xC2\x8C" => "\xC5\x92",     // U+0152 latin capital ligature oe
   "\xC2\x8E" => "\xC5\xBD",     // U+017D latin capital letter z with caron
   "\xC2\x91" => "\xE2\x80\x98", // U+2018 left single quotation mark
   "\xC2\x92" => "\xE2\x80\x99", // U+2019 right single quotation mark
   "\xC2\x93" => "\xE2\x80\x9C", // U+201C left double quotation mark
   "\xC2\x94" => "\xE2\x80\x9D", // U+201D right double quotation mark
   "\xC2\x95" => "\xE2\x80\xA2", // U+2022 bullet
   "\xC2\x96" => "\xE2\x80\x93", // U+2013 en dash
   "\xC2\x97" => "\xE2\x80\x94", // U+2014 em dash
   "\xC2\x98" => "\xCB\x9C",     // U+02DC small tilde
   "\xC2\x99" => "\xE2\x84\xA2", // U+2122 trade mark sign
   "\xC2\x9A" => "\xC5\xA1",     // U+0161 latin small letter s with caron
   "\xC2\x9B" => "\xE2\x80\xBA", // U+203A single right-pointing angle quotation mark
   "\xC2\x9C" => "\xC5\x93",     // U+0153 latin small ligature oe
   "\xC2\x9E" => "\xC5\xBE",     // U+017E latin small letter z with caron
   "\xC2\x9F" => "\xC5\xB8",     // U+0178 latin capital letter y with diaeresis
);
$chr = array_keys  ($normalization_map); // but: for efficiency you should
$rpl = array_values($normalization_map); // pre-calculate these two arrays
$str = str_replace($chr, $rpl, $str);

Answer 2

您可以使用此功能转换所有字符：

$output = iconv('UTF-8', 'ASCII//TRANSLIT', $input);

确保将类型更改为所需的类型。

（注意：这是从此处找到的另一个类似问题）。