如何在Ruby中检测字符串中的某些Unicode字符？

Question

给定Ruby 1.8.7中的字符串(没有支持带有\ p {}的Unicode属性的真棒Oniguruma正则表达式引擎),我希望能够确定该字符串是否包含一个或多个中文,日文或韩文字符; 即

class String
  def contains_cjk?
    ...
  end
end

>> '???'.contains_cjk?
=> true
>> '?? ????'.contains_cjk?
=> true
>> '????????'.contains_cjk?
=> true
>> 'Watashi ha bakana gaijin desu.'.contains_cjk?
=> false

我怀疑这将归结为查看字符串中的任何字符是否在Unihan CJKV Unicode块中,但我认为值得问一下是否有人知道Ruby中的现有解决方案.

Answer 1

(红宝石1.9.2)

#encoding: UTF-8
class String
  def contains_cjk?
    !!(self =~ /\p{Han}|\p{Katakana}|\p{Hiragana}|\p{Hangul}/)
  end
end

strings= ['??', '?? ????', '????????', 'Watashi ha bakana gaijin desu.']
strings.each{|s| puts s.contains_cjk?}

#true
#true
#true
#false

\ p {}匹配角色的Unicode脚本.
支持以下脚本:阿拉伯语,亚美尼亚语,巴厘岛语,孟加拉语,Bopomofo,盲文,Buginese,Buhid,Canadian_Aboriginal,Carian,Cham,Cherokee,Common,Coptic,Cuneiform,Cypriot,Cyrillic,Deseret,Devanagari,Ethiopic,Georgian,Glagolitic, Gothic,Greek,Gujarati,Gurmukhi,Han,Hangul,Hanunoo,Hebrew,Hiragana,Inherited,Kannada,Katakana,Kayah_Li,Kharoshthi,Khmer,Lao,Latin,Lepcha,Limbu,Linear_B,Lycian,Lydian,Malayalam,Mongolian,Myanmar, New_Tai_Lue,Nko,Ogham,Ol_Chiki,Old_Italic,Old_Persian,Oriya,Osmanya,Phags_Pa,Phoenician,Rejang,Runic,Saurashtra,Shavian,Sinhala,Sundanese,Syloti_Nagri,Syriac,Tagalog,Tagbanwa,Tai_Le,Tamil,Telugu,Thaana,Thai,西藏人,Tifinagh,Ugaritic,Vai和Yi.

哇.Ruby Regexp源码.

Answer 2

鉴于我的Ruby 1.8.7约束,这是我能做的最好的:

class String
  CJKV_RANGES = [
      (0xe2ba80..0xe2bbbf),
      (0xe2bfb0..0xe2bfbf),
      (0xe38080..0xe380bf),
      (0xe38180..0xe383bf),
      (0xe38480..0xe386bf),
      (0xe38780..0xe387bf),
      (0xe38880..0xe38bbf),
      (0xe38c80..0xe38fbf),
      (0xe39080..0xe4b6bf),
      (0xe4b780..0xe4b7bf),
      (0xe4b880..0xe9bfbf),
      (0xea8080..0xea98bf),
      (0xeaa080..0xeaaebf),
      (0xeaaf80..0xefbfbf),
  ]

  def contains_cjkv?
    each_char do |ch|
      return true if CJKV_RANGES.any? {|range| range.member? ch.unpack('H*').first.hex }
    end
    false
  end
end


strings = ['??', '?? ????', '????????', 'Watashi ha bakana gaijin desu.']
strings.each {|s| puts s.contains_cjkv? }

#true
#true
#true
#false

非常朴素,但它的工作原理.它实际上也会检测各种印度语脚本,所以它应该真的被称为contains_asian？

也许我应该为其他陷入Ruby 1.8的可怜的I18N黑客做好准备.