对UTF-8字符串进行排序？

Question

我的std :: strings以UTF-8编码,因此std :: string <运算符不会删除它.我怎么能比较2个utf-8编码的std :: strings？

它不切割的地方是口音,é来自z,它不应该

谢谢

Answer 1

如果你不想要词典排序(这是按字典顺序排序UTF-8编码的字符串会给你),那么你需要根据需要将你的UTF-8编码字符串解码为UCS-2或UCS-4,并且应用您选择的合适比较功能.

重申一点,UTF-8编码机制设计巧妙,如果您通过查看每个8位编码字节的数值进行排序,您将获得与首次将字符串解码为Unicode并进行比较时相同的结果每个代码点的数值.

更新:您更新的问题表明您需要比纯粹的词典排序更复杂的比较功能.您需要解码UTF-8字符串并比较解码后的字符.

Answer 2

该标准std::locale适用于特定于语言环境的事物,例如整理(排序).如果环境包含LC_COLLATE=en_US.utf8或类似,则此程序将根据需要对行进行排序.

#include <algorithm>
#include <functional>
#include <iostream>
#include <iterator>
#include <locale>
#include <string>
#include <vector>
class collate_in : public std::binary_function<std::string, std::string, bool> {
  protected:
    const std::collate<char> &coll;
  public:
    collate_in(std::locale loc)
        : coll(std::use_facet<std::collate<char> >(loc)) {}
    bool operator()(const std::string &a, const std::string &b) const {
        // std::collate::compare() takes C-style string (begin, end)s and
        // returns values like strcmp or strcoll.  Compare to 0 for results
        // expected for a less<>-style comparator.
        return coll.compare(a.c_str(), a.c_str() + a.size(),
                            b.c_str(), b.c_str() + b.size()) < 0;
    }
};
int main() {
    std::vector<std::string> v;
    copy(std::istream_iterator<std::string>(std::cin),
         std::istream_iterator<std::string>(), back_inserter(v));
    // std::locale("") is the locale from the environment.  One could also
    // std::locale::global(std::locale("")) to set up this program's global
    // first, and then use locale() to get the global locale, or choose a
    // specific locale instead of using the environment's.
    sort(v.begin(), v.end(), collate_in(std::locale("")));
    copy(v.begin(), v.end(),
         std::ostream_iterator<std::string>(std::cout, "\n"));
    return 0;
}

$ cat >file
f
é
e
d
^D
$ LC_COLLATE=C ./a.out file
d
e
f
é
$ LC_COLLATE=en_US.utf8 ./a.out file
d
e
é
f

它引起了我的注意,std::locale::operator()(a, b)存在,避免了std::collate<>::compare(a, b) < 0我上面写的包装.

#include <algorithm>
#include <iostream>
#include <iterator>
#include <locale>
#include <string>
#include <vector>
int main() {
    std::vector<std::string> v;
    copy(std::istream_iterator<std::string>(std::cin),
         std::istream_iterator<std::string>(), back_inserter(v));
    sort(v.begin(), v.end(), std::locale(""));
    copy(v.begin(), v.end(),
         std::ostream_iterator<std::string>(std::cout, "\n"));
    return 0;
}