Java分裂字符串表现

Question

这是我的应用程序中的当前代码:

String[] ids = str.split("/");

在分析应用程序时,我注意到分割字符串花费了不可忽略的时间.

我还了解到split实际上需要一个正则表达式,这对我来说没用.

所以我的问题是,为了优化字符串拆分,我可以使用哪种替代方法？我见过StringUtils.split但它更快吗？

(我会自己尝试和测试,但分析我的应用程序需要花费很多时间,所以如果有人已经知道答案,那么节省了一些时间)

Answer 1

String.split(String)如果您的模式只有一个字符长,则不会创建正则表达式.当按单个字符拆分时,它将使用非常有效的专用代码.StringTokenizer在这种特殊情况下,速度并不快.

这是在OpenJDK7/OracleJDK7中引入的.这是一个错误报告和提交.我在这里做了一个简单的基准测试.

$ java -version
java version "1.8.0_20"
Java(TM) SE Runtime Environment (build 1.8.0_20-b26)
Java HotSpot(TM) 64-Bit Server VM (build 25.20-b23, mixed mode)

$ java Split
split_banthar: 1231
split_tskuzzy: 1464
split_tskuzzy2: 1742
string.split: 1291
StringTokenizer: 1517

Answer 2

如果你可以使用第三方库,那么当你不要求它时,Guava Splitter不会产生正则表达式的开销,并且作为一般规则非常快.(披露:我向番石榴捐款.)

Iterable<String> split = Splitter.on('/').split(string);

(另外,Splitter是作为一项规则更具可预测性比String.split.)

Answer 3

StringTokenizer 对于像这样的简单解析来说要快得多(我做了一些基准测试并且你获得了巨大的加速).

StringTokenizer st = new StringTokenizer("1/2/3","/");
String[] arr = st.countTokens();
arr[0] = st.nextToken();

如果你想要获得更多的性能,你也可以手动完成:

String s = "1/2/3"
char[] c = s.toCharArray();
LinkedList<String> ll = new LinkedList<String>();
int index = 0;

for(int i=0;i<c.length;i++) {
    if(c[i] == '/') {
        ll.add(s.substring(index,i));
        index = i+1;
    }
}

String[] arr = ll.size();
Iterator<String> iter = ll.iterator();
index = 0;

for(index = 0; iter.hasNext(); index++)
    arr[index++] = iter.next();

Answer 4

鉴于我正在大规模工作，我认为这将有助于提供更多基准测试，包括我自己的一些实现（我在空格上分开，但这应该说明一般需要多长时间）：

我正在处理一个 426 MB 的文件，包含 2622761 行。唯一的空白是普通空格 (" ") 和行 ("\n")。

首先，我用空格替换所有行，并对一大行进行基准解析：

.split(" ")
Cumulative time: 31.431366952 seconds

.split("\s")
Cumulative time: 52.948729489 seconds

splitStringChArray()
Cumulative time: 38.721338004 seconds

splitStringChList()
Cumulative time: 12.716065893 seconds

splitStringCodes()
Cumulative time: 1 minutes, 21.349029036000005 seconds

splitStringCharCodes()
Cumulative time: 23.459840685 seconds

StringTokenizer
Cumulative time: 1 minutes, 11.501686094999997 seconds

然后我逐行对分割进行基准测试（这意味着函数和循环会执行多次，而不是一次全部完成）：

.split(" ")
Cumulative time: 3.809014174 seconds

.split("\s")
Cumulative time: 7.906730124 seconds

splitStringChArray()
Cumulative time: 4.06576739 seconds

splitStringChList()
Cumulative time: 2.857809996 seconds

Bonus: splitStringChList(), but creating a new StringBuilder every time (the average difference is actually more like .42 seconds):
Cumulative time: 3.82026621 seconds

splitStringCodes()
Cumulative time: 11.730249921 seconds

splitStringCharCodes()
Cumulative time: 6.995555826 seconds

StringTokenizer
Cumulative time: 4.500008172 seconds

这是代码：

// Use a char array, and count the number of instances first.
public static String[] splitStringChArray(String str, StringBuilder sb) {
    char[] strArray = str.toCharArray();
    int count = 0;
    for (char c : strArray) {
        if (c == ' ') {
            count++;
        }
    }
    String[] splitArray = new String[count+1];
    int i=0;
    for (char c : strArray) {
        if (c == ' ') {
            splitArray[i] = sb.toString();
            sb.delete(0, sb.length());
        } else {
            sb.append(c);
        }
    }
    return splitArray;
}

// Use a char array but create an ArrayList, and don't count beforehand.
public static ArrayList<String> splitStringChList(String str, StringBuilder sb) {
    ArrayList<String> words = new ArrayList<String>();
    words.ensureCapacity(str.length()/5);
    char[] strArray = str.toCharArray();
    int i=0;
    for (char c : strArray) {
        if (c == ' ') {
            words.add(sb.toString());
            sb.delete(0, sb.length());
        } else {
            sb.append(c);
        }
    }
    return words;
}

// Using an iterator through code points and returning an ArrayList.
public static ArrayList<String> splitStringCodes(String str) {
    ArrayList<String> words = new ArrayList<String>();
    words.ensureCapacity(str.length()/5);
    IntStream is = str.codePoints();
    OfInt it = is.iterator();
    int cp;
    StringBuilder sb = new StringBuilder();
    while (it.hasNext()) {
        cp = it.next();
        if (cp == 32) {
            words.add(sb.toString());
            sb.delete(0, sb.length());
        } else {
            sb.append(cp);
        }
    }

    return words;
}

// This one is for compatibility with supplementary or surrogate characters (by using Character.codePointAt())
public static ArrayList<String> splitStringCharCodes(String str, StringBuilder sb) {
    char[] strArray = str.toCharArray();
    ArrayList<String> words = new ArrayList<String>();
    words.ensureCapacity(str.length()/5);
    int cp;
    int len = strArray.length;
    for (int i=0; i<len; i++) {
        cp = Character.codePointAt(strArray, i);
        if (cp == ' ') {
            words.add(sb.toString());
            sb.delete(0, sb.length());
        } else {
            sb.append(cp);
        }
    }

    return words;
}

这就是我使用 StringTokenizer 的方式：

    StringTokenizer tokenizer = new StringTokenizer(file.getCurrentString());
    words = new String[tokenizer.countTokens()];
    int i = 0;
    while (tokenizer.hasMoreTokens()) {
        words[i] = tokenizer.nextToken();
        i++;
    }