如何在使用简单的CoreNLP API时设置tokenizer选项？

Question

我知道CoreNLP中提供的tokenizer选项,我知道如何在标准版本中设置它们.

有没有办法untokenizable=noneKeep在使用Simple CoreNLP接口时传递选项,例如？

Answer 1

您可以构建具有属性的文档。

package edu.stanford.nlp.examples;

import edu.stanford.nlp.simple.*;

import java.util.*;

public class SimpleExample {

    public static void main(String[] args) {
        Properties props = new Properties();
        props.setProperty("tokenize.options", "untokenizable=allKeep");
        Document doc = new Document(props, "Joe Smith was born in California.  He moved to Chicago last year.");
        for (Sentence sent : doc.sentences()) {
            System.out.println(sent.tokens());
            System.out.println(sent.nerTags());
            System.out.println(sent.parse());
        }
    }

}