实施 n-gram 进行下一个单词预测

Question

我正在尝试利用三元组来预测下一个单词。

我已经能够上传语料库并根据频率识别最常见的三元组。我在 R 中使用了“ngrams”、“RWeka”和“tm”包。我按照这个问题寻求指导：

我需要什么算法来查找 n 元语法？

text1<-readLines("MyText.txt", encoding = "UTF-8")
corpus <- Corpus(VectorSource(text1))

BigramTokenizer <- function(x) NGramTokenizer(x, Weka_control(min = 3, max =       3))
tdm <- TermDocumentMatrix(corpus, control = list(tokenize =      BigramTokenizer)) 

如果用户要输入一组单词，我将如何生成下一个单词？例如，如果用户输入“can of”，我将如何检索三个最可能的单词（例如啤酒、苏打水、油漆等）？

Answer 1

作为初学者，这是一种方法：

f <- function(queryHistoryTab, query, n = 2) {
  require(tau)
  trigrams <- sort(textcnt(rep(tolower(names(queryHistoryTab)), queryHistoryTab), method = "string", n = length(scan(text = query, what = "character", quiet = TRUE)) + 1))
  query <- tolower(query)
  idx <- which(substr(names(trigrams), 0, nchar(query)) == query)
  res <- head(names(sort(trigrams[idx], decreasing = TRUE)), n)
  res <- substr(res, nchar(query) + 2, nchar(res))
  return(res)
}
f(c("Can of beer" = 3, "can of Soda" = 2, "A can of water" = 1, "Buy me a can of soda, please" = 2), "Can of")
# [1] "soda" "beer"