将topicmodels输出转换为JSON

Question

我使用以下函数将topicmodels输出转换为JSON输出以在ldavis中使用.

topicmodels_json_ldavis <- function(fitted, corpus, doc_term){
     ## Required packages
     library(topicmodels)
     library(dplyr)
     library(stringi)
     library(tm)
     library(LDAvis)

     ## Find required quantities
     phi <- posterior(fitted)$terms %>% as.matrix
     theta <- posterior(fitted)$topics %>% as.matrix
     vocab <- colnames(phi)
     doc_length <- vector()
     for (i in 1:length(corpus)) {
          temp <- paste(corpus[[i]]$content, collapse = ' ')
          doc_length <- c(doc_length, stri_count(temp, regex = '\\S+'))
     }
     temp_frequency <- inspect(doc_term)
     freq_matrix <- data.frame(ST = colnames(temp_frequency),
                               Freq = colSums(temp_frequency))
     rm(temp_frequency)

     ## Convert to json
     json_lda <- LDAvis::createJSON(phi = phi, theta = theta,
                                    vocab = vocab,
                                    doc.length = doc_length,
                                    term.frequency = freq_matrix$Freq)

     return(json_lda)
}

但是我收到以下错误

LDAvis :: createJSON中的错误(phi = phi,theta = theta,vocab = vocab,doc.length = doc_length,:doc.length的长度不等于theta中的行数;两者应该等于文档的数量在数据中.

这是我的完整代码:

data <- read.csv("textmining.csv")


corpus <- Corpus(DataframeSource(data.frame(data$reasonforleaving))) 

# Remove punctuations and numbers because they are generally uninformative.
corpus <- tm_map(corpus, removePunctuation)
corpus <- tm_map(corpus, removeNumbers)
# Convert all words to lowercase.
corpus <- tm_map(corpus, content_transformer(tolower))
# Remove stopwords such as "a", "the", etc.
corpus <- tm_map(corpus, removeWords, stopwords("english"))
# Use the SnowballC package to do stemming.
library(SnowballC)
corpus <- tm_map(corpus, stemDocument)


# remove extra words
toSpace <- content_transformer(function (x , pattern ) gsub(pattern, " ", x))
corpus <- tm_map(corpus, toSpace, "still")
corpus <- tm_map(corpus, toSpace, "also")

# Remove excess white spaces between words.

corpus <- tm_map(corpus, stripWhitespace)
# Inspect the first document to see what it looks like.
corpus[[1]]$content 

dtm <- DocumentTermMatrix(corpus)

# remove empty documents
library(slam)
dtm = dtm[row_sums(dtm)>0,]

# Use topicmodels package to conduct LDA analysis.

burnin <- 500
iter <- 1000
keep <- 30
k <- 5

result55 <- LDA(dtm, 5)
ldaoutput = topicmodels_json_ldavis(result55,corpus, dtm)

你知道我收到错误的原因吗？

谢谢

Answer 1

我有相同的代码相同的问题,并在此处找到此函数:

topicmodels2LDAvis <- function(x, ...){
    post <- topicmodels::posterior(x)
    if (ncol(post[["topics"]]) < 3) stop("The model must contain > 2 topics")
    mat <- x@wordassignments
    LDAvis::createJSON(
        phi = post[["terms"]], 
        theta = post[["topics"]],
        vocab = colnames(post[["terms"]]),
        doc.length = slam::row_sums(mat, na.rm = TRUE),
        term.frequency = slam::col_sums(mat, na.rm = TRUE)
    )
}

使用起来要简单得多,只需将LDA结果作为参数:

result55 <- LDA(dtm, 5)
serVis(topicmodels2LDAvis(result55))