小编Abr*_*ial的帖子

我在使用utf-8编码读取文本文件时遇到问题我正在使用带有netbeans 7.2.1平​​台的java

我已经配置了java项目来处理UTF-8 javaproject ==>右键单击==> properties ==> source ==> UTF-8

但仍然得到未知的字符输出:

代码:

File fileDirs = new File("C:\\file.txt");

BufferedReader in = new BufferedReader(
new InputStreamReader(new FileInputStream(fileDirs), "UTF-8"));

String str;

while ((str = in.readLine()) != null) {
    System.out.println(str);
}

还有其他想法吗？

谢谢

我正在使用Pipeline和FeatureUnion的scikit-learn来从不同的输入中提取特征.我的数据集中的每个样本(实例)都指的是具有不同长度的文档.我的目标是独立计算每个文档的顶部tfidf,但我不断收到此错误消息:

ValueError:blocks [0,:]具有不兼容的行维度.得到块[0,1] .shape [0] == 1,预计2000.

2000是训练数据的大小.这是主要代码:

book_summary= Pipeline([
   ('selector', ItemSelector(key='book')),
   ('tfidf', TfidfVectorizer(analyzer='word', ngram_range(1,3), min_df=1, lowercase=True, stop_words=my_stopword_list, sublinear_tf=True))
])

book_contents= Pipeline([('selector3', book_content_count())]) 

ppl = Pipeline([
    ('feats', FeatureUnion([
         ('book_summary', book_summary),
         ('book_contents', book_contents)])),
    ('clf', SVC(kernel='linear', class_weight='balanced') ) # classifier with cross fold 5
]) 

我写了两个类来处理每个管道功能.我的问题是book_contents管道,它主要处理每个样本并独立返回每本书的TFidf矩阵.

class book_content_count(): 
  def count_contents2(self, bookid):
        book = open('C:/TheCorpus/'+str(int(bookid))+'_book.csv', 'r')       
        book_data = pd.read_csv(book, header=0, delimiter=',', encoding='latin1',error_bad_lines=False,dtype=str)
                      corpus=(str([user_data['text']]).strip('[]')) 
        return corpus

    def transform(self, data_dict, y=None):
        data_dict['bookid'] #from here take the name 
        text=data_dict['bookid'].apply(self.count_contents2)
        vec_pipe= Pipeline([('vec', TfidfVectorizer(min_df = 1,lowercase …