我在scikit中使用TfidfVectorizer学习从文本数据创建矩阵.现在我需要保存此对象以便以后重用.我试图使用pickle,但它给出了以下错误.
loc=open('vectorizer.obj','w')
pickle.dump(self.vectorizer,loc)
*** TypeError: can't pickle instancemethod objects
我尝试在sklearn.externals中使用joblib,这再次给出了类似的错误.有没有办法保存这个对象,以便我以后可以重用它？
这是我的完整对象:
class changeToMatrix(object):
def __init__(self,ngram_range=(1,1),tokenizer=StemTokenizer()):
from sklearn.feature_extraction.text import TfidfVectorizer
self.vectorizer = TfidfVectorizer(ngram_range=ngram_range,analyzer='word',lowercase=True,\
token_pattern='[a-zA-Z0-9]+',strip_accents='unicode',tokenizer=tokenizer)
def load_ref_text(self,text_file):
textfile = open(text_file,'r')
lines=textfile.readlines()
textfile.close()
lines = ' '.join(lines)
sent_tokenizer = nltk.data.load('tokenizers/punkt/english.pickle')
sentences = [ sent_tokenizer.tokenize(lines.strip()) ]
sentences1 = [item.strip().strip('.') for sublist in sentences for item in sublist]
chk2=pd.DataFrame(self.vectorizer.fit_transform(sentences1).toarray()) #vectorizer is transformed in this step
return sentences1,[chk2]
def get_processed_data(self,data_loc):
ref_sentences,ref_dataframes=self.load_ref_text(data_loc)
loc=open("indexedData/vectorizer.obj","w")
pickle.dump(self.vectorizer,loc) #getting error here
loc.close()
return ref_sentences,ref_dataframes
我正在使用OneVsRestClassifier来解决多标签分类问题.我正在将RandomForestClassifier传递给它.
from sklearn.multiclass import OneVsRestClassifier
from sklearn.ensemble import RandomForestClassifier
clf = OneVsRestClassifier(RandomForestClassifier(random_state=0,class_weight='auto',min_samples_split=10,n_estimators=50))
clf.fit(train,dv_train)
print clf.feature_importances_
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'OneVsRestClassifier' object has no attribute 'feature_importances_'
如何在OneVsRestClassifier中获得每个随机森林的特征重要性？