小编joe*_*oel的帖子

我最近在 jupyter 中安装了 imblearn 包

!pip show imbalanced-learn

但我无法导入这个包。

from tensorflow.keras import backend
from imblearn.over_sampling import SMOTE

我收到以下错误

---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-20-f19c5a0e54af> in <module>
      1 # from sklearn.utils import resample
      2 from tensorflow.keras import backend
----> 3 from imblearn.over_sampling import SMOTE
      4 
      5 

~/.virtualenvs/p3/lib/python3.6/site-packages/imblearn/__init__.py in <module>
     32     Module which allowing to create pipeline with scikit-learn estimators.
     33 """
---> 34 from . import combine
     35 from . import ensemble
     36 from . import exceptions

~/.virtualenvs/p3/lib/python3.6/site-packages/imblearn/combine/__init__.py in …

我无法在 python 3 虚拟环境中的 jupyter notebook 中导入category_encoders模块。

错误

---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-15-86725efc8d1e> in <module>()
      9 from plotly import graph_objs
     10 from datetime import datetime
---> 11 import category_encoders as ce
     12 
     13 import sklearn

ModuleNotFoundError: No module named 'category_encoders'

“哪个点”的输出

/opt/virtual_env/py3/bin/pip

“pip show category_encoders”的输出是

Name: category-encoders
Version: 1.3.0
Summary: A collection sklearn transformers to encode categorical variables as numeric
Home-page: https://github.com/wdm0006/categorical_encoding
Author: Will McGinnis
Author-email: will@pedalwrencher.com
License: BSD
Location: /opt/virtual_env/py3/lib/python3.6/site-packages
Requires: numpy, pandas, statsmodels, …

我正在使用Tensorflow BERT语言模型处理二进制分类问题。这是Google colab 的链接。保存并加载模型后，进行预测时出现错误。

保存模型

def serving_input_receiver_fn():
  feature_spec = {
      "input_ids" : tf.FixedLenFeature([MAX_SEQ_LENGTH], tf.int64),
      "input_mask" : tf.FixedLenFeature([MAX_SEQ_LENGTH], tf.int64),
      "segment_ids" : tf.FixedLenFeature([MAX_SEQ_LENGTH], tf.int64),
      "label_ids" :  tf.FixedLenFeature([], tf.int64)
  }
  serialized_tf_example = tf.placeholder(dtype=tf.string,
                                         shape=[None],
                                         name='input_example_tensor')
  print(serialized_tf_example.shape)
  receiver_tensors = {'example': serialized_tf_example}
  features = tf.parse_example(serialized_tf_example, feature_spec)
  return tf.estimator.export.ServingInputReceiver(features, receiver_tensors)

export_path = '/content/drive/My Drive/binary_class/bert/'
estimator._export_to_tpu = False  # this is important
estimator.export_saved_model(export_dir_base=export_path,serving_input_receiver_fn=serving_input_receiver_fn)

预测虚拟文字

pred_sentences = [
  "A novel, simple method to get insights from reviews"
]

def getPrediction1(in_sentences):
  labels = ["Irrelevant", "Relevant"]
  input_examples = …

我正在尝试对 Huggingface 库中的 pytorch 预训练模型进行动态量化（量化权重和激活）。我已经参考了此链接，发现动态量化最合适。我将在 CPU 上使用量化模型。

链接到这里的拥抱模型。

火炬版本：1.6.0（通过pip安装）

预训练模型

tokenizer = AutoTokenizer.from_pretrained("microsoft/BiomedNLP-PubMedBERT-base-uncased-abstract-fulltext")
model = AutoModel.from_pretrained("microsoft/BiomedNLP-PubMedBERT-base-uncased-abstract-fulltext")

动态量化

quantized_model = torch.quantization.quantize_dynamic(
    model, qconfig_spec={torch.nn.Linear}, dtype=torch.qint8
)

print(quantized_model)

错误

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-7-df2355c17e0b> in <module>
      1 quantized_model = torch.quantization.quantize_dynamic(
----> 2     model, qconfig_spec={torch.nn.Linear}, dtype=torch.qint8
      3 )
      4 
      5 print(quantized_model)

~/.virtualenvs/python3/lib64/python3.6/site-packages/torch/quantization/quantize.py in quantize_dynamic(model, qconfig_spec, dtype, mapping, inplace)
    283     model.eval()
    284     propagate_qconfig_(model, qconfig_spec)
--> 285     convert(model, mapping, inplace=True)
    286     _remove_qconfig(model)
    287     return model

~/.virtualenvs/python3/lib64/python3.6/site-packages/torch/quantization/quantize.py in convert(module, mapping, …

我正在尝试在 R 中转换 Python 中的双 for 循环。当我在粘贴函数中使用它时，它可以正常工作。我试图弄清楚为什么没有它就无法访问这些内容。

在Python中

l1 = ['appetizer','main course']
l2 = ['italian','mexican','french']

for i in l1:
    for j in l2:
        print(i,j)

在R中

l1 = list('appetizer','main course')
l2 = list('italian','mexican','french')

这会引发错误

for (i in l1) {
  for (j in l2) {
    print(i,j)
  }
}

错误

Error in print.default(i, j) : invalid 'digits' argument
In addition: Warning message:
In print.default(i, j) : NAs introduced by coercion
> for (i in l1) {

这不会引发错误

for (i in l1) {
  for …

我试图在 siaseme LSTM 中重现结果以比较这里两个句子的语义相似性：- https://github.com/dhwajraj/deep-siamese-text-similarity

我正在使用 tensorflow 1.4 & python 2.7

train.py 工作正常。为了评估模型，我创建了一个 match_valid.tsv 文件，它是那里可用的“train_snli.txt”的子集。我修改了 input_helpers.py 文件中的 getTsvTestData 函数。

def getTsvTestData(self, filepath):
        print("Loading testing/labelled data from "+filepath+"\n")
        x1=[]
        x2=[]
        y=[]
        # positive samples from file
        for line in open(filepath):
            l=line.strip().split("\t")
            if len(l)<3:
                continue
            x1.append(l[1].lower()) # text
            x2.append(l[0].lower()) # text
            y.append(int(l[2])) # similarity score 0 or 1
        return np.asarray(x1),np.asarray(x2),np.asarray(y)

我从 eval.py 中的这部分代码中收到错误

for db in batches:
            x1_dev_b,x2_dev_b,y_dev_b = zip(*db)
            #x1_dev_b = tf.convert_to_tensor(x1_dev_b,)
            print("type x1_dev_b {}".format(type(x1_dev_b))) # tuple
            print("type x2_dev_b {}".format(type(x2_dev_b))) # …