小编Iss*_*sac的帖子

所有,

当您使用大量样本训练大型模型时,某些样本可能会在参数更新时导致NaN梯度.

我想找出这些样品.同时我不希望这批样本的梯度更新模型的参数,因为它可能导致模型的参数为NaN.

所以任何人都有好主意处理这个问题？

我的代码如下:

    # Create an optimizer.
    params = tf.trainable_variables()
    opt = tf.train.AdamOptimizer(1e-3)
    gradients = tf.gradients(self.loss, params)

    max_gradient_norm = 10
    clipped_gradients, self.gradient_norms = tf.clip_by_global_norm(gradients,
                                                     max_gradient_norm)

    self.optimizer = opt.apply_gradients(zip(clipped_gradients, params))

我对Theano的实施有疑问.theano如何通过以下函数(T.grad)获得每个损失函数的梯度？谢谢您的帮助.

 gparams = T.grad(cost, self.params) 

读取包列表...完成构建依赖关系树
读取状态信息...完成某些包无法安装.这可能意味着您已经请求了一个不可能的情况,或者如果您正在使用不稳定的分发,某些必需的包尚未创建或已被移出Incoming.以下信息可能有助于解决此问题:

The following packages have unmet dependencies:
 libopencv-dev : Depends: libopencv-objdetect-dev (= 2.4.8+dfsg1-2ubuntu1) but it is not going to be installed
                 Depends: libopencv-highgui-dev (= 2.4.8+dfsg1-2ubuntu1) but it is not going to be installed
                 Depends: libopencv-legacy-dev (= 2.4.8+dfsg1-2ubuntu1) but it is not going to be installed
                 Depends: libopencv-contrib-dev (= 2.4.8+dfsg1-2ubuntu1) but it is not going to be installed
                 Depends: libopencv-videostab-dev (= 2.4.8+dfsg1-2ubuntu1) but it is not going to be installed
                 Depends: libopencv-superres-dev (= 2.4.8+dfsg1-2ubuntu1) but it is not going to …