Viv*_*ath 12 backpropagation neural-network
我正在通过使用随机梯度下降的反向传播训练XOR神经网络.将神经网络的权重初始化为-0.5和0.5之间的随机值.神经网络在80%的时间内成功训练自己.然而有时它会在反向传播时"卡住".通过"卡住",我的意思是我开始看到纠错率下降.例如,在成功培训期间,总误差会随着网络的学习而迅速下降,如下所示:
...
...
Total error for this training set: 0.0010008071327708653
Total error for this training set: 0.001000750550254843
Total error for this training set: 0.001000693973929822
Total error for this training set: 0.0010006374037948094
Total error for this training set: 0.0010005808398488103
Total error for this training set: 0.0010005242820908169
Total error for this training set: 0.0010004677305198344
Total error for this training set: 0.0010004111851348654
Total error for this training set: 0.0010003546459349181
Total error for this training set: 0.0010002981129189812
Total error for this training set: 0.0010002415860860656
Total error for this training set: 0.0010001850654351723
Total error for this training set: 0.001000128550965301
Total error for this training set: 0.0010000720426754587
Total error for this training set: 0.0010000155405646494
Total error for this training set: 9.99959044631871E-4
Testing trained XOR neural network
0 XOR 0: 0.023956746649767453
0 XOR 1: 0.9736079194769579
1 XOR 0: 0.9735670067093437
1 XOR 1: 0.045068688874314006
然而,当它被卡住时,总错误正在减少,但似乎是在下降的速度:
...
...
Total error for this training set: 0.12325486644721295
Total error for this training set: 0.12325486642503929
Total error for this training set: 0.12325486640286581
Total error for this training set: 0.12325486638069229
Total error for this training set: 0.12325486635851894
Total error for this training set: 0.12325486633634561
Total error for this training set: 0.1232548663141723
Total error for this training set: 0.12325486629199914
Total error for this training set: 0.12325486626982587
Total error for this training set: 0.1232548662476525
Total error for this training set: 0.12325486622547954
Total error for this training set: 0.12325486620330656
Total error for this training set: 0.12325486618113349
Total error for this training set: 0.12325486615896045
Total error for this training set: 0.12325486613678775
Total error for this training set: 0.12325486611461482
Total error for this training set: 0.1232548660924418
Total error for this training set: 0.12325486607026936
Total error for this training set: 0.12325486604809655
Total error for this training set: 0.12325486602592373
Total error for this training set: 0.12325486600375107
Total error for this training set: 0.12325486598157878
Total error for this training set: 0.12325486595940628
Total error for this training set: 0.1232548659372337
Total error for this training set: 0.12325486591506139
Total error for this training set: 0.12325486589288918
Total error for this training set: 0.12325486587071677
Total error for this training set: 0.12325486584854453
在我阅读神经网络时,我偶然发现了关于局部最小值和全局最小值的讨论,以及神经网络如何真正"知道"它应该朝向哪个最小值.
我的网络是否陷入了局部最小值而不是全局最小值？
是的,神经网络可能会陷入局部最小值,具体取决于误差表面.然而,这个摘要表明在XOR问题的错误表面中没有局部最小值.但是我无法获得全文,所以我无法验证作者做了什么来证明这一点以及它如何适用于您的问题.
也可能有其他因素导致这个问题.例如,如果你在一些陡峭的山谷中下降非常快,如果你只是使用一阶梯度下降,你可能会到达相反的斜率并且一直来回反弹.你可以尝试在每次迭代时给出所有权重的平均变化,以测试你是否真的有一个"卡住"的网络,或者更确切地说是一个刚刚进入限制周期的网络.
你应该首先尝试摆弄你的参数(学习率,动力,如果你实施它等).如果您可以通过更改参数来解决问题,那么您的算法可能没问题.