我有兴趣挖掘如何在tensorflow中实现FTRL的细节.我在文件"gen_training_ops.py"夹中的文件中找到了一些信息/tensorflow/python/training.在该文件中,FTRL算法的公式描述如下:
def apply_ftrl(var, accum, linear, grad, lr, l1, l2, lr_power,
use_locking=None, name=None):
r"""Update '*var' according to the Ftrl-proximal scheme
accum_new = accum + grad * grad ------ (1)
linear += grad + (accum_new^(-lr_power) - accum^(-lr_power)) / lr * var ------ (2)
quadratic = 1.0 / (accum_new^(lr_power) * lr) + 2 * l2 ------ (3)
var = (sign(linear) * l1 - linear) / quadratic if |linear| > l1 else 0.0 ------(4)
accum = accum_new ------(5)
我还在阅读KDD'13中的谷歌"广告点击预测:来自战壕的观点"一文.FTRL算法的公式在本文的第2页给出.这两种实现方式比较,我们发现一些连接:
var …