将隐藏层添加到第一个简单的神经网络

Question

我已经阅读了许多张量流的初学者书籍并制作了这段代码.这只需要N维数据并输出1维.

它很迷人!

现在,我想为此添加隐藏层,但是我无法创建它并找到简单的教程或示例来了解如何添加隐藏层.

有良好的做法或想法？或"为此样本添加隐藏层"是进一步学习的正确方法???

tf.set_random_seed(0)
N = 10

w = tf.Variable(tf.zeros([N,1])) 
b = tf.Variable(tf.zeros([1]))

x = tf.placeholder(tf.float32,shape=[None,N]) 
t = tf.placeholder(tf.float32,shape=[None,1])
y = tf.nn.sigmoid(tf.matmul(x,w) + b)

cross_entropy = - tf.reduce_sum(t * tf.log(y) + (1 -t) * tf.log(1 -y))
train_step = tf.train.GradientDescentOptimizer(0.1).minimize(cross_entropy)
correct_prediction = tf.equal(tf.to_float(tf.greater(y,0.5)),t)


init = tf.global_variables_initializer()
sess = tf.Session()
sess.run(init)

for epoch in range(2000):
    sess.run(train_step,feed_dict={
        x: X,
        t: Y
    })

classified = correct_prediction.eval(session=sess,feed_dict={
    x:X,
    t:Y
    })

print(classified)

prob = y.eval(session=sess ,feed_dict={
    x:X,
    t:Y
})
print(prob)

print('w:',sess.run(w))
print('b:',sess.run(b))

Answer 1

隐藏层将位于输入和输出层之间,因此输入隐藏层将通过大小的权重连接[input_size, hidden_size],隐藏输出层将通过大小的权重连接[hidden_size, output_size].并且每层都会有激活.

您的代码应如下所示:

N = 10
n_hidden = 20
# dont initialise weights to zero but to a small number
w_h = tf.Variable(tf.truncated_normal([N,n_hidden], stddev=0.001)) 
b_h = tf.Variable(tf.zeros([n_hidden]))

w = tf.Variable(tf.truncated_normal([n_hidden,1], stddev=0.001)) 
b = tf.Variable(tf.zeros([1]))

x = tf.placeholder(tf.float32,shape=[None,N]) 
t = tf.placeholder(tf.float32,shape=[None,1])

h = tf.nn.relu(tf.matmul(x, w_h) + b_h)

y = tf.matmul(h, w) + b

#remove sigmoid from last layer and use the stable implementation:
cross_entropy = tf.reduce_mean(tf.nn.sigmoid_cross_entropy_with_logits(logits=y, labels=t))
train_step = tf.train.GradientDescentOptimizer(0.1).minimize(cross_entropy)
correct_prediction = tf.equal(tf.to_float(tf.greater(y,0.5)),t)