在训练为numpy类型/数组期间在每个时期保存图层权重？将TensorFlow变量转换为numpy数组？

Question

我是keras第一次使用该软件，并尝试编写一个自定义格式keras.callbacks.Callback，以节省期间每个模型层的权重fit。我在将类型转换keras.models.layers.weights为numpy数组（或从中可以提取原始类型值的任何东西）时遇到麻烦。

根据我的判断keras.models.layers.weights（对于我的模型），是一个tensorflow.python.ops.variables.Variable包含(1, 1)矩阵或(1,)矩阵的列表。我只是似乎无法获得该变量类型作为float（dtype矩阵的）。

这是我的问题的SSCCE。

import keras
import numpy as np

xONE = np.mat([[0], [1]])
yNOT = np.mat([[1], [0]])

# Simple Keras callback which saves not only loss and acc
# but also the layer weights at each epoch end
class History_LAW(keras.callbacks.Callback):
    def on_train_begin(self, logs={}):
        self.epoch = []
        self.weights = []
        self.history = {}
        self.weights.append(self.model.weights)

    def on_epoch_end(self, epoch, logs={}):
        logs = logs or {}
        self.epoch.append(epoch)
        self.weights.append(self.model.weights)
        for k, v in logs.items():
            self.history.setdefault(k, []).append(v)

# Used to programmatically generaly a simple model
# given the data that it is meant to fit to
def generateModel(xShape, yShape):
    model = keras.models.Sequential()
    model.add(
        keras.layers.Dense(
            yShape[1], input_shape = (xShape[1],)))
    model.add(keras.layers.Activation('hard_sigmoid'))
    model.summary()
    return model

# Generate a model based on the simply NOT relationship
model = generateModel(xONE.shape, yNOT.shape)
model.compile(
    optimizer = 'adam', 
    loss = 'binary_crossentropy', 
    metrics = ['accuracy'])

# Create the custom callback and pass it to fit
model_Hist = History_LAW()
model.fit(xONE, yNOT, 
    epochs = 4, batch_size = 2, verbose = 1, callbacks = [model_Hist])

# Display the raw weight structure and its type
print('========== Raw w output')
for weight in model_Hist.weights:
    print([w for w in weight])

print('========== Type of w')
for weight in model_Hist.weights:
    print([type(w) for w in weight])

我确实找到了这个SO答案，它建议使用eval()（我认为是一个TensorFlow函数）将张量（不确定确切的含义，但我认为那是适当的术语）“评估”为它们的基础类型-我认为这是一个numpy矩阵或数组。尽管遵循此步骤似乎会导致另一个错误的陷阱（传递session = tf.InteractiveSession)给eval，导致“张量图与会话图不同”）。

我发现了各种回调，它们可以在拟合期间记录图层权重，尽管据我所知，这些回调都没有以所需的格式返回它们。

如何将这些权重转换为某些numpy对象？除了我在SSCCE中提出的建议以外，还有没有更简单的方法可以实现我想要的？

---

供参考，上面的SSCCE的输出是：

========== Raw w output
[<tf.Variable 'dense_1/kernel:0' shape=(1, 1) dtype=float32_ref>, <tf.Variable 'dense_1/bias:0' shape=(1,) dtype=float32_ref>]
[<tf.Variable 'dense_1/kernel:0' shape=(1, 1) dtype=float32_ref>, <tf.Variable 'dense_1/bias:0' shape=(1,) dtype=float32_ref>]
[<tf.Variable 'dense_1/kernel:0' shape=(1, 1) dtype=float32_ref>, <tf.Variable 'dense_1/bias:0' shape=(1,) dtype=float32_ref>]
[<tf.Variable 'dense_1/kernel:0' shape=(1, 1) dtype=float32_ref>, <tf.Variable 'dense_1/bias:0' shape=(1,) dtype=float32_ref>]
[<tf.Variable 'dense_1/kernel:0' shape=(1, 1) dtype=float32_ref>, <tf.Variable 'dense_1/bias:0' shape=(1,) dtype=float32_ref>]
========== Type of w
[<class 'tensorflow.python.ops.variables.Variable'>, <class 'tensorflow.python.ops.variables.Variable'>]
[<class 'tensorflow.python.ops.variables.Variable'>, <class 'tensorflow.python.ops.variables.Variable'>]
[<class 'tensorflow.python.ops.variables.Variable'>, <class 'tensorflow.python.ops.variables.Variable'>]
[<class 'tensorflow.python.ops.variables.Variable'>, <class 'tensorflow.python.ops.variables.Variable'>]
[<class 'tensorflow.python.ops.variables.Variable'>, <class 'tensorflow.python.ops.variables.Variable'>]

Answer 1

我发现我需要使用该方法keras.model.layer.get_weights()来获取权重为numpy数组。

例如我的回调将更改为

def on_epoch_end(self, epoch, logs={}):
    logs = logs or {}
    self.epoch.append(epoch)
    for k, v in logs.items():
        self.history.setdefault(k, []).append(v)

    modelWeights = []
    for layer in model.layers:
        layerWeights = []
        for weight in layer.get_weights():
            layerWeights.append(weight)
        modelWeights.append(layerWeights)
    self.weights.append(modelWeights)