Mag*_*eed 5 python machine-learning computer-vision keras tensorflow
我正在开发一个用于年龄和性别识别的android应用程序。我在GitHub中找到了一个有用的模型。他们正在基于第一名获奖论文构建Keras模型（tensorflow后端）。他们提供了python模块来训练和构建网络，已经训练好的权重文件供下载和使用，以及网络摄像头上的有效演示。
我想在演示中使用提供的权重将其模型转换为.pb文件，以便它也可以在android上执行。
我使用以下代码进行了与模型相关的较小修改：
from keras.models import Sequential
from keras.models import model_from_json
from keras import backend as K
import tensorflow as tf
from tensorflow.python.tools import freeze_graph
import os
# Load existing model.
with open("model.json",'r') as f:
modelJSON = f.read()
model = model_from_json(modelJSON)
model.load_weights("weights.18-4.06.hdf5")
print(model.summary())
# All new operations will be in test mode from now on.
K.set_learning_phase(0)
# Serialize the model and get its weights, for quick re-building.
config = model.get_config()
weights = model.get_weights()
# Re-build a model where the learning phase is now hard-coded to 0.
#new_model = model.from_config(config)
#new_model.set_weights(weights)
temp_dir = "graph"
checkpoint_prefix = os.path.join(temp_dir, "saved_checkpoint")
checkpoint_state_name = "checkpoint_state"
input_graph_name = "input_graph.pb"
output_graph_name = "output_graph.pb"
# Temporary save graph to disk without weights included.
saver = tf.train.Saver()
checkpoint_path = saver.save(K.get_session(), checkpoint_prefix, global_step=0, latest_filename=checkpoint_state_name)
tf.train.write_graph(K.get_session().graph, temp_dir, input_graph_name)
input_graph_path = os.path.join(temp_dir, input_graph_name)
input_saver_def_path = ""
input_binary = False
output_node_names = "dense_1/Softmax,dense_2/Softmax" # model dependent
restore_op_name = "save/restore_all"
filename_tensor_name = "save/Const:0"
output_graph_path = os.path.join(temp_dir, output_graph_name)
clear_devices = False
# Embed weights inside the graph and save to disk.
freeze_graph.freeze_graph(input_graph_path, input_saver_def_path,
input_binary, checkpoint_path,
output_node_names, restore_op_name,
filename_tensor_name, output_graph_path,
clear_devices, "")
我直接在演示中生成了model.json文件。带有model.json的demo.py文件的主要功能代码是：
def main():
args = get_args()
depth = args.depth
k = args.width
weight_file = args.weight_file
if not weight_file:
weight_file = get_file("weights.18-4.06.hdf5", pretrained_model, cache_subdir="pretrained_models",
file_hash=modhash, cache_dir=os.path.dirname(os.path.abspath(__file__)))
# for face detection
detector = dlib.get_frontal_face_detector()
# load model and weights
img_size = 64
model = WideResNet(img_size, depth=depth, k=k)()
model.load_weights(weight_file)
print(model.summary())
# write model to json
model_json = model.to_json()
with open("model.json", "w") as json_file:
json_file.write(model_json)
for img in yield_images():
input_img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
img_h, img_w, _ = np.shape(input_img)
# detect faces using dlib detector
detected = detector(input_img, 1)
faces = np.empty((len(detected), img_size, img_size, 3))
if len(detected) > 0:
for i, d in enumerate(detected):
x1, y1, x2, y2, w, h = d.left(), d.top(), d.right() + 1, d.bottom() + 1, d.width(), d.height()
xw1 = max(int(x1 - 0.4 * w), 0)
yw1 = max(int(y1 - 0.4 * h), 0)
xw2 = min(int(x2 + 0.4 * w), img_w - 1)
yw2 = min(int(y2 + 0.4 * h), img_h - 1)
cv2.rectangle(img, (x1, y1), (x2, y2), (255, 0, 0), 2)
# cv2.rectangle(img, (xw1, yw1), (xw2, yw2), (255, 0, 0), 2)
faces[i, :, :, :] = cv2.resize(img[yw1:yw2 + 1, xw1:xw2 + 1, :], (img_size, img_size))
# predict ages and genders of the detected faces
results = model.predict(faces)
predicted_genders = results[0]
ages = np.arange(0, 101).reshape(101, 1)
predicted_ages = results[1].dot(ages).flatten()
# draw results
for i, d in enumerate(detected):
label = "{}, {}".format(int(predicted_ages[i]),
"F" if predicted_genders[i][0] > 0.5 else "M")
draw_label(img, (d.left(), d.top()), label)
cv2.imshow("result", img)
key = cv2.waitKey(30)
if key == 27:
break
if __name__ == '__main__':
main()
该代码成功编译并生成多个检查点文件以及一个.pb文件。
这是模型的图形摘要：
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
input_1 (InputLayer) (None, 64, 64, 3) 0
__________________________________________________________________________________________________
conv2d_1 (Conv2D) (None, 64, 64, 16) 432 input_1[0][0]
__________________________________________________________________________________________________
batch_normalization_1 (BatchNor (None, 64, 64, 16) 64 conv2d_1[0][0]
__________________________________________________________________________________________________
activation_1 (Activation) (None, 64, 64, 16) 0 batch_normalization_1[0][0]
__________________________________________________________________________________________________
conv2d_2 (Conv2D) (None, 64, 64, 128) 18432 activation_1[0][0]
__________________________________________________________________________________________________
batch_normalization_2 (BatchNor (None, 64, 64, 128) 512 conv2d_2[0][0]
__________________________________________________________________________________________________
activation_2 (Activation) (None, 64, 64, 128) 0 batch_normalization_2[0][0]
__________________________________________________________________________________________________
conv2d_3 (Conv2D) (None, 64, 64, 128) 147456 activation_2[0][0]
__________________________________________________________________________________________________
conv2d_4 (Conv2D) (None, 64, 64, 128) 2048 activation_1[0][0]
__________________________________________________________________________________________________
add_1 (Add) (None, 64, 64, 128) 0 conv2d_3[0][0]
conv2d_4[0][0]
__________________________________________________________________________________________________
batch_normalization_3 (BatchNor (None, 64, 64, 128) 512 add_1[0][0]
__________________________________________________________________________________________________
activation_3 (Activation) (None, 64, 64, 128) 0 batch_normalization_3[0][0]
__________________________________________________________________________________________________
conv2d_5 (Conv2D) (None, 64, 64, 128) 147456 activation_3[0][0]
__________________________________________________________________________________________________
batch_normalization_4 (BatchNor (None, 64, 64, 128) 512 conv2d_5[0][0]
__________________________________________________________________________________________________
activation_4 (Activation) (None, 64, 64, 128) 0 batch_normalization_4[0][0]
__________________________________________________________________________________________________
conv2d_6 (Conv2D) (None, 64, 64, 128) 147456 activation_4[0][0]
__________________________________________________________________________________________________
add_2 (Add) (None, 64, 64, 128) 0 conv2d_6[0][0]
add_1[0][0]
__________________________________________________________________________________________________
batch_normalization_5 (BatchNor (None, 64, 64, 128) 512 add_2[0][0]
__________________________________________________________________________________________________
activation_5 (Activation) (None, 64, 64, 128) 0 batch_normalization_5[0][0]
__________________________________________________________________________________________________
conv2d_7 (Conv2D) (None, 32, 32, 256) 294912 activation_5[0][0]
__________________________________________________________________________________________________
batch_normalization_6 (BatchNor (None, 32, 32, 256) 1024 conv2d_7[0][0]
__________________________________________________________________________________________________
activation_6 (Activation) (None, 32, 32, 256) 0 batch_normalization_6[0][0]
__________________________________________________________________________________________________
conv2d_8 (Conv2D) (None, 32, 32, 256) 589824 activation_6[0][0]
__________________________________________________________________________________________________
conv2d_9 (Conv2D) (None, 32, 32, 256) 32768 activation_5[0][0]
__________________________________________________________________________________________________
add_3 (Add) (None, 32, 32, 256) 0 conv2d_8[0][0]
conv2d_9[0][0]
__________________________________________________________________________________________________
batch_normalization_7 (BatchNor (None, 32, 32, 256) 1024 add_3[0][0]
__________________________________________________________________________________________________
activation_7 (Activation) (None, 32, 32, 256) 0 batch_normalization_7[0][0]
__________________________________________________________________________________________________
conv2d_10 (Conv2D) (None, 32, 32, 256) 589824 activation_7[0][0]
__________________________________________________________________________________________________
batch_normalization_8 (BatchNor (None, 32, 32, 256) 1024 conv2d_10[0][0]
__________________________________________________________________________________________________
activation_8 (Activation) (None, 32, 32, 256) 0 batch_normalization_8[0][0]
__________________________________________________________________________________________________
conv2d_11 (Conv2D) (None, 32, 32, 256) 589824 activation_8[0][0]
__________________________________________________________________________________________________
add_4 (Add) (None, 32, 32, 256) 0 conv2d_11[0][0]
add_3[0][0]
__________________________________________________________________________________________________
batch_normalization_9 (BatchNor (None, 32, 32, 256) 1024 add_4[0][0]
__________________________________________________________________________________________________
activation_9 (Activation) (None, 32, 32, 256) 0 batch_normalization_9[0][0]
__________________________________________________________________________________________________
conv2d_12 (Conv2D) (None, 16, 16, 512) 1179648 activation_9[0][0]
__________________________________________________________________________________________________
batch_normalization_10 (BatchNo (None, 16, 16, 512) 2048 conv2d_12[0][0]
__________________________________________________________________________________________________
activation_10 (Activation) (None, 16, 16, 512) 0 batch_normalization_10[0][0]
__________________________________________________________________________________________________
conv2d_13 (Conv2D) (None, 16, 16, 512) 2359296 activation_10[0][0]
__________________________________________________________________________________________________
conv2d_14 (Conv2D) (None, 16, 16, 512) 131072 activation_9[0][0]
__________________________________________________________________________________________________
add_5 (Add) (None, 16, 16, 512) 0 conv2d_13[0][0]
conv2d_14[0][0]
__________________________________________________________________________________________________
batch_normalization_11 (BatchNo (None, 16, 16, 512) 2048 add_5[0][0]
__________________________________________________________________________________________________
activation_11 (Activation) (None, 16, 16, 512) 0 batch_normalization_11[0][0]
__________________________________________________________________________________________________
conv2d_15 (Conv2D) (None, 16, 16, 512) 2359296 activation_11[0][0]
__________________________________________________________________________________________________
batch_normalization_12 (BatchNo (None, 16, 16, 512) 2048 conv2d_15[0][0]
__________________________________________________________________________________________________
activation_12 (Activation) (None, 16, 16, 512) 0 batch_normalization_12[0][0]
__________________________________________________________________________________________________
conv2d_16 (Conv2D) (None, 16, 16, 512) 2359296 activation_12[0][0]
__________________________________________________________________________________________________
add_6 (Add) (None, 16, 16, 512) 0 conv2d_16[0][0]
add_5[0][0]
__________________________________________________________________________________________________
batch_normalization_13 (BatchNo (None, 16, 16, 512) 2048 add_6[0][0]
__________________________________________________________________________________________________
activation_13 (Activation) (None, 16, 16, 512) 0 batch_normalization_13[0][0]
__________________________________________________________________________________________________
average_pooling2d_1 (AveragePoo (None, 16, 16, 512) 0 activation_13[0][0]
__________________________________________________________________________________________________
flatten_1 (Flatten) (None, 131072) 0 average_pooling2d_1[0][0]
__________________________________________________________________________________________________
dense_1 (Dense) (None, 2) 262144 flatten_1[0][0]
__________________________________________________________________________________________________
dense_2 (Dense) (None, 101) 13238272 flatten_1[0][0]
==================================================================================================
Total params: 24,463,856
Trainable params: 24,456,656
Non-trainable params: 7,200
__________________________________________________________________________________________________
我采用了输出的模型，并使用以下脚本进行了优化：
python -m tensorflow.python.tools.optimize_for_inference --input output_graph.pb --output g.pb --input_names=input_1 --output_names=dense_1/Softmax,dense_2/Softmax
在操作过程中，终端会给我很多这样的警告。
FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
from ._conv import register_converters as _register_converters
WARNING:tensorflow:Incorrect shape for mean, found (0,), expected (16,), for node batch_normalization_1/FusedBatchNorm
WARNING:tensorflow:Incorrect shape for mean, found (0,), expected (128,), for node batch_normalization_2/FusedBatchNorm
WARNING:tensorflow:Didn't find expected Conv2D input to 'batch_normalization_3/FusedBatchNorm'
WARNING:tensorflow:Incorrect shape for mean, found (0,), expected (128,), for node batch_normalization_4/FusedBatchNorm
WARNING:tensorflow:Didn't find expected Conv2D input to 'batch_normalization_5/FusedBatchNorm'
WARNING:tensorflow:Incorrect shape for mean, found (0,), expected (256,), for node batch_normalization_6/FusedBatchNorm
WARNING:tensorflow:Didn't find expected Conv2D input to 'batch_normalization_7/FusedBatchNorm'
WARNING:tensorflow:Incorrect shape for mean, found (0,), expected (256,), for node batch_normalization_8/FusedBatchNorm
WARNING:tensorflow:Didn't find expected Conv2D input to 'batch_normalization_9/FusedBatchNorm'
WARNING:tensorflow:Incorrect shape for mean, found (0,), expected (512,), for node batch_normalization_10/FusedBatchNorm
WARNING:tensorflow:Didn't find expected Conv2D input to 'batch_normalization_11/FusedBatchNorm'
WARNING:tensorflow:Incorrect shape for mean, found (0,), expected (512,), for node batch_normalization_12/FusedBatchNorm
WARNING:tensorflow:Didn't find expected Conv2D input to 'batch_normalization_13/FusedBatchNorm'
这些警告似乎很糟糕！
我已经在我的android应用上尝试了这两个文件。当非优化文件可执行时，优化文件根本无法工作，但会产生无意义的结果“ 例如GUESSING ”。
我知道这个问题有点长，但这是整个工作日的总结，我不想错过任何细节。
我不知道问题出在哪里。它是在输出节点名称中，冻结图，使用权重实例化模型还是在优化推理脚本中使用。
经过研究，终于解决了随机猜测的问题。
问题不在于将模型转换为.pb文件，正如我最初期望的那样，而是关于将图像正确地馈送到Android中的模型。
我再次尝试转换模型。以下几点将总结我的工作。
# save the model to .h5 file.
model.save('./saved_model/model.h5')其次，我将生成的.h5文件转换为.pb文件。我使用了此存储库中的代码。如果您无法通过超链接访问该链接，请访问：https : //github.com/amir-abdi/keras_to_tensorflow。该存储库的代码证明了其可靠性。它将模型转换为.pb文件，并立即对其进行优化以进行推理。太神奇了！
第三，.pb为了将它与我的应用程序一起配置，我将生成的文件带到android资产文件夹中。
第四，我将目标图像转换为像素值，并按位进行移位以提取颜色。获取有关此代码的帮助以完成此任务。请记住，getPixels方法保留颜色通道。因此，如果需要反转颜色通道，请遵循以下代码。我从这个答案中得到了帮助。
Bitmap bitmap = createScaledBitmap(faces[0], INPUT_SIZE , INPUT_SIZE , true);
// get pixel values
bitmap.getPixels(intValues, 0, bitmap.getWidth(), 0, 0, bitmap.getWidth(), bitmap.getHeight());
for (int i = 0; i < intValues.length; ++i) {
final int val = intValues[i];
// extract colors using bit-wise shifting.
floatValues[i * 3 + 0] = ((val >> 16) & 0xFF );
floatValues[i * 3 + 1] = ((val >> 8) & 0xFF );
floatValues[i * 3 + 2] = (val & 0xFF );
// reverse the color orderings.
floatValues[i*3 + 2] = Color.red(val);
floatValues[i*3 + 1] = Color.green(val);
floatValues[i*3] = Color.blue(val);
}
最后，我可以使用张量流推断方法将图像输入模型，进行推断并输出结果。