如何计算张量流模型中可训练参数的总数？

Question

是否有函数调用或其他方法来计算张量流模型中的参数总数？

通过参数我的意思是:可训练变量的N dim向量具有N个参数,NxM矩阵具有N*M参数等.因此,基本上我想在张量流会话中求和所有可训练变量的形状维度的乘积.

Answer 1

循环遍历每个变量的形状tf.trainable_variables().

total_parameters = 0
for variable in tf.trainable_variables():
    # shape is an array of tf.Dimension
    shape = variable.get_shape()
    print(shape)
    print(len(shape))
    variable_parameters = 1
    for dim in shape:
        print(dim)
        variable_parameters *= dim.value
    print(variable_parameters)
    total_parameters += variable_parameters
print(total_parameters)

更新:由于这个答案,我写了一篇文章来阐明Tensorflow中的动态/静态形状:https://pgaleone.eu/tensorflow/2018/07/28/understanding-tensorflow-tensors-shape-static-dynamic/

Answer 2

我有一个更短的版本,使用numpy的一线解决方案:

np.sum([np.prod(v.get_shape().as_list()) for v in tf.trainable_variables()])

Answer 3

不确定给出的答案是否实际运行(我发现你需要将dim对象转换为int以使其工作).这是一个有效的,您可以复制粘贴函数并调用它们(也添加了一些注释):

def count_number_trainable_params():
    '''
    Counts the number of trainable variables.
    '''
    tot_nb_params = 0
    for trainable_variable in tf.trainable_variables():
        shape = trainable_variable.get_shape() # e.g [D,F] or [W,H,C]
        current_nb_params = get_nb_params_shape(shape)
        tot_nb_params = tot_nb_params + current_nb_params
    return tot_nb_params

def get_nb_params_shape(shape):
    '''
    Computes the total number of params for a given shap.
    Works for any number of shapes etc [D,F] or [W,H,C] computes D*F and W*H*C.
    '''
    nb_params = 1
    for dim in shape:
        nb_params = nb_params*int(dim)
    return nb_params 

Answer 4

如果您正在研究自己计算参数的数量,那么现有的两个答案都很好.如果您的问题更像是"有一种简单的方式来描述我的TensorFlow模型吗？",我强烈建议您查看tfprof.它描述您的模型,包括计算参数的数量.

Answer 5

适用于我的TF v2.9。归功于这个答案

import numpy as np

trainable_params = np.sum([np.prod(v.get_shape()) for v in model.trainable_weights])
non_trainable_params = np.sum([np.prod(v.get_shape()) for v in model.non_trainable_weights])
total_params = trainable_params + non_trainable_params
    
print(trainable_params)
print(non_trainable_params)
print(total_params)