将大小调整图层添加到keras顺序模型

Question

如何添加调整大小图层

model = Sequential()

运用

model.add(...)

要将图像从形状(160,320,3)调整为(224,224,3)？

Answer 1

我认为你应该考虑使用tensorflow的resize_images图层.

https://www.tensorflow.org/api_docs/python/tf/image/resize_images

似乎keras不包括这个,也许是因为theano中不存在该功能.我写了一个自定义keras层,它做了同样的事情.这是一个快速的黑客,所以它可能不适合你的情况.

import keras
import keras.backend as K
from keras.utils import conv_utils
from keras.engine import InputSpec
from keras.engine import Layer
from tensorflow import image as tfi

class ResizeImages(Layer):
    """Resize Images to a specified size

    # Arguments
        output_size: Size of output layer width and height
        data_format: A string,
            one of `channels_last` (default) or `channels_first`.
            The ordering of the dimensions in the inputs.
            `channels_last` corresponds to inputs with shape
            `(batch, height, width, channels)` while `channels_first`
            corresponds to inputs with shape
            `(batch, channels, height, width)`.
            It defaults to the `image_data_format` value found in your
            Keras config file at `~/.keras/keras.json`.
            If you never set it, then it will be "channels_last".

    # Input shape
        - If `data_format='channels_last'`:
            4D tensor with shape:
            `(batch_size, rows, cols, channels)`
        - If `data_format='channels_first'`:
            4D tensor with shape:
            `(batch_size, channels, rows, cols)`

    # Output shape
        - If `data_format='channels_last'`:
            4D tensor with shape:
            `(batch_size, pooled_rows, pooled_cols, channels)`
        - If `data_format='channels_first'`:
            4D tensor with shape:
            `(batch_size, channels, pooled_rows, pooled_cols)`
    """
    def __init__(self, output_dim=(1, 1), data_format=None, **kwargs):
        super(ResizeImages, self).__init__(**kwargs)
        data_format = conv_utils.normalize_data_format(data_format)
        self.output_dim = conv_utils.normalize_tuple(output_dim, 2, 'output_dim')
        self.data_format = conv_utils.normalize_data_format(data_format)
        self.input_spec = InputSpec(ndim=4)

    def build(self, input_shape):
        self.input_spec = [InputSpec(shape=input_shape)]

    def compute_output_shape(self, input_shape):
        if self.data_format == 'channels_first':
            return (input_shape[0], input_shape[1], self.output_dim[0], self.output_dim[1])
        elif self.data_format == 'channels_last':
            return (input_shape[0], self.output_dim[0], self.output_dim[1], input_shape[3])

    def _resize_fun(self, inputs, data_format):
        try:
            assert keras.backend.backend() == 'tensorflow'
            assert self.data_format == 'channels_last'
        except AssertionError:
            print "Only tensorflow backend is supported for the resize layer and accordingly 'channels_last' ordering"
        output = tfi.resize_images(inputs, self.output_dim)
        return output

    def call(self, inputs):
        output = self._resize_fun(inputs=inputs, data_format=self.data_format)
        return output

    def get_config(self):
        config = {'output_dim': self.output_dim,
                  'padding': self.padding,
                  'data_format': self.data_format}
        base_config = super(ResizeImages, self).get_config()
        return dict(list(base_config.items()) + list(config.items()))

Answer 2

可接受的答案使用Reshape层，该层的工作方式类似于NumPy的reshape，可用于将4x4矩阵整形为2x8矩阵，但这将导致图像丢失位置信息：

0 0 0 0
1 1 1 1    ->    0 0 0 0 1 1 1 1
2 2 2 2          2 2 2 2 3 3 3 3
3 3 3 3

相反，应该使用例如Tensorflowsimage_resize缩放图像数据/“调整大小” 。但是要注意正确的用法和错误！如相关问题所示，它可以与lambda层一起使用：

model.add( keras.layers.Lambda( 
    lambda image: tf.image.resize_images( 
        image, 
        (224, 224), 
        method = tf.image.ResizeMethod.BICUBIC,
        align_corners = True, # possibly important
        preserve_aspect_ratio = True
    )
))

对于您的情况，由于您具有160x320的图像，因此还必须决定是否保留宽高比。如果要使用预先训练的网络，则应该使用与训练网络相同的调整大小。

Answer 3

通常你会使用Reshape该层：

model.add(Reshape((224,224,3), input_shape=(160,320,3))

但由于您的目标维度不允许保存输入维度 ( 224*224 != 160*320) 中的所有数据，因此这不起作用。Reshape仅当元素数量不变时才可以使用。

如果您愿意丢失图像中的一些数据，您可以指定自己的有损重塑：

model.add(Reshape(-1,3), input_shape=(160,320,3))
model.add(Lambda(lambda x: x[:50176])) # throw away some, so that #data = 224^2
model.add(Reshape(224,224,3))

也就是说，这些转换通常是在将数据应用到模型之前完成的，因为如果在每个训练步骤中都这样做，这本质上是浪费计算时间。