Error: ValueError: The last dimension of the inputs to `Dense` should be defined. Found `None`

Question

I'm trying to build a lstm model for text classification and I'm receiving an error. This is my entire code that I've tried.

Please let me know what's the reason behind the error and how to fix it.

input1.shape # text data integer coded
(37788, 130)
input2.shape # multiple category columns(one hot encoded) concatenated together
(37788, 104)

train_data = [input1, input2] # this is the train data.

i1 = Input(shape=(130,), name='input')
embeddings = Embedding(input_dim=20000, output_dim=100, input_length=130)(i1)
lstm = LSTM(100)(embeddings)
flatten = Flatten()(lstm)

i2  = Input(shape=(None, 104))
c1 = Conv1D(64, 2, padding='same', activation='relu', kernel_initializer='he_uniform')(i2)
c2 = Conv1D(32, kernel_size=3, activation='relu', kernel_initializer='he_uniform')(c1)
flatten1 = Flatten()(c2)

concat = concatenate([flatten, flatten1])
dense1 = Dense(32, 'relu', kernel_initializer='he_uniform')(concat)

I tried to print shape of conv1d layers and I was getting None for flatten layer. I think it might be the reason for the error.

Tensor("conv1d_81/Identity:0", shape=(None, None, 64), dtype=float32)
Tensor("conv1d_82/Identity:0", shape=(None, None, 32), dtype=float32)
Tensor("flatten_106/Identity:0", shape=(None, None), dtype=float32)

This is the error I'm getting. How to fix it?

---------------------------------------------------------------------------
    ValueError                                Traceback (most recent call last)
    <ipython-input-531-31a53fbf3d37> in <module>
         14 concat = concatenate([flatten, flatten1])
    ---> 15 dense1 = Dense(32, 'relu', kernel_initializer='he_uniform')(concat)
         16 drop = Dropout(0.5)(dense1)

    ~\Anaconda3\lib\site-packages\tensorflow\python\keras\engine\base_layer.py in __call__(self, inputs, *args, **kwargs)
        614           # Build layer if applicable (if the `build` method has been
        615           # overridden).
    --> 616           self._maybe_build(inputs)
        617 
        618           # Wrapping `call` function in autograph to allow for dynamic control

    ~\Anaconda3\lib\site-packages\tensorflow\python\keras\engine\base_layer.py in _maybe_build(self, inputs)
       1964         # operations.
       1965         with tf_utils.maybe_init_scope(self):
    -> 1966           self.build(input_shapes)
       1967       # We must set self.built since user defined build functions are not
       1968       # constrained to set self.built.

    ~\Anaconda3\lib\site-packages\tensorflow\python\keras\layers\core.py in build(self, input_shape)
       1003     input_shape = tensor_shape.TensorShape(input_shape)
       1004     if tensor_shape.dimension_value(input_shape[-1]) is None:
    -> 1005       raise ValueError('The last dimension of the inputs to `Dense` '
       1006                        'should be defined. Found `None`.')
       1007     last_dim = tensor_shape.dimension_value(input_shape[-1])

    ValueError: The last dimension of the inputs to `Dense` should be defined. Found `None`.

Answer 1

You have None in the length of the sequence in the second model.

i2  = Input(shape=(None, 104))

You can't flatten a variable length and have a known size.
You need a known size for Dense.

Either you use a fixed length instead of None, or you use a GlobalMaxPooling1D or a GlobalAveragePooling1D instead of Flatten.