Using dynamic input shape in keras

Question

I am working with some data that contains some features in some continues days and the shape of the array of each of these data is as below:
(number of days, 1, number of features)

Number of features in each of these data is different.
I want to feed each of these data, separately to my lstm model. So I want to implement my model in a way that its input shape is dynamic.

I have used this code:

model = Sequential()
model.add(LSTM(4, return_sequences=True, input_shape=(1, None)))
model.add(LSTM(256, dropout=0.2,recurrent_dropout=0.2, return_sequences=True))
model.add(LSTM(256, dropout=0.2,recurrent_dropout=0.2, return_sequences=True))
model.add(LSTM(128, dropout=0.2,recurrent_dropout=0.2, return_sequences=True))
model.add(LSTM(128))
model.add(Dense(1, activation='sigmoid'))

model.compile (
    loss='mean_squared_error',
    optimizer=keras.optimizers.Adam(0.001)
)

That None in the first layer is for number of features. But I get this error for this layer when I start to fit the model on (X_train and y_train):

TypeError: unsupported operand type(s) for +: 'NoneType' and 'int'

I am using tensorflow version '2.3.0-tf'

Can you help me to fix this error?

user11530462 · Accepted Answer · 2020-06-08 11:16:10Z

2

The Function, pad_sequences will be useful in this case.

For example, Input Sequences has different number of Features as shown below:

sequences = [
    [1, 2, 3, 4],
       [1, 2, 3],
             [1]
    ]

We can make all the Features of equal length using pad_sequences, as shown below:

padded = pad_sequences(sequences)

That will make Input Sequence:

[[1 2 3 4]
 [0 1 2 3]
 [0 0 0 1]]

That is, it will pad the other Features with Zeros and will make the Number of Features of all the Samples as 4 (Maximum among them).

The Padding with Zeros can be done at the start or at the end by adjusting the argument, 'padding'. For more details about this Function, please refer this Tensorflow Documentation.

Complete working code with variable number of Features is shown below:

from tensorflow.keras.preprocessing.sequence import pad_sequences
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, LSTM
import tensorflow as tf
import numpy as np


# define sequences
sequences = [
    [1, 2, 3, 4],
       [1, 2, 3],
             [1]
    ]

# pad sequence
padded = pad_sequences(sequences)
print(padded)
X = np.expand_dims(padded, axis = 0)
print(X.shape) # (1, 3, 4)

y = np.array([1,0,1])
y = y.reshape(1,-1)
print(y.shape) # (1, 3)

model = Sequential()
model.add(LSTM(4, return_sequences=False, input_shape=(None, X.shape[2])))
model.add(Dense(1, activation='sigmoid'))

model.compile (
    loss='mean_squared_error',
    optimizer=tf.keras.optimizers.Adam(0.001))

model.fit(x = X, y = y)

Output of above code is :

[[1 2 3 4]
 [0 1 2 3]
 [0 0 0 1]]

(1, 3, 4)

(1, 3)

1/1 [==============================] - 0s 1ms/step - loss: 0.2601

Hope this helps. Happy Learning!

answered Jun 8, 2020 at 11:16

user11530462

Sign up to request clarification or add additional context in comments.

2 Comments

Mahdi Over a year ago

Thanks for your help, but I had a question. When we pad the array with zeros, won't this zeros affect the result of the model?

user11530462 Over a year ago

In such case, you can use Masking. Refer the link, tensorflow.org/guide/keras/masking_and_padding#masking for more details.

Collectives™ on Stack Overflow

Using dynamic input shape in keras

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related