Connect Encoder from AutoEncoder to LSTM

Question

I have an auto encoder defined like this

inputs = Input(batch_shape=(1,timesteps, input_dim))

encoded = LSTM(4,return_sequences = True)(inputs)
encoded = LSTM(3,return_sequences = True)(encoded)
encoded = LSTM(2)(encoded)
decoded = RepeatVector(timesteps)(encoded) 
decoded =  LSTM(3,return_sequences = True)(decoded)                                   
decoded =  LSTM(4,return_sequences = True)(decoded)
decoded =  LSTM(input_dim,return_sequences = True)(decoded)

sequence_autoencoder = Model(inputs, decoded)

encoder = Model(inputs,encoded)

I want the encoder to be connected to a LSTM layer like this

f_input = Input(batch_shape=(1, timesteps, input_dim))

encoder_input = encoder(inputs=f_input)

single_lstm_layer = LSTM(50, kernel_initializer=RandomUniform(minval=-0.05, maxval=0.05))(encoder_input)
drop_1 = Dropout(0.33)(single_lstm_layer)
output_layer = Dense(12, name="Output_Layer"
                         )(drop_1)

final_model = Model(inputs=[f_input], outputs=[output_layer])

But it gives me a dimension error.

Input 0 is incompatible with layer lstm_3: expected ndim=3, found ndim=2

How can I do this properly.?

The output of encoder (i.e.encoder_input) is of shape (1, 2). That's why it cannot be fed to the following LSTM layer. — today
– today, Commented Sep 6, 2018 at 10:15
I don't know what the problem you are working on is and why you are doing this, but if ask for resolving this issue in anyway, you must somehow have a Tensor of shape (1, ?, 2) to be able to feed it to LSTM layer. To achieve this, one way is to use RepeatVector layer on encoder_input. However, this may or may not be appropriate thing to do depending on the problem you are working on and the results you want to achieve. — today
– today, Commented Sep 6, 2018 at 10:21
@today I am trying to do something like this eng.uber.com/wp-content/uploads/2017/06/… — Sreeram TP
– Sreeram TP, Commented Sep 6, 2018 at 11:07

thushv89 · Accepted Answer · 2018-09-06 10:58:22Z

3

I think the main issue rises up from the fact that the very last encoded is not a repeat vector. To feed the encoder output to the LSTM, it needs to be sent through a RepeatVector layer. In other words, the last output of the encoder needs to have [batch_size, time_steps, dim] shape to be able to be fed into a LSTM. This is probably what you're looking for?

inputs = Input(batch_shape=(1,timesteps, input_dim))

encoded = LSTM(4,return_sequences = True)(inputs)
encoded = LSTM(3,return_sequences = True)(encoded)
encoded = LSTM(2)(encoded)
encoded_repeat = RepeatVector(timesteps)(encoded) 

decoded =  LSTM(3,return_sequences = True)(encoded_repeat)                                   
decoded =  LSTM(4,return_sequences = True)(decoded)
decoded =  LSTM(input_dim,return_sequences = True)(decoded)

sequence_autoencoder = Model(inputs, decoded)

encoder = Model(inputs,encoded_repeat)

f_input = Input(batch_shape=(1, timesteps, input_dim))

encoder_input = encoder(inputs=f_input)

single_lstm_layer = LSTM(50, kernel_initializer=RandomUniform(minval=-0.05, maxval=0.05))(encoder_input)
drop_1 = Dropout(0.33)(single_lstm_layer)
output_layer = Dense(12, name="Output_Layer"
                         )(drop_1)

final_model = Model(inputs=[f_input], outputs=[output_layer])

I have renamed your first decoded to encode_repeat

answered Sep 6, 2018 at 10:58

thushv89

11.4k1 gold badge32 silver badges42 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Sreeram TP Over a year ago

this is what I am looking for. Thanks

dennis-w · Accepted Answer · 2018-09-06 10:21:32Z

1

Your code already gives the answer. encoder has in its last layer lstm with two dimension (number_batch, number_features) instead of (number_batches, number_timesteps, number_features). This is because you did not set return_sequences = True(this is your intended behaviour).

But what you want to do is the same, as what you do with your decoder: You apply the RepeatVector layer to make the input shape 3 dimensional and therefore able to be feeded into a LSTM Layer.

answered Sep 6, 2018 at 10:21

dennis-w

2,1561 gold badge16 silver badges23 bronze badges

2 Comments

Sreeram TP Over a year ago

So, I want to change the encoder decoder archtecture.?

dennis-w Over a year ago

Yes if the encoded representation is meant to be fed inside another lstm layer then it would make sense to encode to a 3 dimesnional shape. So simply put return_sequences = True in your third lstm layer and delete the Repeat Vector Layer.

Collectives™ on Stack Overflow

Connect Encoder from AutoEncoder to LSTM

2 Answers 2

1 Comment

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related