AssertionError when add a TimeDistributed(Dense) layer in Keras sequence model

Question

I am building a RNN to predict a many to one question.

#Input_X:
[
[1,2,3,4,5,6,7,8,9,10],
[2,3,4,5,6,7,8,9,10,11]
]

#Input_Y:
[
11,
12
]
#Each number represent a category

X=np.reshape(Input_X,(len(Input_X), 10, 1))

y=np.utils.to_catgeorical(Input_Y)  #one hot encode,

My model setup:

#####This works 
model=Sequential()
model.add(LSTM(256, input_shape(X.shape[1], X.shape[2])))
model.add(Dense(y.shape[1], activation='softmax'))
model.compile(loss='categorical_crossentrophy', optimizer='adam', metrics=['accuracy'])
model.fit(X,y, ........)

I want to try the TimeDistributed(Dense) layer instead, for example: https://keras.io/layers/wrappers/. So I changed above to below:

model=Sequential()
model.add(LSTM(256, input_shape(X.shape[1], X.shape[2])))
model.add(TimeDistributed(Dense(y.shape[1], activation='softmax')))
model.compile(loss='categorical_crossentrophy', optimizer='adam', metrics=['accuracy'])
model.fit(X,y, ........)

I am getting a AssertionError. Which report the matrix size is not what expected. What steps I missed?

if I modify my my input_y to be length of 2 for each of the output. I still get the AssertionError — Hua Ye
– Hua Ye, Commented Aug 3, 2017 at 15:06

Dom · Accepted Answer · 2018-04-11 16:20:56Z

1

I think you need to add return_sequences=True to the LSTM cell

```
model=Sequential()
model.add(LSTM(256, return_sequences=True, input_shape(X.shape[1], X.shape[2])))
model.add(TimeDistributed(Dense(y.shape[1], activation='softmax')))
model.compile(loss='categorical_crossentrophy', optimizer='adam', metrics=['accuracy'])
model.fit(X,y, ........)

```

answered Apr 11, 2018 at 16:20

Dom

1712 silver badges5 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Avio Over a year ago

Nope, when using return_sequences=True I get

ValueError: Error when checking target: expected time_distributed_11 to have 3 dimensions, but got array with shape (2, 50)

DonAriston · Accepted Answer · 2019-07-13 04:41:29Z

0

return_sequences=True works for me.

In the OP's question, y.shape is (2,1) which has 2 samples and only 1 feature , so it's not suited for Many to Many model.

answered Jul 13, 2019 at 4:41

DonAriston

1111 silver badge5 bronze badges

Collectives™ on Stack Overflow

AssertionError when add a TimeDistributed(Dense) layer in Keras sequence model

2 Answers 2

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related