Unable to merge keras models after adding Lambda layer

Question

I have a list of 3 keras models that each have an output shape of (None, 2). I also have a common keras base model that produces their input. My goal is to combine the 4 models but to only take the first output from each of the models in the list (so the final output should have shape (None, 3). My problem occurs when I try to use a Lambda layer to extract the first output from each model.

If I omit the Lambda step and simply combine the models as follows, it creates a model that gives the correct output with shape (None, 6):

>>> sequentials = [Sequential([base_model, m]) for m in models]
>>> output = merge([s.output for s in sequentials], mode='concat')
>>> combined = Model(input=base_model.layers[0].input, output=output)
>>> combined.predict(X)
array([[  8.52127552e-01,   1.47872433e-01,   1.89960217e-13,
          1.00000000e+00,   7.56258190e-01,   2.43741751e-01]], dtype=float32)

The problem occurs when I first use a Lambda layer to extract the first value from each model:

>>> print([m.output_shape for m in models])
[(None, 2), (None, 2), (None, 2)]
>>> for m in models:
        m.add(Lambda(lambda x: x[0], output_shape=(1,)))
>>> print([m.output_shape for m in models])
[(None, 1), (None, 1), (None, 1)]
>>> sequentials = [Sequential([base_model, m]) for m in models]
>>> print([s.output_shape for s in sequentials])
[(None, 1), (None, 1), (None, 1)]
>>> output = merge([s.output for s in sequentials],
                   output_shape=(len(sequentials),), mode='concat')
>>> combined = Model(base_model.layers[0].input, output=output)
>>> print(combined.output_shape)
(None, 3)
>>> combined.predict(X)
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-3-4f4ed3bd605d> in <module>()
----> 1 ann.combined.predict(X)

./.virtualenvs/py3/lib/python3.4/site-packages/keras/engine/training.py in predict(self, x, batch_size, verbose)
   1217         f = self.predict_function
   1218         return self._predict_loop(f, ins,
-> 1219                                   batch_size=batch_size, verbose=verbose)
   1220
   1221     def train_on_batch(self, x, y,

./.virtualenvs/py3/lib/python3.4/site-packages/keras/engine/training.py in _predict_loop(self, f, ins, batch_size, verbose)
    904
    905             for i, batch_out in enumerate(batch_outs):
--> 906                 outs[i][batch_start:batch_end] = batch_out
    907             if verbose == 1:
    908                 progbar.update(batch_end)

ValueError: could not broadcast input array from shape (6) into shape (1)

What is the right way to merge these models while only taking the single output value from each one?

Note that I can successfully use a Lambda function if I apply it after merging the models as follows:

>>> sequentials = [Sequential([base_model, m]) for m in models]
>>> output = merge([s.output for s in sequentials], mode='concat')
>>> filtered = Lambda(lambda x: x[:,::2], lambda s: (s[-1] / 2,))(output)
>>> combined = Model(input=base_model.layers[0].input, output=filtered)
>>> combined.predict(X)
array([[  1.89960217e-13,   7.56258249e-01,   8.52127552e-01]], type=float32)

But I would like to know how to apply it before the merge.

do you compile the models at some point? not sure if it will change anything but it's just an idea — Nassim Ben
– Nassim Ben, Commented Mar 7, 2017 at 17:48
No, the models are all loaded from files (they were compiled & trained before saving). — bogatron
– bogatron, Commented Mar 7, 2017 at 18:29
@MarcinMożejko You were close. That changes the error to ValueError: could not broadcast input array from shape (3) into shape (1). It runs successfully if I use lambda x: x[:,:1]. Since you got me going in the right direction, feel free to post as an answer (preferably with explanation) and I'll accept. Thanks! — bogatron
– bogatron, Commented Mar 7, 2017 at 18:58
Ah right. I think that m.add(Lambda(lambda x: x[:,[0]], output_shape=(1,))) should also work correct. — Marcin Możejko
– Marcin Możejko, Commented Mar 7, 2017 at 18:59

Marcin Możejko · Accepted Answer · 2017-03-07 19:05:25Z

3

The problem lied with a little inconsistency in Lambda slicing. Although the shape of output doesn't take into account the batch dimension - one should remember that tensor provided to Lambda layers also have this additional dimension. This is why the following line causes an error:

m.add(Lambda(lambda x: x[0], output_shape=(1,)))

This should be changed to:

m.add(Lambda(lambda x: x[:,:1], output_shape=(1,)))

Beware of the following way of slicing:

m.add(Lambda(lambda x: x[:,0], output_shape=(1,)))

as it changes dimensionality of a tensor.

answered Mar 7, 2017 at 19:05

Marcin Możejko

40.6k11 gold badges113 silver badges123 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Unable to merge keras models after adding Lambda layer

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related