Make different loss functions for different features in Keras

Question

I have a model that has 5 input features, the input looks like this:

- 500.0  --  499.752  --  499.813  --  0.061  --  -0.016 -
- 499.91  --  499.763  --  499.828  --  0.064  --  0.005 -
- 499.83  --  499.768  --  499.828  --  0.06  --  0.021 -
- 499.91  --  499.779  --  499.841  --  0.062  --  0.035 -
- 499.95  --  499.792  --  499.858  --  0.066  --  0.045 -
- 500.0  --  499.807  --  499.879  --  0.073  --  0.054 -
- 500.0  --  499.821  --  499.898  --  0.077  --  0.06 -
- 500.0  --  499.834  --  499.914  --  0.079  --  0.065 -
- 500.0  --  499.847  --  499.927  --  0.08  --  0.069 -
- 499.96  --  499.855  --  499.932  --  0.077  --  0.071 -
- 500.0  --  499.866  --  499.943  --  0.077  --  0.072 -
- 500.0  --  499.876  --  499.951  --  0.076  --  0.074 -
- 500.0  --  499.885  --  499.959  --  0.074  --  0.075 -
- 500.0  --  499.894  --  499.965  --  0.072  --  0.076 -
- 499.99  --  499.901  --  499.969  --  0.068  --  0.075 -

As you see, Features 1,2 and 3 move around 500, and features 4 and 5 move around zero. I have a single MSE loss function, which is making the model predict a similar number across all features.

An example:

Should have predicted:
- 500.0  --  499.866  --  499.943  --  0.077  --  0.072 -

But predicted:
- 34.875  --  22.658  --  42.792  --  -4.824  --  -24.389 -

You can see how it tries to produce numbers that all similar. You can also see this in training when the model will get more accurate but have higher losses.

What I am looking for is a way to make a separate MSE loss for features 1, 2, and 3, and then another for 4 and 5, so the model can actually output in a feature specific range.

I also thought about scaling features 4 and 5 up to the same range as 1, 2, and 3, but I don't know if that will work as well.

If there are any other possible solutions please share. Thanks!

score 1 · Accepted Answer · 2020-08-28 19:51:32Z

1

Regardless of the problem above, are you sure your model implementation is correct?
However, for your case I think it is good to use Min-Max scaler which scale the values of all features between 0 and 1 (by default) and then apply model and compare results.

edited Aug 28, 2020 at 19:51

answered Aug 28, 2020 at 19:39

user13959036

Sign up to request clarification or add additional context in comments.

Comments

Catalina Chircu · Accepted Answer · 2020-08-28 20:14:58Z

0

I shall add to @Meysam's answer that MinMaxScaller in scikit learn have an inverse_transform function which allows to reproduce the initial values.

For example, you may have :

from sklearn.preprocessing import MinMaxScaler

scaler = MinMaxScaler()
scaler.fit(Y)

(... training here ...)

Y_pred = model.predict(X_test)

Your predicted model will be scaled obviously, so if you want to get the values at the initial scaling, you should use the same scaler:

Y_pred_original_scaling = scaler.inverse_transform(Y_pred)

answered Aug 28, 2020 at 20:14

Catalina Chircu

1,5742 gold badges10 silver badges20 bronze badges

Collectives™ on Stack Overflow

Make different loss functions for different features in Keras

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related