0

I have a function

def preprocess(image):
  image = cv2.cvtColor(image, cv2.COLOR_RGB2YUV)
  image = cv2.GaussianBlur(image,  (3, 3), 0)
  image = cv2.resize(image, (200, 66))
  return image

I want to use this function in a Lambda layer of a NN like this

  model=Sequential()

  model.add(Lambda(preprocess,input_shape=(160,320,3)))
  #model.add(Lambda(lambda x: (x / 255.0) - 0.5, input_shape=(160,320,3)))
  model.add(Lambda(lambda x: (x / 255.0) - 0.5))
  model.add(Cropping2D(cropping=((60,25), (0,0))))

However I got the error

model = nvidia_model()
print(model.summary())

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-11-9f6554917809> in <module>()
----> 1 model = nvidia_model()
      2 print(model.summary())

6 frames
<ipython-input-9-5e5ec9bc70c2> in preprocess(image)
      1 def preprocess(image):
      2   
----> 3   image = cv2.cvtColor(image, cv2.COLOR_RGB2YUV)
      4   image = cv2.GaussianBlur(image,  (3, 3), 0)
      5   image = cv2.resize(image, (200, 66))

TypeError: Expected Ptr<cv::UMat> for argument '%s'

How can I use this function as a layer of the NN?? (note that just using the lambda x:(x/255.0) works well.

Note: I don't want to do this externally. (Because I already did, it works fine, but now I am trying to do it internally)

4
  • 2
    I don't think you can, because there are no gradients associated with opencv operations. Also, processing should usually happen before model training. Commented Oct 23, 2020 at 2:29
  • All right. However normalization and cropping do work... Commented Oct 23, 2020 at 2:42
  • But you're not using opencv for those, right? You are using Keras layers, which have gradients implemented. You implemented the normalization with operations that use Keras/Tensorflow tensors, and the Cropping2D layer [is implemented in TensorFlow with tensors as well]. Those support automatic differentiation, which is necessary for model training. Numpy/opencv do not support this. Commented Oct 23, 2020 at 2:48
  • Alright. I found this stackoverflow.com/a/49830980/4451521 but could not figure it out how this works Commented Oct 23, 2020 at 3:45

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.