Android convert byte array from Camera API to color Mat object openCV

Question

I'm trying to take data captured off the camera API and do color postprocessing. My issue is that while I can convert a byte[] object to a Mat object, I cannot get the color channels without receiving an error. I started with the following post: How to get the Mat object from the Byte[] in openCV android? but that implementation does not work.

Here is the relevant code:

@Override
public void onPictureTaken(byte[] data, Camera camera) {
    // The camera preview was automatically stopped. Start it again.
    mCamera.startPreview();
    mCamera.setPreviewCallback(this);

    // Write the image in a file (in jpeg format)
    File pictureFile_interp = getOutputMediaFile(MEDIA_TYPE_IMAGE);

    try {

        //Now try to convert and save.
        Parameters params = camera.getParameters();
        Size sz = params.getPictureSize();

        //ERROR IN THE LINE BELOW!!!!
        Mat raw = new Mat(sz.height,sz.width,CvType.CV_8UC4);
        raw.put(0, 0, data);
        Mat targ = Highgui.imdecode(raw, 0);

        orig = new Mat();
        Imgproc.cvtColor(targ, orig, Imgproc.COLOR_GRAY2RGB);
        //Imgproc.cvtColor(targ, fixed, Imgproc.COLOR_BGR2RGB);

        //now we have the target we want...let's interpolate.
        interp = interpExperiment(orig,interpBy);
        Highgui.imwrite(pictureFile_interp.toString(), interp);

    } finally{

    }

}

When I try that code, I get an exception:

Provided data element number should be multiple of the Mat channels count (3)

Instead I replace the problematic line with

Mat raw = new Mat(sz.height,sz.width,CvType.CV_8U);
raw.put(0, 0, data);
Mat targ = Highgui.imdecode(raw, 0);

I can get a grayscale image. What am I doing wrong with

Mat raw = new Mat(sz.height,sz.width,CvType.CV_8UC3);

? I've looked at quite a few stackoverflow posts addressing this and none work for the color matrix. Any help is much appreciated.

The exception happens at the raw.put(0,0,data); line

user3502402
– user3502402

2014-04-21 17:37:58 +00:00
Commented Apr 21, 2014 at 17:37 — user3502402
– user3502402, Commented Apr 21, 2014 at 17:37

user3502402 · Accepted Answer · 2014-04-23 18:55:19Z

3

I found a workaround to this: using Bitmapfactory, you can convert the byte[] array to a bitmap, which you can in turn correctly convert into a Mat.

Bitmap bmp = BitmapFactory.decodeByteArray(data , 0, data.length);
Mat orig = new Mat(bmp.getHeight(),bmp.getWidth(),CvType.CV_8UC3);
Bitmap myBitmap32 = bmp.copy(Bitmap.Config.ARGB_8888, true);
Utils.bitmapToMat(myBitmap32, orig);

Imgproc.cvtColor(orig, orig, Imgproc.COLOR_BGR2RGB,4);

Seems there should be a better way but this works.

answered Apr 23, 2014 at 18:55

user3502402

1133 silver badges9 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Sipty Over a year ago

Have you found a more direct way of doing this?

Idol · Accepted Answer · 2014-11-18 22:00:10Z

3

public Mat Byte_to_Mat(byte[] data) {

    Mat jpegData = new Mat(1, data.length, CvType.CV_8UC1);
    jpegData.put(0, 0, data);


    Mat bgrMat = new Mat();
    bgrMat = Highgui.imdecode(jpegData, Highgui.IMREAD_COLOR);

answered Nov 18, 2014 at 22:00

Idol

836 bronze badges

2 Comments

Tal Over a year ago

This code is work. i didn't know how to use Highgui.imdecode But now I am know :) Thanks @Idol

aProperFox Over a year ago

I don't know how this isn't getting more votes. This worked for me, although sadly it was slower than decoding to a bitmap :(

timegalore · Accepted Answer · 2014-04-21 20:35:13Z

1

When you use:

raw.put(0, 0, data);

you would normally be setting all the channel values for pixel at point (0,0); so if you have a 3 channel image, then you would provide a byte array of three values like:

byte data[] = {1,2,3}

You are allowed to set more than one pixel at a time but the key point is that your data array needs to be multiple of the channels as otherwise opencv won't know what to see all the pixel values to. So, for example, again for a 3 channel Mat, you are allowed:

byte data[] = {1,2,3,4,5,6};

this will set pixel value (0,0) to (1,2,3) and (0,1) to (4,5,6)

but not:

byte data[] = {1,2,3,4,5,6,7};

as this as 7 elements,which is not a multiple of 3.

You get the error you are seeing as your data array length is not a multiple of the number of channels.

You should check what format your data array is actually in. From my experience on my devices it tends to be YUV which has different dimensions to RGB so there will be some conversion required.

answered Apr 21, 2014 at 20:35

timegalore

7317 silver badges9 bronze badges

1 Comment

user3502402 Over a year ago

Ahh, this makes sense - I just wasn't sure what the proper Mat construction was. It would make sens for the data format to be YUV though.

Froyo · Accepted Answer · 2014-04-22 08:42:40Z

1

Most likely the data you will be getting would be in YUV format. It is generally 1 channel matrix with height as 1.5 times the height of the image if it were RGB or grayscale and same width.

Mat raw = new Mat(sz.height*1.5,sz.width,CvType.CV_8UC1); // YUV data
raw.put(0, 0, data);
orig = new Mat(sz.height, sz.height, CvType.CV_8UC4); // RGBA image
Imgproc.cvtColor(raw, orig, Imgproc.COLOR_YUV2RGBA_NV21); // This might vary depending on the camera image type

answered Apr 22, 2014 at 8:42

Froyo

18.5k8 gold badges47 silver badges73 bronze badges

3 Comments

user3502402 Over a year ago

Thanks! The Mat class doesn't appear to have a constructor that takes (double,int,int), so I converted the 1.5*sz.height to integer and tried. The result was a completely green image.

Froyo Over a year ago

After converting to RGBA? Or you saved raw Mat?

user3502402 Over a year ago

I tried the exact code posted, resulted in the green images. I think something got messed up in the type conversion (i.e. the 4th line you posted). I am using a Nexus 5. I thought I saw somewhere that this phone saved as YUV... I converted to a bitmap and it worked well as per my most recent post.

Gaurang Gupta · Accepted Answer · 2016-05-03 10:16:22Z

0

The changes made by @user3502402 are absolutely correct. You just needed to change the flag to 1 from 0 in Highgui.imdecode(raw, 1) and the code gives perfect coloured photographs

answered May 3, 2016 at 10:16

Gaurang Gupta

1

Comments

Rohit Patil · Accepted Answer · 2022-01-19 10:00:25Z

-1

The correct answer is ===>

int width = camera.getParameters().getPreviewSize().width; // get width of camera preview size

int height = camera.getParameters().getPreviewSize().height; // get height of camera preview size

Mat mat = new Mat(height + height/2,width, CvType.CV_8UC1); //Create mat object for NV21 byte[]

mat.put(0, 0, bytes); //Put byte[] into mat to generate raw mat

Mat mRgba = new Mat(); //Create new mat object to convert NV21 mat to RGBA

Imgproc.cvtColor( mat, mRgba, Imgproc.COLOR_YUV2RGB_NV21,4); //Convert mat ot new rgba mat

Bitmap bitmap = Bitmap.createBitmap(width,height, Bitmap.Config.ARGB_8888); //Create same bitmap which camera preview size has. o.w. it will throw error.

Utils.matToBitmap(mRgba,bitmap); // Finally convert mat object to bitmap.

Holaaaaa....

Tested on personal project and it is working 100%

answered Jan 19, 2022 at 10:00

Rohit Patil

1,62512 silver badges10 bronze badges

1 Comment

Rohit Patil Over a year ago

Let me know what issue you are facing with above code. And then vote ;)

Collectives™ on Stack Overflow

Android convert byte array from Camera API to color Mat object openCV

6 Answers 6

1 Comment

2 Comments

1 Comment

3 Comments

Comments

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

6 Answers 6

1 Comment

2 Comments

1 Comment

3 Comments

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related