Why do values in my 3D numpy array change when I write it to file?

Question

strange problem in that I have a 3D array full of labels (lets say from 1-36) called labelled_stack. These are just areas where the value in the array is equal to the label given. A quick 2D example with 5 labels would be something like:

labelled_stack = (0 0 0 0 0 0 0 0 0 0)
                 (0 1 1 0 0 0 2 2 2 0)
                 (0 1 0 0 3 0 0 2 2 0)
                 (0 0 0 3 3 0 0 0 0 0)
                 (0 4 0 0 3 0 0 0 5 0)
                 (0 4 0 0 0 0 5 5 5 0)
                 (0 0 0 0 0 0 0 0 0 0)

But imagine its a numpy array...

I have tried using cv2.imwrite and scipy.misc.imsave to save the stack, but when i do this and then open them, their values have changed so that the values = 5, now equal 255, and the values equal to 1, equal 51 etc. This is not acceptable. I need the values to stay as they are.. increasing values of integers in steps of 1. i know cv2 and scipy.misc both write as 8bit images but that shouldnt mean I get this error.

I have even re-read the images back into python to check this is going on and it is.

My labelled_stack is of type np.uint32

Edit to include save commands:

for j in np.arange(0,l,1):
     misc.imsave(save_path+Folder+'/Labelled/slice_{:04d}of{:}.tif'.format(j+1,l),labelled_stack[:,:,j])

or...

cv2.imwrite(save_path+Folder+'/Labelled/slice_{:04d}of{:}.tif'.format(j+1,l),labelled_stack[:,:,j])

Why not use a numpy method to save the numpy array? Something like savetxt can handle just outputting a numpy array to a file. — zephyr
– zephyr, Commented May 27, 2016 at 13:06
Can you show how you call imwrite ? And posting the image you output could be useful too. — Sunreef
– Sunreef, Commented May 27, 2016 at 13:09
Sorry I should have been clear, I need the image in a tif format for the next stage of processing. Sunreef, what way do you suggest posting a 3D array? — Scott Alistair
– Scott Alistair, Commented May 27, 2016 at 13:19

Community · Accepted Answer · 2017-05-23 11:44:09Z

3

The function imwrite saves the data in an 8 bit format. When the function converts your 32 bit data to 8 bit, it scales the data to use the full bit depth of 8 bits, so for example the maximum of your data is scaled up to 255. See scipy imsave saves wrong values for another example.

To avoid this, convert your data to numpy.uint8 before saving it. In the following example, a has data type numpy.uint32.

In [98]: from scipy.misc import imsave, imread

In [99]: a
Out[99]: 
array([[0, 3, 2, 2, 4, 0, 3, 0],
       [2, 0, 0, 3, 3, 1, 3, 0],
       [2, 4, 4, 0, 2, 3, 1, 3],
       [0, 1, 3, 1, 0, 0, 0, 4],
       [2, 1, 1, 2, 1, 1, 3, 1],
       [0, 4, 0, 1, 0, 0, 2, 3],
       [3, 1, 3, 3, 3, 2, 3, 4],
       [0, 4, 1, 4, 2, 2, 0, 2]], dtype=uint32)

The astype method is used to convert the array to numpy.uint8 before giving it to the imsave function.

In [100]: imsave("a.tif", a.astype(np.uint8))  # Convert to 8 bit before saving

Read it back with scipy.misc.imread:

In [101]: b = imread("a.tif")

In [102]: b
Out[102]: 
array([[0, 3, 2, 2, 4, 0, 3, 0],
       [2, 0, 0, 3, 3, 1, 3, 0],
       [2, 4, 4, 0, 2, 3, 1, 3],
       [0, 1, 3, 1, 0, 0, 0, 4],
       [2, 1, 1, 2, 1, 1, 3, 1],
       [0, 4, 0, 1, 0, 0, 2, 3],
       [3, 1, 3, 3, 3, 2, 3, 4],
       [0, 4, 1, 4, 2, 2, 0, 2]], dtype=uint8)

Note that when the data is read with scipy.misc.imread, the result has data type numpy.uint8.

Another recommended option, especially if you are trying to preserve data that requires 16 bits or 32 bits, is to use tifffile.

In this example, a has data type numpy.uint32, with some values that are larger than can fit in an unsigned 8 bit number.

In [152]: from tifffile import imsave, imread

In [153]: a
Out[153]: 
array([[  0,   5,  10,  15,  20,  25,  30,  35,  40,  45],
       [ 50,  55,  60,  65,  70,  75,  80,  85,  90,  95],
       [100, 105, 110, 115, 120, 125, 130, 135, 140, 145],
       [150, 155, 160, 165, 170, 175, 180, 185, 190, 195],
       [200, 205, 210, 215, 220, 225, 230, 235, 240, 245],
       [250, 255, 260, 265, 270, 275, 280, 285, 290, 295],
       [300, 305, 310, 315, 320, 325, 330, 335, 340, 345],
       [350, 355, 360, 365, 370, 375, 380, 385, 390, 395],
       [400, 405, 410, 415, 420, 425, 430, 435, 440, 445],
       [450, 455, 460, 465, 470, 475, 480, 485, 490, 495]], dtype=uint32)

In [154]: imsave("a.tif", a)

In [155]: b = imread("a.tif")

In [156]: b
Out[156]: 
array([[  0,   5,  10,  15,  20,  25,  30,  35,  40,  45],
       [ 50,  55,  60,  65,  70,  75,  80,  85,  90,  95],
       [100, 105, 110, 115, 120, 125, 130, 135, 140, 145],
       [150, 155, 160, 165, 170, 175, 180, 185, 190, 195],
       [200, 205, 210, 215, 220, 225, 230, 235, 240, 245],
       [250, 255, 260, 265, 270, 275, 280, 285, 290, 295],
       [300, 305, 310, 315, 320, 325, 330, 335, 340, 345],
       [350, 355, 360, 365, 370, 375, 380, 385, 390, 395],
       [400, 405, 410, 415, 420, 425, 430, 435, 440, 445],
       [450, 455, 460, 465, 470, 475, 480, 485, 490, 495]], dtype=uint32)

edited May 23, 2017 at 11:44

CommunityBot

11 silver badge

answered May 27, 2016 at 13:21

Warren Weckesser

116k20 gold badges207 silver badges224 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Scott Alistair Over a year ago

Do you have any suggestions for when the no. of labels is unknown on each occasion, and could easily be less than 255 or above 255??

Warren Weckesser Over a year ago

@ScottAlistair For tiff files, a nice option is the tifffile package. It supports 16 and 32 bit files with no special coding required. I have updated my answer with an example.

Scott Alistair Over a year ago

This is something I also found today, but was worried this would act in the same way the my original code would and scale up the data, but i'm glad to see it doesnt! excellent, thank you

Berriel · Accepted Answer · 2016-05-27 13:10:00Z

3

As you can read in the documentation:

The function imwrite saves the image to the specified file. The image format is chosen based on the filename extension (see imread() for the list of extensions). Only 8-bit (or 16-bit unsigned (CV_16U) in case of PNG, JPEG 2000, and TIFF) single-channel or 3-channel (with ‘BGR’ channel order) images can be saved using this function.

You said that "labelled_stack is of type np.uint32" (32-bit unsigned int). That is the problem. You need to handle the type convertion yourself.

answered May 27, 2016 at 13:10

Berriel

13.8k4 gold badges51 silver badges73 bronze badges

2 Comments

Scott Alistair Over a year ago

change it to what, int?

Berriel Over a year ago

@ScottAlistair uint8 or uint16 (read the limitations described in the documentation)

Collectives™ on Stack Overflow

Why do values in my 3D numpy array change when I write it to file?

2 Answers 2

3 Comments

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

3 Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related