Byte swapping using Numpy in Python

Question

I'm trying to do byte-swapping using NumPy in Python.

I've created the function swap32() which byte swaps the bytes representation of the array that is composed of 32-bit ints. I provide a little-endian bytes object (brr) as the input in the code below and then the function swap it's bytes to convert it to big-endian. The output that my code produces is shown below (which is wrong btw).

>>> import numpy as np
>>> def swap32(x):
...   y = bytearray(x)
...   a = np.array(y, dtype=np.uint32)
...   return bytes(a.byteswap())

>>> arr = [1,2,3,4,5]
>>> brr = bytes(arr)
>>> brr
b'\x01\x02\x03\x04\x05'

>>> swap32(brr)
b'\x00\x00\x00\x01\x00\x00\x00\x02\x00\x00\x00\x03\x00\x00\x00\x04\x00\x00\x00\x05'

If I byteswap brr again then I get the following result which is incorrect because a swap32(swap32(brr)) should give back original brr.

>>> swap32(y)
b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x04\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x05'

I'm certainly going wrong somewhere which I'm unable to figure out. Isn't there any standard way (or easy way) to swap bytes of a bytes object in Python?

What do you mean "a 32-bit bytes object"? Your bytes object, bytes([1,2,3,4,5]) is not 32-bits in any sense that I can figure — juanpa.arrivillaga
– juanpa.arrivillaga, Commented Sep 11, 2020 at 11:41
@juanpa.arrivillaga I tried initializing array with int32 as well. But got the same result - \x00\x00\x00\x01. — Masquerade
– Masquerade, Commented Sep 11, 2020 at 11:41
@juanpa.arrivillaga You are right that my bytes object is not 32 bits. What I meant to say is that I have a bytes object and I want to do byteswapping assuming that the array is made of 32bit ints.I'm updating the post. My apologies. — Masquerade
– Masquerade, Commented Sep 11, 2020 at 11:43
@juanpa.arrivillaga Ok, I tried using np.uint8 and this removed the excess zeros but the result is still not byte swapped. ``` arr => [1, 2, 3, 4] swap32(bytes(arr)) => b'\x01\x02\x03\x04' — Masquerade
– Masquerade, Commented Sep 11, 2020 at 11:48

juanpa.arrivillaga · Accepted Answer · 2020-09-11 12:01:50Z

1

Ok, I think I understand you now. You should use np.frombuffer instead of np.array, because np.array treats it's input as an iterable, an iterating over bytes just gives you integers. You want to treat the bytes as a raw buffer.

Note the difference:

>>> np.array(bytearray([1,2,3,4]), dtype=np.uint32)
array([1, 2, 3, 4], dtype=uint32)
>>> np.frombuffer(bytearray([1,2,3,4]), dtype=np.uint32)
array([67305985], dtype=uint32)

So, you want something like:

def swap32(x):
    return np.frombuffer(x, dtype=np.uint32).byteswap().tobytes()

Note, this requires that the length of your bytes object be a multiple of the item-size of the array, i.e. in this case, a multiple of 4.

Anyway, in action:

>>> swap32(bytes([1,2,3,4]))
b'\x04\x03\x02\x01'

edited Sep 11, 2020 at 12:01

answered Sep 11, 2020 at 11:53

juanpa.arrivillaga

97.7k14 gold badges141 silver badges190 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Masquerade Over a year ago

Thanks for your answer. I'm able to reproduce the results for 32 bit. Just a question, if I use this with np.uint64 (assuming the length of my bytes object is in multiples of 8) it would work, right?

juanpa.arrivillaga Over a year ago

@PriyanshuKhandelwal yes. And signed/unsigned shouldn't actually matter in this case, actually, but the bitlength of the type does matter.

Collectives™ on Stack Overflow

Byte swapping using Numpy in Python

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related