Create Numpy array from list of arbitrary sized bites

Question

How can I create a numpy array from a python list of bytes objects of an arbitrary (but known) size?

Example:

size = 10
byte_list = [np.random.default_rng().bytes(size) for i in range(100)]

numpy_array = # make array from byte_list

# do something with the array
test_vals = np.random.default_rng().choice(numpy_array, size=10)

I tried to do something like this, but got an error that it didn't understand 'B10' as a data type.

numpy_array = np.fromiter(byte_list, dtype=np.dtype(f'B{size}'), count=100)

What would you like the output to look like? What kind of shape? — Chrysophylaxs
– Chrysophylaxs, Commented Jan 29, 2023 at 18:36

Corralien · Accepted Answer · 2023-01-29 19:25:07Z

1

I think you should use S dtype and not B

numpy_array = np.fromiter(byte_list, dtype=np.dtype(f'S{size}'), count=100)
#                                              HERE --^

# Unsigned byte (only one)
>>> np.dtype('B')
dtype('uint8')

# Byte string
>>> np.dtype('S10')
dtype('S10')

answered Jan 29, 2023 at 19:25

Corralien

121k8 gold badges44 silver badges69 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

aabceh Over a year ago

This almost works, but since the S dtype is for a zero terminated string, if the value ends with \x00 it will get stripped off. I need the values in the array to be exactly the same as in the initial list.

Collectives™ on Stack Overflow

Create Numpy array from list of arbitrary sized bites

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related