Send numpy array as bytes from python to JS through Flask

Question

A numpy array should be sent from python to JS through flask. I don't want to jsonify it and send as it would increase the size of the response and eventually the response time. So I thought I could convert numpy into bytes using tobytes(), send the bytes to JS through flask and convert the bytes back to float in JS and found Converting a string of packed bytes into an array of floats in Javascript answer helpful.

The list of operations are

byte_arr = np.array([5.6], dtype=np.float32).tobytes()  # Used one value just to make it simple
return byte_arr  # Bytes can be directly sent from flask as response

In JS,

str = response.text
bytes = Uint8Array.from(str, c => c.charCodeAt(0))
floats = new Float32Array(bytes.buffer)

But when converting to float in JS, I am not getting the correct values. When debugging I found that in JS some values in Uint8 array are not matching with the byte values that are sent from python.

JS: console.log(bytes)  // [51, 51, 255, 64]
Python: for val in byte_arr: print(val, end=" ")  // 51, 51, 179, 64

Just to verify, I created another client in python, made the same request and converted the response bytes to float. I encountered the same issue here also. So I narrowed down that the problem is on the server side not on the client side. One more interesting thing I found when playing with multiple values are, only the numbers which are greater than 127 (I guess so) are getting converted to 255.

While trying some random hacks, I tried to convert the byte integer to character using chr() method of python before sending the response and the conversion on the client side worked.

byte_arr = np.array([5.6], dtype=np.float32).tobytes()
byte_arr_char = "".join([chr(i) for i in byte_arr])
return byte_arr_char

But the question I have is, Is this an ideal solution or am I doing some hack here to make it work? Can anyone help me to understand why sending plain bytes without chr() doesn't work?

Can you edit into the question what the value of byte_arr is in python, and what the value of response.text is in JS? — mostsquares
– mostsquares, Commented Mar 1, 2019 at 21:39
How is the response received? Are you making sure to set a suitable response type? — Martijn Pieters
– Martijn Pieters, Commented Mar 3, 2019 at 14:50
@CharlieWindolf Sure. But the value in JS script is all question marks. — Suba Selvandran
– Suba Selvandran, Commented Mar 6, 2019 at 9:11

gdlmx · Accepted Answer · 2019-03-02 04:23:07Z

11

+50

If you don't set the MIME type explicitly, I think Flask will treat it as text data. Your browser seems to have decoded the binary data with ASCII, which can explain why only values larger than 127 were affected.

Therefore, please try setting the Content-Type of the response in Flask:

@app.route('/your/url/to/numpy/data')
def get_nparray():
    your_np_array = np.array([5.6], dtype=np.float32)
    response = flask.make_response(your_np_array.tobytes())
    response.headers.set('Content-Type', 'application/octet-stream')
    # response.headers.set('Content-Disposition', 'attachment', filename='np-array.bin')
    return response

Alternatively, there is a helper function flask.send_file to construct the response in a single line. Please find an example here.

Besides this error, also pay attention to the endianness of your binary data, which is hardware specific. I would refer you to this answer (Javascript Typed Arrays and Endianness).

edited Mar 2, 2019 at 4:23

answered Mar 2, 2019 at 4:12

gdlmx

6,8691 gold badge25 silver badges44 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Suba Selvandran Over a year ago

Thanks for the answer. It works the way I wanted. Also, I read about the usage of send_file but the data will be received as an image, in that case, I wasn't sure how to receive the image as an array/text.

Suba Selvandran Over a year ago

I actually encountered another issue here. I am able to get the correct response in python but in JS where I am using angular's $http to make the response, I am still stuck with the same issue. The response I get is not a "application/octet-stream" response. This summarizes the problem github.com/angular/angular.js/issues/10349. Any solution for this?

Collectives™ on Stack Overflow

Send numpy array as bytes from python to JS through Flask

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related