How to preserve the Ascii encoding type with the POST request in Python?

Question

How do I send the ASCII encoded text via POST request in Python? The length of true_input I received via the POST is always different from the length I sent.

def insert_true_input(level, iteration, true_input):
    url = master_url + "/insert_true_input?"
    data = {'level': level, 'iteration': iteration, 'true_input': true_input}
    headers = {'Content-Type': 'text/plain'}
    res = requests.post(url, params=data, headers=headers).text
    return res

The sample true_input that I want to send is directly from numpy.ndarray.tostring() and looks like

'\x00\x00\x00\x00\x00\x00\xf0?\x00\x00\x00\x00\x00\x00\x00@\x00\x00\x00\x00\x00\x00\x08@\x00\x00\x00\x00\x00\x00\xf0?\x00\x00\x00\x00\x00\x007@\x00\x00\x00\x00\x00\xc0^@\x00\x00\x00\x00\x00\xc0^@\x00\x00\x00\x00\x00\xc0^@\x00\x00\x00\x00\x00\x00(@\x00\x00\x00\x00\x00\x00?@'

What do you want to do exactly ? Do you want to POST your data in the request body, or pass it in the query string ? By using the params parameter to requests.post(), you are actually using a query string to post your data, not the request body. If you want to pass it as body, use the data parameter instead, and remove your custom Content-Type header (requests will use Content-Type application/x-www-form-urlencoded automatically). — Guillaume
– Guillaume, Commented Dec 5, 2016 at 9:28
Please provide a sample input for "true_input" and sample output (full HTTP request headers and body), it will make it easier to understand what you want. — Guillaume
– Guillaume, Commented Dec 5, 2016 at 9:31
@Guillaume Thanks for you answer! Please see my updated question. — xxx222
– xxx222, Commented Dec 5, 2016 at 9:33
Your input is full of null characters (all bits set to 0), represented by \x00. That is not something easy to pass by text. Can you encode it using base64 first ? Do you have control over the other side which will receive the request ? — Guillaume
– Guillaume, Commented Dec 5, 2016 at 9:39
Could you explain how to do that? I have Java code in the other end to receive the data. — xxx222
– xxx222, Commented Dec 5, 2016 at 9:42

User2342342351 · Accepted Answer · 2016-12-05 09:26:41Z

2

You have to encode your string using str.encode('ascii'):

def insert_true_input(level, iteration, true_input):
    url = master_url + "/insert_true_input?"
    data = {'level': level, 'iteration': iteration, 'true_input': true_input.encode('ascii')}
    headers = {'Content-Type': 'text/plain'}
    res = requests.post(url, params=data, headers=headers).text
    return res

edited Dec 5, 2016 at 9:26

answered Dec 5, 2016 at 9:18

User2342342351

2,1741 gold badge14 silver badges23 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

xxx222 Over a year ago

Thanks for your answer! However my actual question is how do I keep the true_input unchanged with the POST process, which doesn't related with the res returned to me. The POST method will send my data to the other end which I didn't post here.

Guillaume · Accepted Answer · 2016-12-05 09:54:54Z

1

As explained in the comments, the null characters \x00 are not sendable in raw text. You have to encode them one way or another (URL encoded, Base64, json, etc.). But then the other side that will receive the request must be adapted to decode them accordingly. Actually requests will use URL encoding automatically for the parameters passed in the query string, but I suspect that your java code is not able to decode them properly.

Please post your Java code for the receiving side to see what we can do.

Suggestions on python side, using base64:

import base64
def insert_true_input(level, iteration, true_input):
    url = master_url + "/insert_true_input?"
    data = {'level': level, 'iteration': iteration, 'true_input': base64.b64encode(true_input)}
    res = requests.post(url, params=data, headers=headers).text
    return res

Using json (requests will do the work for you if you use the json parameter to .post()):

 def insert_true_input(level, iteration, true_input):
    url = master_url + "/insert_true_input?"
    data = {'level': level, 'iteration': iteration, 'true_input': true_input}
    res = requests.post(url, json=data, headers=headers).text
    return res

edited Dec 5, 2016 at 9:54

answered Dec 5, 2016 at 9:49

Guillaume

6,1293 gold badges28 silver badges48 bronze badges

2 Comments

xxx222 Over a year ago

I've never tried base64 before, and it turns out it is perfect to use it for the encoding of numpy array.

Guillaume Over a year ago

You're welcome. Actually base64 is perfect whenever you have to transmit or store arbitrary binary data over a text-only medium.

Collectives™ on Stack Overflow

How to preserve the Ascii encoding type with the POST request in Python?

2 Answers 2

1 Comment

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related