python send http request with multiple parameters with same name

Question

I need to send a simple form to a remote server using POST method with python. This form is a bit strange... some fields name are repeated without using the [] but only passing the key=value pair (this is a requisite otherwise the process on server side doesn't work!).

Below an example of the original form:

  <form method="post" action="/remote/test.php">
    <input type=hidden name="value01" value="7777" size="12" maxlength=20>
    <input type="text" name="value02" value="123456789">
    <input type="text" name="value03" value="A1234">
    <input type="text" name="value04" value="B5678">
    <input type=submit value="value05 " class="button">
    <input type="hidden" name="VALUE00" value="9999">
    <input type=hidden name="VALUE99"value="09.01.2015 14:52:40">
    <input type="hidden" name="LIST" value="D0000000033039">
    <input type="hidden" name="LIST" value="D0000000033039">
    <input type="hidden" name="LIST" value="C0000000032032">
    <input type="hidden" name="LIST" value="X0000000031820">    
  </form>

I use "Requests" and as per the documentation, i can pass a dictionary for sending post parameters. So i implement below code. The real data that fill the sourceDataList variable are read from a text file with a windows EOL (CR+LF), so to "emulate" the file for this example, i compile a list with some value similar to real values that are present in the file

import re

sourceDataList = [
 '2017-12-20 08:59:17;Value01\r\n',
 '2017-12-20 08:59:18;Value02\r\n', 
 '2017-12-20 08:59:20;Value03\r\n',
 '2017-12-20 08:59:21;Value04\r\n'
]
dataList = [];
for line in sourceDataList:
  dataList.append({'RESULT' , re.sub('[^A-Za-z0-9]+', '', line.split(';')[1].strip())});

print dataList;

The problem start here.. when I print the dataList I discover that the data are "mixed" in a strange way.. sometime there is a swap between key and value!

as example, the code above produce below result:

[
 set(['Value01', 'RESULT']), 
 set(['RESULT', 'Value02']), 
 set(['RESULT', 'Value03']), 
 set(['Value04', 'RESULT'])
]

where did i go wrong?

- UPDATE 1 -

my purpose is to create a dictionary that i can use with "Requests". If i use above set the call work as expected and send the data to server but in wrong order (some key and value inverted)

If i use a real dictionary like this (notice that now i use : instead , ):

  for line in sourceDataList:
      dataList.append({'RESULT' : re.sub('[^A-Za-z0-9]+', '', line.split(';')[1].strip())});

and use it as parameter for "requests":

r = requests.post(url, data=dataToSend);

I get an error:

  File "test.py", line 46, in <module>
    r = requests.post(url, data=dataToSend);
  File "C:\Python27\lib\site-packages\requests\api.py", line 112, in post
    return request('post', url, data=data, json=json, **kwargs)
  File "C:\Python27\lib\site-packages\requests\api.py", line 58, in request
    return session.request(method=method, url=url, **kwargs)
  File "C:\Python27\lib\site-packages\requests\sessions.py", line 494, in request
    prep = self.prepare_request(req)
  File "C:\Python27\lib\site-packages\requests\sessions.py", line 437, in prepare_request
    hooks=merge_hooks(request.hooks, self.hooks),
  File "C:\Python27\lib\site-packages\requests\models.py", line 308, in prepare
    self.prepare_body(data, files, json)
  File "C:\Python27\lib\site-packages\requests\models.py", line 499, in prepare_body
    body = self._encode_params(data)
  File "C:\Python27\lib\site-packages\requests\models.py", line 97, in _encode_params
    for k, vs in to_key_val_list(data):
ValueError: need more than 1 value to unpack

You can remove the code related to form and HTTP request which is totally irrelevant to your problem. And should explain more about the problem in later code. — Moinuddin Quadri
– Moinuddin Quadri, Commented Jan 2, 2018 at 12:24

Moinuddin Quadri · Accepted Answer · 2018-01-02 12:15:29Z

Here you are storing your data as set. Sets in Python are unordered in nature. {a, b} (note comma , here) is a syntax of a set. Probably you want to store it as dict (if you want it as key-value pair) which has the syntax {a: b} (note here the colon : instead of ,). OR, may be you can store it as a tuple too.

For example:

import re

sourceDataList = [
     '2017-12-20 08:59:17;Value01\r\n',
      '2017-12-20 08:59:18;Value02\r\n',
      '2017-12-20 08:59:20;Value03\r\n',
      '2017-12-20 08:59:21;Value04\r\n'
]

# --- Using Dictionary ---
dataList = []
for line in sourceDataList:
    dataList.append({'RESULT' : re.sub('[^A-Za-z0-9]+', '', line.split(';')[1].strip())})

# --- Using tuple ---
dataList = []
for line in sourceDataList:
    dataList.append(('RESULT', re.sub('[^A-Za-z0-9]+', '', line.split(';')[1].strip())))

Here are one-liners to get the same result:

# For Dictionary 
dataList = [{'RESULT' : re.sub('[^A-Za-z0-9]+', '', line.split(';')[1].strip())} for line in sourceDataList]

# For Dictionary 
dataList = [('RESULT', re.sub('[^A-Za-z0-9]+', '', line.split(';')[1].strip())) for line in sourceDataList]

Here are the sample of the result that you'll receive:

# --- For Dictionary ---
[{'RESULT': 'Value01'}, {'RESULT': 'Value02'}, {'RESULT': 'Value03'}, {'RESULT': 'Value04'}]

# --- For tuple ---
[('RESULT', 'Value01'), ('RESULT', 'Value02'), ('RESULT', 'Value03'), ('RESULT', 'Value04')]

Marco · Accepted Answer · 2018-01-03 07:36:54Z

after some tests and after reading a lot of documentation about requests I finally found a solution to my specific problem. To sum up, i need to send some data to a web address using POST method with one parameter. This parameter can contain different data with same field name. As example: TESTFIELD= test01, TESTFIELD= test02

and so on..

now, the problem can be solved using this syntax:

import re

def sendData():


  sourceDataList = [
    '2017-12-20 08:59:17;Value01\r\n',
    '2017-12-20 08:59:18;Value02\r\n', 
    '2017-12-20 08:59:20;Value03\r\n',
    '2017-12-20 08:59:21;Value04\r\n'
   ]

  dataList = [];
  for line in sourceDataList:
    dataList.extend([re.sub('[^A-Za-z0-9]+', '', line.split(';')[1].strip())]);
  return {'TESTFIELD': dataList}

this function return below dictionary:

{'TESTFIELD': ['test01', 'test02', 'test03', 'test04']}

that request convert in a POST body like this:

TESTFIELD=test01&TESTFIELD=test02&TESTFIELD=test03&TESTFIELD=test04

FINAL WARNING! this is specific uses case, where the field name must be always the same, this imply that also the server side code must be able to read this specific set-up. I must use this service that is not made by me and i can't change anything on server side so i need to find this solution. The correct way to pass multiple value with same field name is to use []. As example:

TESTFIELD[] = test01, TESTFIELD[] = test02 and so on...

This return an array on server side. Instead, if you use this solution some server side languages can't handle it with standard functions.

As example, with PHP you can't rely on $_POST superglobal you will always find ONLY ONE VALUE because this is a non standard way to pass multiple value for same parameters. If you use PHP you must use file_get_contents("php://input") to obtain all values and you must parse it by yourself.

Collectives™ on Stack Overflow

python send http request with multiple parameters with same name

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related