GET and POST in JSON using requests in Python

Question

I am working on a project where I have to GET data from a link and then POST the data in another server after I have extracted the information that I needed from the data fetched. I am using the library requests for my GET and POST, and here's the code for extracting the data needed:

    ''' original data fetched
     result= {"version": "1.0","cmd":"list_metering","status":"success","devices":
     [{"deviceid":"xxxxx","model":"xxxx"},{"deviceid":"xxxxx","model":"xxxx"}]} '''

for devices in result['devices']:


    # delete the parameters I don't need
    final_data = removekey(devices,'model')

    # x.update(final_data) -> Trying dicts
    # x.append(final_data) -> Trying lists

    # Sending directly 
    resp = requests.post(url,json=final_data,headers=headers)

    no+=1
    if no== len(result['devices']):
        break

This is where I will call a function that will delete keys that are not needed and then I will take the rest and post it.

and I tried using the update() function for dictionaries but it didn't work due to the fact that I have the same keys so only one of the data will be considered. Lists work but I will get the data in this form:

[{"deviceid":"xxxxx","model":"xxxx"},{"deviceid":"xxxxx","model":"xxxx"}]

and I tried to use json=data and x.json() but both didn't work out for lists. However, if I send the data directly as it is shown in my first code, due to delays and I am not so sure what else, if i am lucky, full data will be sent, otherwise I keep losing some part of the data fetched.

How do I GET data in json and then send it back again in json for this case? My aim is to send the data as one bundle, all the devices together so I don't lose anything.

Here are the GET and POST I am using:

 # GET 
url_source = 'https://website'
url = requests.get(url_source)
result = url.json()

 # POST 
headers = {'charset':'utf-8','Content-Type':'application/json'}
url = "http://xxxx/_get_v1.php"
data = final_data
resp = requests.post(url,json=data,headers=headers)

Could you make it more clearly? Add sample GET/POST requests if possible. — LittleQ
– LittleQ, Commented Jun 22, 2015 at 10:55

LittleQ · Accepted Answer · 2015-06-23 02:44:55Z

1

As I commented, devices is not an invalid input(though a valid json). According @AhmedAl-haddad reply, we should send {}-formatted data.

If it means to remove 'model' from devices and send the updated GET response data as below.

# POST
headers = {'charset':'utf-8','Content-Type':'application/json'}
url = "http://xxxx/_get_v1.php"

# remove 'models'
map(lambda x: x.pop('model'), result['devices'])

# send result
resp = requests.post(url,json=result,headers=headers)

Or we need to send device in devices separately?

# send result  
for device in result['devices']:
    resp = requests.post(url,json=device,headers=headers)

As @JonDeen mentioned, you should read the document to get more details

edited Jun 23, 2015 at 2:44

answered Jun 23, 2015 at 2:38

LittleQ

1,9251 gold badge14 silver badges14 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Ahmed Al-haddad Over a year ago

True. In this case I might send them as list of dictionaries or a dictionary once a time. The reason why I am trying to send them all at once is because I fear losing a few of the data sent due to timeout or unstable network etc. In this case I have 2 'devices' but if I have more then the chance of losing the data while sending them is higher. But seemingly it is not possible to and I might need to contact the admin. The receiver is in php, but I don't know it so I am not sure what I should say to the admin? How can he fix it? He is receiving the data in json as shown in my post request.

LittleQ Over a year ago

@AhmedAl-haddad use 'keep-alive' to send the requests, check the response and send the same json until success

JonDeen · Accepted Answer · 2015-06-23 06:29:55Z

0

Are you asking how to remove a key from a dictionary? If so, use the syntax

 del myJson["myKey"]

Also consider using the json-API for simplicity if you want to manipulate the dataset and restructure it from lists vs. strings.

Also check the Requests documentation for example on submitting json.

Example (updated to remove unnecessary list update):

devices=result['devices']
for i,device in enumerate(devices):

    # delete the parameters I don't need
    del device['model']

resp = requests.post(url,json=devices,headers=headers)

edited Jun 23, 2015 at 6:29

answered Jun 22, 2015 at 7:28

JonDeen

418 bronze badges

12 Comments

LittleQ Over a year ago

device is a dict object, you don't need devices[i]=device

JonDeen Over a year ago

Pardon, I may have gotten confused, but isn't device a dict, and devices a list?

LittleQ Over a year ago

@AhmedAl-haddad I guess you isolate devices from the result which is a list make it an invalid Json. You should replace devices in the result instead.

LittleQ Over a year ago

@JonDeen (devices is a list of dict :) @AhmedAI-haddad post requests.post(url,json=result,headers=headers), it would work.

JonDeen Over a year ago

@AhmedAl-haddad - when sending json data, even though the server does not necessarily like the contents of the json, the server is able to generate a status json-message, which technically makes it a valid web-page response (200). I'm not familiar with the json-argument in connection with requests.post, but try using a data-argument with json stringified data as mentioned in the requests documentation I linked to. Without knowing exactly what fields/format the server is requesting, we cannot possibly know how to format your json correctly, in which case you should contact the admin.

|

Collectives™ on Stack Overflow

GET and POST in JSON using requests in Python

2 Answers 2

2 Comments

12 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

12 Comments

Your Answer

Sign up or log in

Post as a guest

Related