Requests module is multiplying api calls in Python

Question

I am using the Requests python module to do RESTFUL POST API calls to create a new item.

PROBLEM: Items are being duplicated. Sometimes 3 of each item get created. No check for existing data is in place in the final api prevent duplication.

GOAL: I want to send data to create a single new item in a third party application called Netbox which is made with Django.

WHAT HAVE I DONE?:

I have tried writing code to check for existing items and it still makes duplicates.
I have tried set a timer to pause execution of code by 10 seconds until the request is complete.

The process flow looks like this:

Python Backend (Flask)         ---> Python Backend (Flask)           --> Netbox (Django)

Data Origin sent via Requests  ---> Data Netbox Calls (Via PyNetBox) --> Final DataStore

The two backends are hosted in Cloud Foundry while Netbox is spun up on a Linux on Prem Server.

I have a dictionary like below that represents each item I need to create in Netbox.

Ex. payload = {'site':'1', 'device':'switch01'}

I send each piece of data to my middleware backend via a loop to make the final api call to netbox via pynetbox. I am not able to hit netbox directly.

for payload in payloads:
     URL = f'{MIDDLEWARE_URL}/netbox/create_item'
     print(f'URL is: {URL} | Payload is {payload}')

     headers = { 'content-type': 'application/json', 'cache-control': "no-cache"}
     proxies={'http' : None, 'https': None}

     r = requests.post(URL, data=json.dumps(payload), headers=headers, verify=False, proxies=proxies)

I would appreciate any help as to why this is happening and any best practices to improving my baseline code logic.

All the best, Faraz

This is the kind of problem that we can not solve for you. Requests is blocking IO, then adding a pause after a requests does not change anything on your python code. Try to debug step by step, and check all the requests you make. You are probably sending duplicate requests. "I am not able to hit netbox directly." This looks like a network problem. — Victor
– Victor, Commented May 28, 2020 at 16:49
BTW: instead of data=json.dumps(payload) you can use json=payload and it should also add 'content-type': 'application/json'automatically. — furas
– furas, Commented May 28, 2020 at 18:31

Faraz Ahmad · Accepted Answer · 2020-05-28 17:29:01Z

1

So I figured it out after some more research and it turns out flask was creating two processes and therefore running my functions twice each time.

This was due to debug being true so used this:

app.run(debug=True, use_reloader=False)

answered May 28, 2020 at 17:29

Faraz Ahmad

216 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Requests module is multiplying api calls in Python

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related