GET Request in For Loop only returning response from the final loop index. Preceding ones return empty

Question

I'm running get requests to a URL, using lines in a text file as arguments in said get requests. The issue I'm finding is that the responseind.json() is only returning the get request related to the LAST line of the txt file. All preceding lines return []. Code below, any ideas?

with open('industries.txt') as industry_file:
       for line in industry_file:
        start = time.time()
        responseind = requests.get("https:URL" + "".join(line) + "?token=My key")
        print(responseind.json())

is it possible that you are sending too many requests to the server? — Andrew Ryan
– Andrew Ryan, Commented Feb 6, 2022 at 0:12
could you show what response get for last line of text file also print responseid.text. my guess is perhaps the json happens to be an empty list for those — kamster
– kamster, Commented Feb 6, 2022 at 0:20
Please check that response status is also 200 or 201 so that the request was succesful. Are you sure that endpoint encoding is json ? — Michał Darowny
– Michał Darowny, Commented Feb 6, 2022 at 0:55
@AndrewRyan: i tried adding a wait between the consecutive calls to slow it down (went up to 15 seconds of wait), didn't help. — Amostafa
– Amostafa, Commented Feb 6, 2022 at 1:01
@MichałDarowny: i checked the response status within the for loop to confirm success, 200 every time. — Amostafa
– Amostafa, Commented Feb 6, 2022 at 1:02

Andrew Ryan · Accepted Answer · 2022-02-06 16:35:00Z

1

The reason why your requests are not working properly is that you are just using the entire line in the text file (which ends in a new line character '\n'). The reason why your last request may be getting a response is that there is no new line after that id (the file ends) therefore not having that character which gives a valid id to look for.

with open('industries.txt') as industry_file:
    for line in industry_file:
        start = time.time()
        responseind = requests.get("https:URL" + "".join(line).replace('\n', '') + "?token=My key")
        print(responseind.json())

answered Feb 6, 2022 at 16:35

Andrew Ryan

1,6313 gold badges17 silver badges24 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

GET Request in For Loop only returning response from the final loop index. Preceding ones return empty

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related