How to write a python loop to change a value for a dictionary key in API request?

Question

I am writing an API request that gives paginated results. To get results from the next page I need to take a value of 'next_page_cursor' and put it in the parameters of my request that is a dictionary.

This is what I have tried so far. Need to keep changing cursor value in params until there are no more pages.

params = {'title': 'Cybertruck',
          'per_page':100,
          'cursor': '*'
         }


response = requests.get("https://api.aylien.com/news/stories", 
                       headers = headers,   params=params).json()

if "next_page_cursor" in response:
    cursor = response["next_page_cursor"]

You don't pass it explicitly as 'cursor': * - that's invalid, what's your initial params dict? — RomanPerekhrest
– RomanPerekhrest, Commented Dec 8, 2019 at 16:52
The initial dict is what I have in the post, the default value for 'cursor' is * which I need to swap for 'next_page_cursor" — alinaz
– alinaz, Commented Dec 8, 2019 at 16:54
you can't have * explicitly as value - that will raise SyntaxError: invalid syntax . Update your code — RomanPerekhrest
– RomanPerekhrest, Commented Dec 8, 2019 at 16:59

nicoring · Accepted Answer · 2019-12-08 21:09:15Z

1

You can use a while loop:

params = {
    "title": "Cybertruck",
    "per_page": 100,
    "cursor": "initial_cursor"
}

def make_request(params)
    return requests.get("https://api.aylien.com/news/stories", 
                        headers=headers, params=params).json()
result = []
response = make_request(params)
while "next_page_cursor" in response:
    params["cursor"] = response["next_page_cursor"]
    response = make_request(params)
    result.append(response["information_your_are_interested_in"])

edited Dec 8, 2019 at 21:09

answered Dec 8, 2019 at 17:01

nicoring

7236 silver badges13 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

alinaz Over a year ago

Thanks @nicoring! The resulting dictionary gives me only the values of the final page, rather than values from all pages. Do you know how to change that? Otherwise - this is very helpful, I'm much closer to what I was looking for

nicoring Over a year ago

You have to accumulate the results somewhere. This depends on how the response looks like in this case. I will update the answer to give you some pointers how to do it.

Collectives™ on Stack Overflow

How to write a python loop to change a value for a dictionary key in API request?

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related