1

import json import pandas as pd

resp = requests.get('https://data.cms.gov/provider-data/api/1/datastore/query/77hc-ibv8/0?offset=0&count=true&results=true&schema=true&keys=true&format=json')
data = resp.json()
data = json.dumps(resp_dict,indent=4)
print(data)

I am trying to unnest the json data into a dataframe using Python. There are a few ways to do it but wasn't able to fully unnest the json data. What would be the best way to accomplish this?enter image description here

Improve this question
2
  • kindly post your expected output dataframe; also share the code you tried that worked/failed? Commented Oct 10, 2022 at 4:04
  • What's nested? Do you mean the results? pd.DataFrame(data['results']) will make the frame. Commented Oct 10, 2022 at 4:07

1 Answer 1

Reset to default
1

Nothing special just pass it to pd.DataFrame and refer to the main key.

import json

import pandas as pd

import requests


def get_data() -> pd.DataFrame:
    url = (
        "https://data.cms.gov/provider-data/api/1/datastore/query/77hc-ibv8/0?"
        "offset=0&"
        "count=true&"
        "results=true&"
        "schema=true&"
        "keys=true&"
        "format=json"
    )

    with requests.Session() as request:
        response = request.get(url, timeout=10)
    if response.status_code != 200:
        print(response.raise_for_status())

    data = json.loads(response.text)

    return pd.DataFrame(data=data["results"])


print(get_data())
Improve this answer
Sign up to request clarification or add additional context in comments.

1 Comment

Hi Jason, running this script returns the first 500 rows. Is there a way to return the entire rows? thanks alot!

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.