237 questions
0
votes
2
answers
92
views
How to Create a Pandas Dataframe from JSON Nested Objects [closed]
I'm trying to create a Pandas DataFrame from a JSON file that looks like this:
{
"GameID": "1,218,463,841",
"Date - Start": "1761097369",
"Date - End&...
2
votes
2
answers
94
views
Unnesting a pandas json column and keeping an "id" column
I am working on some nested NoSQL data. I would like to unnest it using json_normalize but keep the "id de transação" column so I could merge the resulting dataframe into other dataframes.
...
1
vote
0
answers
38
views
Attempting to flatten a list of dictionaries to a Dataframe but can't get to the lowest level information with json-normalise [duplicate]
Below is the output of an API call - it is a dictionary which contains a list of dictionaries I'm interested in under 'response'
I filter fixture_ids for 'response' because the rest is not useful ...
0
votes
1
answer
100
views
I'm trying to normalize this Json structure
I'm finding an error message when I try to normalize a Json structure that follows.
I pasted the JSON structure and the python code to normalize that is giving the ERROR message
GOAL: Normalize ...
-3
votes
2
answers
172
views
Using json_normalize with pandas
I'm using the following code in python to flatten the json structure below, however it doesn't work for all levels. I'm interested in the tags.tags column data specifically shown on the picture below
...
0
votes
1
answer
143
views
How do I use json_normalize on a json containing a list of nested lists
I'm dealing with a complex JSON string in Python, and I'm having trouble getting the data into a Pandas Data Frame. Sample data, current output, and expected output are all below. I'm attempting to ...
1
vote
1
answer
106
views
pandas retaining column after flattening json
I have a pandas dataframe where each row corresponds to a news article, and each cell in the row contains one dictionary that contains stock ticker information as well as some metrics. The dataframe ...
0
votes
1
answer
93
views
i have a nested list json file and i am using json_normalize to have the data but i am facing TypeError
i have a dataset of 3 objects, i am intrested of the data in object 1 which has, but it have nested lists, the file looks like this :
[{'ob1': [{'values':{'ID':2133 ,'STATUS': closed,'name':'Sara'}}
...
1
vote
1
answer
230
views
Flatten list of dict (array) as new columns not working with json normalize
Apologies as I'm quite inexperienced with Python and looking for assistance here 🙏
I am trying to flatten a list of dicts into new columns in an existing dataframe, with using key as column names and ...
0
votes
1
answer
65
views
Parse dict column and add new columns to dataframe [duplicate]
The pandas df is created by reading CSV file with more than 100+ columns and 2+ mil rows which looks similar to
project_code,dict_data
A001,{"a": "1", "b": "2", ...
0
votes
1
answer
155
views
Unable to explode the nested list within a json using pd.json_normalize
I tried different ways to get this done, but unable to do so. Please help here.
I am trying to explode a nested json list using python pd.json_normalize. After exploding the output is coming in two ...
0
votes
1
answer
559
views
How to normalise complex JSON with multiple levels of nested information in Python
I get the data in a predefined format I cannot change:
data =
{
"id": 12345,
"name": "Doe",
"gender": {
"textEn": "Masculin"
},
...
0
votes
1
answer
123
views
Flattening of json file
I am having problems while converting the json file to the dataframe. The json file structure is like this:
"results": [
{
"submissions": [
{
"...
0
votes
1
answer
521
views
Flattening the nested json file to dataframe using pandas json_normalise
I hace a big json file data and I want to convert it in to tabular form. I am trying to flatten the data in to dataframe using json_nomalise. so Far I have this :
code so far
I want to further flatten ...
0
votes
1
answer
136
views
How to Normalize JSON data into a pandas dataframe with json_normalize
I'm trying to transform this complex object into a pandas dataframe
data={
"customers": [
{
"a": 'true',
"addresses": [{"city&...
2
votes
1
answer
110
views
How to normalize nested fields with json_normalizer()?
I have a json with nested objects (nested list of objects):
json:
{
"uniqueIdentifier": {
"identity": {
"textIdentifier": "MysticFoliage"...
0
votes
1
answer
115
views
Json normalize meta issue [duplicate]
I am trying to convert a nested JSON to a dataframe. The JSON file is deeply nested and I am used the meta parameter to specify the nested structure so that all the JSON attributes can be stored as ...
1
vote
1
answer
57
views
How to Create new columns based on other column value and column names and assign value in pandas?
I have a dataframe with 8 columns, lets call it df.
Duration Location a b c d e f
3M Tuscaloosa 1 2 3 4 5 6
LCM Tuscaloosa 7 8 7 2 3 5
CW Tuscaloosa 9 1 0 ...
1
vote
2
answers
74
views
Create normalized dataframe from a nested json
I am trying to create a dataframe from nested json file but running into trouble.
[
{
"rec_id": "1",
"user": {
"id": "12414&...
2
votes
1
answer
81
views
How to use pd.json_normalize to retrieve the data I need from 2 parts (I'm almost there already, only need last piece of data)
I have this JSON list in Python:
[{'id': 'TD3$-FFA',
'shortCode': 'TD3$-FFA',
'dataSet': {'id': 'TD3C',
'shortCode': 'TD3C',
'shortDescription': 'Dirty Middle East Gulf to China',
'...
0
votes
1
answer
71
views
What is the right way to read csv with dictionary in one column in Python?
I have a CSV file named test.csv with the following data:
,ts,snapshot
0,1686321823605,"{'asks': [['26566.1', '1.451456']], 'bids': [['26566', '4.17579913']]}"
1,1686321823705,"{'asks': ...
1
vote
1
answer
138
views
From Python's nested dictionary to flat Pandas dataframe
I have a nested dictionary of the public information of employment history of people and I would like to construct the panel data similar to the following table.
Here is the nested dictionary.
The ...
0
votes
1
answer
104
views
Pandas's JSON_normalize for the nested dictionary
I have the following nested dictionary that contains the information that a person
publicly reported to the organization.
`person_json={'basicInformation': {'individualId': 5429958,
'firstName': '...
1
vote
1
answer
54
views
More pythonic (and faster) way to extract list of dicts from field, parse, and add as rows to DF
I have a DF with a bunch of fields (calls) that contains lists of JSON information.
NOTE: That for some of these pd.json_normalize() works, but for others I have to parse out the data manually (don't ...
0
votes
1
answer
38
views
How json normalize parses the below ISO Dates?
How json_normalize from pandas parses the ISODates that include timezone hours?
i.e.
"startDate" : ISODate("2021-02-01T22:37:37.000+02:00"),
"entryDate" : ISODate(...
1
vote
1
answer
30
views
Is there a way to get the ordinality of the array in the JSON using json_normalize or anyother way of doing it in python?
Is there a way to get the ordinality of the array in the JSON using json_normalize or anyother way of doing it in python ?
I have written the below code to flatten, but I need to add the ordinality, ...
10
votes
2
answers
27k
views
ImportError: cannot import name 'json_normalize' from 'pandas.io.json'
python 3.9.2-3
pandas 2.0.0
pandas-io 0.0.1
Error:
from pandas.io.json import json_normalize
ImportError: cannot import name 'json_normalize' from 'pandas.io.json' (/home/casaos/.local/lib/...
1
vote
1
answer
2k
views
Must be list or null Exception when attempting to pandas json_normalize
I have a pandas dataframe containing json data that I am attempting to normalize using pandas json_normalize.
import pandas as pd
df = pd.DataFrame({
'id': [0, 1],
...
2
votes
1
answer
379
views
JSON Normalize with Value as Column
I have the following JSON;
{
"data": [
{
"gid": "1203715497540179",
"completed": false,
"custom_fields": [
{
...
0
votes
0
answers
147
views
How to read Json file and to make it data frame in pandas
I want to read json file into pandas and make a data frame for further analysis like reject inferencing.
{
"loans": [
{
"loanId": "11111111111111111",
&...
0
votes
1
answer
86
views
Normalize a nested json file in Python
Suppose a list of objects: one, two, three,...
Every object is composed of name, foo1, and foo2 fields.
[{
'name':'one',
'foo2':{
'id':'1.1',
'id':'1.2'
},
'foo1':[
...
0
votes
1
answer
93
views
Python Converting Nested array of objects in JSON to flattened excel
Python Converting Nested array of objects in JSON to flattened excel
Current Code
def read_json(filename):
jsonData = {}
try:
with open(filename, "r", encoding="utf-8") as f:
...
1
vote
2
answers
2k
views
How to use pd.json_normalize to retrieve the data I need
I have this JSON list in Python:
[{'id': 'TC2-FFA',
'shortCode': 'TC2-FFA',
'dataSet': {'datumPrecision': 2,
'id': 'TC2_37',
'shortCode': 'TC2_37',
'shortDescription': 'Clean Continent to ...
-1
votes
1
answer
1k
views
Normalizing a Nested JSON in Python and Converting it to a Pandas Dataframe
I have created a simpler version of some JSON data I've been working with here:
[
{
"id": 1,
"city": "Philadelphia",
"Retaillocations&...
0
votes
1
answer
282
views
How to split a column with json string into their own columns
I have a dataframe like (with one example row):
raw_data = [{'id': 1, 'name': 'FRANK', 'attributes': '{"deleted": false, "rejected": true, "handled": true, "order&...
0
votes
1
answer
38
views
parse a quite nested Json file with Pandas/Python, the json thing is now in one column of a dataframe
I have retrieved data I need in one dataframe, one of the column has this list of dict
[{'date': '2023-02-03T00:00:00', 'groups': [{'periodType': 'm',
'...
0
votes
2
answers
66
views
Pandas | JSON Normalize | Deeply Nested Sections
I have a data structure which is a very complex/nested JSON.
After reading other solutions on SO, I am comfortable with the fact that I may need to make multiple DFs and merge them together (e.g ...
0
votes
1
answer
745
views
Join Multiple Json Files into one CSV
I have multiple new line delimited json files (they are nested) and I need to join all of them into one big csv. They all have the same schema (field names).
I have read several solutions on flatten ...
0
votes
1
answer
349
views
Why I'm getting KeyError when the column exists?
Using json_normalize I created a dataframe, the data I parsed includes a list of dictionaries, hence I used the parameters "record_path" and "meta".
The issue is that I get a ...
2
votes
0
answers
185
views
How to normalize complex JSON structure with 4 levels of nested arrays?
As I'm fairly new to python I've tried various ways based on answers found here but can't manage to normalize my json file.
As I checked in Postman it has 4 levels of nested arrays. For suppliers I ...
-1
votes
2
answers
633
views
How to get metadata when reading nested json with pandas
I'm trying to get the metadata out from a json using pandas json_normalize, but it does not work as expected.
I have a json fine with the following structure
data=[
{'a':'aa',
'b':{'b1':'bb1','...
0
votes
1
answer
104
views
List indicies must be integers or slices, not str
When I try to expose nested attributes in a json using json_normalise, I can't bring back the attributes of one of the nested objects. I'm trying to return the 'type' as a column but the following ...
0
votes
1
answer
64
views
Python: parce json with 2 arrays via json_normalize
Would you help, please, to parce 2-arrayed json via python, json_normalize.
Here is the code:
import json
from pandas.io.json import json_normalize
data5 = {
"id": "0001",
&...
0
votes
1
answer
53
views
Modification of a row value in pandas python
I have input data in this format:
{'entities': [('72, 78', '72, 78,Population'), ('56, 68', '56, 68,subtype_1')]}
It represents a row value in a csv. I want to transform the value into the following ...
0
votes
2
answers
791
views
convert a nested json response from API to a pandas dataframe using normalize
I've been trying to convert a json response from an api to a full panadas dataframe. I tried json normalize to achieve it unfortunately i was able to split it only to one level.
response = {
"...
1
vote
2
answers
148
views
How to use json_normalize to create subcolumns in csv file
I need some help on using json_normalizer to create multiple subcolumns from main columns ? The below code is able to generate an output file but its puts everything in one column however i need ...
1
vote
2
answers
608
views
Creating a pandas DataFrame from json object and appending a column to it
From an API, I get information about who has filled a particular form and when they have done it as a json object. I get the below data from 2 forms formid = ["61438732", "48247759dc&...
2
votes
2
answers
568
views
Normalizing json using pandas with inconsistent nested lists/dictionaries
I've been using pandas' json_normalize for a bit but ran into a problem with specific json file, similar to the one seen here: https://github.com/pandas-dev/pandas/issues/37783#issuecomment-1148052109
...
1
vote
1
answer
71
views
Extract data from Json into Pandas (Python)
I'm trying to extract data in a dataframe. My attempts with pd.json_normalize did not work... I must be doing something wrong.
Exemple :
{
"data": [
{
"date"...
0
votes
1
answer
3k
views
Python Converting Nested JSON with Array of Objects, and Objects to Excel
Can someone help me in converting the below JSON file to Excel?
Excel Output format
metaData.user report.from report.to lossInfo.locationName.Name lossInfo.locationName.locNbr lossInfo....