Extracting text from a nested JSON file where each JSON object has variable number of entries in Python

Question

I have a json file containing multiple nested json objects which look like this:

{
"coordinates": null,
"acoustic_features": {
    "instrumentalness": "0.00479",
    "liveness": "0.18",
    "speechiness": "0.0294",
    "danceability": "0.634",
    "valence": "0.342",
    "loudness": "-8.345",
    "tempo": "125.044",
    "acousticness": "0.00035",
    "energy": "0.697",
    "mode": "1",
    "key": "6"
},
"artist_id": "b2980c722a1ace7a30303718ce5491d8",
"place": null,
"geo": null,
"tweet_lang": "en",
"source": "Share.Radionomy.com",
"track_title": "8eeZ",
"track_id": "cd52b3e5b51da29e5893dba82a418a4b",
"artist_name": "Dominion",
"entities": {
    "hashtags": [{
        "text": "nowplaying",
        "indices": [0, 11]
    }, {
        "text": "goth",
        "indices": [51, 56]
    }, {
        "text": "deathrock",
        "indices": [57, 67]
    }, {
        "text": "postpunk",
        "indices": [68, 77]
    }],
    "symbols": [],
    "user_mentions": [],
    "urls": [{
        "indices": [28, 50],
        "expanded_url": "cathedral13.com/blog13",
        "display_url": "cathedral13.com/blog13",
        "url": "t.co/Tatf4hEVkv"
    }]
},
"created_at": "2014-01-01 05:54:21",
"text": "#nowplaying Dominion - 8eeZ Tatf4hEVkv #goth #deathrock #postpunk",
"user": {
    "location": "middle of nowhere",
    "lang": "en",
    "time_zone": "Central Time (US & Canada)",
    "name": "Cathedral 13",
    "entities": null,
    "id": 81496937,
    "description": "I\u2019m a music junkie who is currently responsible for 
Cathedral 13 internet radio (goth, deathrock, post-punk)which has been 
online since 06/20/02."
},
"id": 418243774842929150
}

Each objects contain variable number of hashtags. I want to obtain a csv file containing the hashtag texts. I have written the following code to do it:

import csv
with open('jsonpart.json') as data_file:
    data = json.load(data_file)
    #print (data)
    header = ['hashtags']

# open a file for writing
data_csv = open('hashtags.csv', 'wb')
# create the csv writer object
csvwriter = csv.writer(data_csv)

# write the csv header
csvwriter.writerow(header)

for entry in data:
    csvwriter.writerow([entry['entities']['hashtags']])

data_csv.close()

I get the following output file:

"[{u'indices': [0, 11], u'text': u'nowplaying'}, {u'indices': [51, 56], 
 u'text': u'goth'}, {u'indices': [57, 67], u'text': u'deathrock'}, 
{u'indices': [68, 77], u'text': u'postpunk'}]"
"[{u'indices': [23, 34], u'text': u'NowPlaying'}, {u'indices': [75, 79], 
u'text': u'80s'}, {u'indices': [80, 86], u'text': u'Retro'}, {u'indices': 
[87, 91], u'text': u'Fun'}]"
"[{u'indices': [0, 11], u'text': u'nowplaying'}]"
"[{u'indices': [54, 65], u'text': u'nowplaying'}, {u'indices': [66, 77], 
u'text': u'listenlive'}]"

I am stuck from here. How do I get my target file as:

nowplaying
goth
deathrock
postpunk
NowPlaying  
80's
Retro
Fun
nowplaying
nowplaying
listenlive

I don't see this values 80's Retro Fun nowplaying nowplaying listenlive within your json — RomanPerekhrest
– RomanPerekhrest, Commented Jul 7, 2017 at 7:32
@RomanPerekhrest These are the values of the 'entities' 'hashtags' 'text' of the next JSON object. — Asmita Poddar
– Asmita Poddar, Commented Jul 7, 2017 at 7:38

jlaur · Accepted Answer · 2017-07-07 10:28:40Z

1

You could use a simple list comprehension. Given that you have json object called json_chunk you could create the list like so:

text_list = [hashtag['text'] for hashtag in json_chunk['entities']['hashtags']]

Now you have a list. Iterate it (some element apparently have a new line character others dont - so strip all and add new line character to all) and write each element to a file like so:

with open(r'C:\outputfile.csv', 'a', encoding='utf-8') as fd:
    for line in text_list:
    fd.write(line.strip()+'\n')

edited Jul 7, 2017 at 10:28

answered Jul 7, 2017 at 8:02

jlaur

7406 silver badges13 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Asmita Poddar Over a year ago

Thank you, @jlaur. I get my result file in the format: nowplaying, goth, deathrock, postpunk \n NowPlaying, 80's, Retro, Fun \n nowplaying \n nowplaying, listenlive ... How do I get it in the above mentioned format as in the question?

jlaur Over a year ago

I updated the code. But you should try to figure it out before asking how to do it. You have a list. How do you iterate through a list and write it to a file. Ask Google. There are tons of tutorials on stuff like that on the internet.

Collectives™ on Stack Overflow

Extracting text from a nested JSON file where each JSON object has variable number of entries in Python

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related