Looping through dictionaries within a list and add row to DataFrame

Question

I have a list of items I am trying to unpack from a dictionary which is nested inside another list.

I then want to add the row to a DataFrame.

headers = ['minutesPlayed', 'goals', 'goalsAssist', 'shotsOnTarget',
           'shotsOffTarget', 'shotsBlocked', 'hitWoodwork', 'totalContest', 
           'penaltyMiss', 'penaltyWon', 'bigChanceMissed']

Python Variables

The code I have tried is:

rows = []
for groups in data['groups']:
    row = []

    #Summary
    row.append(groups['minutesPlayed'])
    row.append(groups['goals'])
    row.append(groups['goalAssist'])

    #Attack
    row.append(groups['shotsOnTarget'])
    row.append(groups['shotsOffTarget'])
    row.append(groups['shotsBlocked'])
    row.append(groups['hitWoodwork'])
    row.append(groups['totalContest'])
    row.append(groups['penaltyMiss'])
    row.append(groups['penaltyWon'])
    row.append(groups['bigChanceMissed'])
    rows.append(row)

df = pd.DataFrame(rows, columns=headers)

However I receive the error:

KeyError: 'shotsOnTarget'

It doesn't allow me to iterate over the second element within the groups list.

Any tips?

EDIT added print of data[group]:

print(data['groups'])
[{'minutesPlayed': "89'", 'goals': '0', 'goalAssist': '1', 'statisticsItems': [{'minutesPlayed': 'Minutes played'}, {'goals': 'Goals'}, {'goalAssist': 'Assists'}], 'groupName': 'Summary'}, {'shotsOnTarget': '0', 'shotsOffTarget': '0', 'shotsBlocked': '1', 'hitWoodwork': '0', 'totalContest': '1 (0)', 'goals': '0', 'goalAssist': '1', 'penaltyMiss': '0', 'penaltyWon': '0', 'bigChanceMissed': '0', 'statisticsItems': [{'shotsOnTarget': 'Shots on target'}, {'shotsOffTarget': 'Shots off target'}, {'shotsBlocked': 'Shots blocked'}, {'totalContest': 'Dribble attempts (succ.)'}], 'groupName': 'Attack'}, {'touches': 55, 'accuratePass': '26 (70.3%)', 'keyPass': '1', 'totalCross': '0 (0)', 'totalLongBalls': '2 (0)', 'bigChanceCreated': '0', 'statisticsItems': [{'touches': 'Touches'}, {'accuratePass': 'Passes (acc.)'}, {'keyPass': 'Key passes'}, {'totalCross': 'Crosses (acc.)'}, {'totalLongBalls': 'Long balls (acc.)'}], 'groupName': 'Passing'}, {'possessionLost': '26', 'groundDuels': '9 (0)', 'aerialDuels': '3 (1)', 'wasFouled': '0', 'fouls': '2', 'offsides': '0', 'statisticsItems': [{'groundDuels': 'Ground duels (won)'}, {'aerialDuels': 'Aerial duels (won)'}, {'possessionLost': 'Possession lost'}, {'fouls': 'Fouls'}, {'wasFouled': 'Was fouled'}], 'groupName': 'Other'}, {'totalClearance': '0', 'clearanceOffLine': '0', 'blockedScoringAttempt': '0', 'interceptionWon': '0', 'totalTackle': '0', 'challengeLost': '1', 'lastManTackle': '0', 'errorLeadToShot': '0', 'errorLeadToGoal': '0', 'ownGoals': '0', 'penaltyConceded': '0', 'statisticsItems': [{'totalClearance': 'Clearances'}, {'blockedScoringAttempt': 'Blocked shots'}, {'interceptionWon': 'Interceptions'}, {'totalTackle': 'Tackles'}, {'challengeLost': 'Dribbled past'}], 'groupName': 'Defence'}]

'shotsOnTarget' is the second element which is accessible only in the second iteration of your loop. — Austin
– Austin, Commented May 10, 2020 at 13:40
Hi Austin, thanks for the quick reply, how would you access the second iteration within the loop? — Sunil Chavda
– Sunil Chavda, Commented May 10, 2020 at 15:25
@shivank98 yes I am using the headers list for the column names within my DataFrame — Sunil Chavda
– Sunil Chavda, Commented May 10, 2020 at 16:51
@SunilChavda can you please print the data['groups'] I am not exactly getting what that is from image — shivankgtm
– shivankgtm, Commented May 10, 2020 at 16:54

shivankgtm · Accepted Answer · 2020-05-10 17:16:32Z

1

Your data['groups'] is a list. whose elements are dicts now your minutesPlayed, goals and goalAssist is in your first element of the list so they are being called when your groupsvariable from for loop is run but your shotsOnTarget and others are in the second element. A nicer way of doing it can be following. you don't need a for loop.

#Summary
row.append(data['groups'][0]['minutesPlayed'])
row.append(data['groups'][0]['goals'])
row.append(data['groups'][0]['goalAssist'])

#Attact
row.append(data['groups'][1]['shotsOnTarget'])
# same for others,

answered May 10, 2020 at 17:16

shivankgtm

1,2421 gold badge10 silver badges21 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Sunil Chavda Over a year ago

thank you for your answer, so this solution partially worked but did not capture all as the elements of the groups could change. For example, data['groups'][1]['shotsOnTarget'] could change to data['groups'][5]['shotsOnTarget'] - being contained within the 5th element. I believe this is why I thought looping could have been a good idea

Collectives™ on Stack Overflow

Looping through dictionaries within a list and add row to DataFrame

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related