In Python is there a way to create data frames from a loop and use the number in the data frame name?

Question

There are 48 files that I am looking to read in from GitHub into Python and am looking to use a loop to do this. Is there a way to name the dataframe using the number from loop?

number = 1  

while number < 48 :  
    gameweek[number] = pd.read_csv("https://raw.githubusercontent.com/vaastav/Fantasy-Premier-League/master/data/2019-20/gws/gw"+str(number)+".csv")

    number = number+1

Open to any suggestions on how best to do this.

The only problem is that gameweek[number] is uninitialized variable, so initialize gameweek to an empty list [] (or empty dict `{}') before you use it. This has nothing to do with pandas or dataframes per se. — smci
– smci, Commented Sep 14, 2020 at 6:34
Why not have one single dataframe and another column holding the gameweek? — Vishnudev Krishnadas
– Vishnudev Krishnadas, Commented Sep 14, 2020 at 6:44
@vishnudev this is a good option! Will look into this if I can't get the separate data frames working. — jpizz_888
– jpizz_888, Commented Sep 14, 2020 at 10:50
@karl_knetchel it wasn't allowing me to have a separate data frame for each gameweek which is what I was trying I achieve. It was suggested that I create a dictionary key for each game week - which works! Although vishnudev did call out this may not be memory efficient when working with larger datasets. — jpizz_888
– jpizz_888, Commented Sep 14, 2020 at 11:19

Ajay A · Accepted Answer · 2020-09-14 06:25:48Z

1

Try this

number = 1  
gameweek = {}
while number < 48 :  
    gameweek['gw_' + str(number)] = pd.read_csv("https://raw.githubusercontent.com/vaastav/Fantasy-Premier-League/master/data/2019-20/gws/gw"+str(number)+".csv")

    number += 1

answered Sep 14, 2020 at 6:25

Ajay A

1,0531 gold badge9 silver badges20 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Ajay A Over a year ago

I'm glad, I was helpful. Another tip exec is a in-built function, using which you can avoid using dictionary as well

Giovanni · Accepted Answer · 2020-09-14 06:30:34Z

1

Your code it's good for what you're trying to do, you just need to initialize your gameweek structure to a dictionary, so you just need to add before your while loop the following line: gameweek = {}

answered Sep 14, 2020 at 6:30

Giovanni

2122 silver badges12 bronze badges

5 Comments

Vishnudev Krishnadas Over a year ago

This solution of using a dict for small data sets is good. but for larger datasets, it will just bloat up memory.

Giovanni Over a year ago

HI @Vishnudev I know, especially if you want to get 48 pages full of data, I think that the best alternative might be saving every page on a different txt file. What do you think?

Vishnudev Krishnadas Over a year ago

How would that help? It would be the same. Reading from local vs remote doesn't help memory @Giovanni.

jpizz_888 Over a year ago

@giovanni thank you, you and Ajay A had the same idea and this works.

jpizz_888 Over a year ago

@vishnudev thankfully this is only using small amounts of data - 528rows and 32columns. I am really interested in knowing your views on what the best solution would be for larger datasets

Collectives™ on Stack Overflow

In Python is there a way to create data frames from a loop and use the number in the data frame name?

2 Answers 2

1 Comment

5 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

5 Comments

Your Answer

Sign up or log in

Post as a guest

Related