Overwriting existing dataframe in loop

Question

I am trying to transform elements in various data frames (standardize numerical values to be between 0 and 1, one-hot encode categorical variables) but when I try to overwrite the dataframe in a loop it doesn't modify the existing dataframe, only the loop variable. Here is a dummy example:

t = pd.DataFrame(np.arange(1, 16).reshape(5, 3))
b = pd.DataFrame(np.arange(1, 16).reshape(5, 3))

for hi in [t, b]:
    hi = pd.DataFrame(np.arange(30, 45).reshape(5, 3))

But when I run this code both t and b have their original values. How can I overwrite the original dataframe (t or b) while in a loop?

The specific problem I'm running into is when trying to use get_dummies function in the loop:

hi = pd.get_dummies(hi, columns=['column1'])

@ColinRicardo I am not reassigning t or b anywhere outside of the loop. When I print t and b they are both still 5x3 dataframe with values 1 through 15 — CrashingWater
– CrashingWater, Commented Apr 16, 2018 at 21:08
@CrashingWater you're not reassigning t or b inside the loop either, so you shouldn't expect them to change. — Colin Burke
– Colin Burke, Commented Apr 16, 2018 at 21:09
Yes, this will not work. You need to refer to the dataframes in your list by index (not by a temporary value such as hi). You can find a SO answer that deals with this here — Oriol Mirosa
– Oriol Mirosa, Commented Apr 16, 2018 at 21:18

hhoke1 · Accepted Answer · 2018-04-16 21:23:59Z

1

You can't change elements of a list while iterating over the list that way. Search "changing list elements loop python" for a bunch of good stack overflow questions on why this is the case. My understanding is that "hi" is value-copied, not a reference to the original variable.

If you want to modify elements in a list iteratively, you can try enumerate(), or list comprehensions. You might want to create a dictionary of lists and iterate over that, instead of using variable names to keep track of all the lists, as suggested here.

answered Apr 16, 2018 at 21:23

hhoke1

2221 silver badge10 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Overwriting existing dataframe in loop

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related