Combining columns and joining non-missing values in Pandas

Question

Imagine that I have a single Dataframe as such:

df = pd.DataFrame([[1,2,3,None],[1,2,3,None],[1,2,3,None],[None,2,3,1]], columns=["A","B","C","AA"])

A	B	C	AA
1	2	3
1	2	3
	2	3	1

Column AA is actually the same as A, but has suffered a typo somewhere in the data processing pipeline precious steps.

How can I actually rename ['AA'] to ['A'] and move the non-missing values? Example:

A	B	C
1	2	3
1	2	3
1	2	3

I imagine that if I do:

df['A'] = df['AA']

Null values will be copied.

So, any hints here?

sammywemmy · Accepted Answer · 2021-02-11 22:23:29Z

1

You could try combine_first:

In [8]: df.assign(A=df.A.combine_first(df.AA)).drop(columns='AA')
Out[8]: 
     A  B  C
0  1.0  2  3
1  1.0  2  3
2  1.0  2  3
3  1.0  2  3

answered Feb 11, 2021 at 22:23

sammywemmy

28.9k4 gold badges21 silver badges35 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Tiago Duque Over a year ago

Just had to do some working to index names by square brackets and moving the drop for another line, but it worked. Thanks. Also, more complete answer that works even for non-numerical values.

mullinscr · Accepted Answer · 2021-02-11 19:22:56Z

0

Sum them both together:

df['A'] = df[['A','AA']].sum(axis=1)

Result is:

     A  B  C   AA
0  1.0  2  3  NaN
1  1.0  2  3  NaN
2  1.0  2  3  NaN
3  1.0  2  3  1.0

answered Feb 11, 2021 at 19:22

mullinscr

1,7681 gold badge8 silver badges14 bronze badges

Comments

William Baker Morrison · Accepted Answer · 2021-02-11 19:30:13Z

0

To add to @mullinscr, first sum the columns and then drop the 'AA' column

df['A'] = df[['A','AA']].sum(axis=1)
df.drop('AA', axis=1, inplace=True)

edited Feb 11, 2021 at 19:30

William Baker Morrison

1,8074 gold badges22 silver badges35 bronze badges

answered Feb 11, 2021 at 19:25

Utpal Dutt

4034 silver badges18 bronze badges

2 Comments

Tiago Duque Over a year ago

Would that work for non-numeric values? While my case is for numbers, that should be considered for a stack overflow answer.

Utpal Dutt Over a year ago

that would not work for non numeric values.

Collectives™ on Stack Overflow

Combining columns and joining non-missing values in Pandas

3 Answers 3

1 Comment

Comments

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related