When using a pandas dataframe, how do I add column if does not exist?

Question

I'm new to using pandas and am writing a script where I read in a dataframe and then do some computation on some of the columns.

Sometimes I will have the column called "Met":

df = pd.read_csv(File, 
  sep='\t', 
  compression='gzip', 
  header=0, 
  names=["Chrom", "Site", "coverage", "Met"]
)

Other times I will have:

df = pd.read_csv(File, 
  sep='\t', 
  compression='gzip', 
  header=0, 
  names=["Chrom", "Site", "coverage", "freqC"]
)

I need to do some computation with the "Met" column so if it isn't present I will need to calculate it using:

df['Met'] = df['freqC'] * df['coverage']

is there a way to check if the "Met" column is present in the dataframe, and if not add it?

YS-L · Accepted Answer · 2014-09-17 17:15:26Z

101

You check it like this:

if 'Met' not in df:
    df['Met'] = df['freqC'] * df['coverage']

answered Sep 17, 2014 at 17:15

YS-L

14.8k4 gold badges52 silver badges62 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

beyarkay Over a year ago

See stackoverflow.com/a/62449676/14555505 for how to add in multiple columns iff they don't exist

Jonatan Samoocha · Accepted Answer · 2021-03-05 14:52:46Z

11

When interested in conditionally adding columns in a method chain, consider using pipe() with a lambda:

df.pipe(lambda d: (
    d.assign(Met=d['freqC'] * d['coverage'])
    if 'Met' not in d else d
))

answered Mar 5, 2021 at 14:52

Jonatan Samoocha

3174 silver badges10 bronze badges

2 Comments

Adrian Torrie Over a year ago

Nice terse solution for chaining

Little Bobby Tables Over a year ago

Better still, you can drop the pipe and drop the negation from the if statement: df.assign(Met=lambda d: d.Met if 'Met' in d else d.freqC * d.coverage).

autonopy · Accepted Answer · 2020-05-27 20:05:54Z

6

If you were creating the dataframe from scratch, you could create the missing columns without a loop merely by passing the column names into the pd.DataFrame() call:

cols = ['column 1','column 2','column 3','column 4','column 5']
df = pd.DataFrame(list_or_dict, index=['a',], columns=cols)

answered May 27, 2020 at 20:05

autonopy

5148 silver badges16 bronze badges

Comments

rachwa · Accepted Answer · 2022-07-02 10:17:08Z

6

Alternatively you can use get:

df['Met'] = df.get('Met', df['freqC'] * df['coverage'])

If the column Met exists, the values inside this column are taken. Otherwise freqC and coverage are multiplied.

edited Jul 2, 2022 at 10:17

answered Jun 7, 2022 at 20:58

rachwa

2,3901 gold badge21 silver badges20 bronze badges

2 Comments

zk82 Over a year ago

I think this solution is correct but it's not as efficient as the others because the assignation is always done and the product is always done, as well.

zk82 Over a year ago

EDIT: In fact it may fail always since the DataFrame either has Met or freqC but not both so in order to be correct you should do something like df['Met'] = df.get('Met', df.get('freqC') * df['coverage']) (notice the new get for freqC)

Collectives™ on Stack Overflow

When using a pandas dataframe, how do I add column if does not exist?

4 Answers 4

1 Comment

2 Comments

Comments

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

1 Comment

2 Comments

Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related