Quickly applying string operations in a pandas DataFrame

Question

Suppose I have a DataFrame with 100k rows and a column name. I would like to split this name into first and last name as efficiently as possibly. My current method is,

def splitName(name):
  return pandas.Series(name.split()[0:2])

df[['first', 'last']] = df.apply(lambda x: splitName(x['name']), axis=1)

Unfortunately, DataFrame.apply is really, really slow. Is there anything I can do to make this string operation nearly as fast as a numpy operation?

Thanks!

If you have pandas 0.8.1 or above, it looks like you should be able to do series.str.split(). Docs here: pandas.pydata.org/pandas-docs/stable/… — Thomas K
– Thomas K, Commented Oct 10, 2012 at 22:36

Wes McKinney · Accepted Answer · 2012-10-11 20:03:34Z

23

Try (requires pandas >= 0.8.1):

splits = x['name'].split()
df['first'] = splits.str[0]
df['last'] = splits.str[1]

answered Oct 11, 2012 at 20:03

Wes McKinney

106k32 gold badges146 silver badges109 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

duckworthd Over a year ago

Perfect! Didn't know about this addition.

LondonRob Over a year ago

Interestingly, this question is identical to this later one but the response has no mention of Series.split(). Has it been removed from pandas?

joris Over a year ago

It is now available as Series.str.split()

Collectives™ on Stack Overflow

Quickly applying string operations in a pandas DataFrame

1 Answer 1

3 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related