count occurrences of phrases in a python dataframe

Question

I have a df like this:

names = ["Internal medicine, Gastroenterology", "Internal medicine, Family and general medicine, Endocrinology", "Pediatrics, Medical genetics, Laboratory medicine", "Internal medicine"]
df = pd.DataFrame(names, columns=['names'])

I would like to know how often each medical term occurs. e.g. here

Internal medicine: 3
Gastroenterology: 1
etc

It works with Counter for words but how do I get it working for phrases such as "Internal medicine"? The ", " separates the phrases.

avarice · Accepted Answer · 2020-11-13 07:52:10Z

1

split by , and then explode and then value_counts

df['names'].str.split(", ").explode().value_counts()

edited Nov 13, 2020 at 7:52

answered Nov 13, 2020 at 7:45

avarice

15.6k3 gold badges19 silver badges38 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

lordy Over a year ago

should be split(", ") otherwise it miscounts but thanks otherwise great!! accepting in 5 min

Mayank Porwal · Accepted Answer · 2020-11-13 07:55:02Z

1

If you want to use collections.Counter, you can do this:

In [1945]: from collections import Counter

In [1946]: d = Counter(df['names'].str.split(", ").explode().tolist())

In [1947]: d
Out[1947]: 
Counter({'Internal medicine': 3,
         'Gastroenterology': 1,
         'Family and general medicine': 1,
         'Endocrinology': 1,
         'Pediatrics': 1,
         'Medical genetics': 1,
         'Laboratory medicine': 1})

answered Nov 13, 2020 at 7:55

Mayank Porwal

34.2k9 gold badges45 silver badges65 bronze badges

Collectives™ on Stack Overflow

count occurrences of phrases in a python dataframe

2 Answers 2

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related