How to filter out values from a pandas data frame for which only one occurrence exists

Question

I have a Pandas data frame with the following columns and values

  Temp  Time grain_size
0 335.0 25.0 14.8
1 335.0 30.0 18.7
2 335.0 35.0 22.1
3 187.6 25.0 9.8
4 227.0 25.0 14.2
5 227.0 30.0 16.2
6 118.5 25.0 8.7

The data frame given the variable name df that has three distinct value which are 335.0, 187.6, 227.0, and 118.5; however, the values 187.6 and 118.5 only occur once. I would like to filter the data frame such that it gets rid of values that only occur once so the final data frame looks like.

  Temp  Time grain_size
0 335.0 25.0 14.8
1 335.0 30.0 18.7
2 335.0 35.0 22.1
4 227.0 25.0 14.2
5 227.0 30.0 16.2

Obviously in this simple case I know the values that only occur once and I can simply user a filtering function to weed them out. However, I would like to automate the process so that Python will determine which values only occur once and autonomously filter them. How can I enable this functionality?

BENY · Accepted Answer · 2019-07-17 14:52:55Z

4

Using duplicated

df[df.Temp.duplicated(keep=False)]
Out[630]: 
    Temp  Time  grain_size
0  335.0  25.0        14.8
1  335.0  30.0        18.7
2  335.0  35.0        22.1
4  227.0  25.0        14.2
5  227.0  30.0        16.2

answered Jul 17, 2019 at 14:52

BENY

324k22 gold badges176 silver badges250 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

tawab_shakeel · Accepted Answer · 2019-07-17 14:55:00Z

0

Try this

df['count']=df.groupby(['Temp']).transform(pd.Series.count)
df = df[df['count']>1]
df.drop(['count'],axis=1,inplace=True)

answered Jul 17, 2019 at 14:55

tawab_shakeel

3,75912 silver badges26 bronze badges

Comments

piRSquared · Accepted Answer · 2019-07-17 15:15:16Z

0

`dict`

This is a dict approach to the same thing done by WeNYoBen

seen = {}
for t in df.Temp:
    seen[t] = t in seen

df[df.Temp.map(seen)]

    Temp  Time  grain_size
0  335.0  25.0        14.8
1  335.0  30.0        18.7
2  335.0  35.0        22.1
4  227.0  25.0        14.2
5  227.0  30.0        16.2

answered Jul 17, 2019 at 15:15

piRSquared

296k68 gold badges509 silver badges654 bronze badges

Collectives™ on Stack Overflow

How to filter out values from a pandas data frame for which only one occurrence exists

3 Answers 3

Comments

Comments

`dict`

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

Comments

dict

Comments

Your Answer

Sign up or log in

Post as a guest

Related

`dict`