0

Okay this is somewhat tricky. I have a DataFrame of people and I want to randomly select 27% of them. I want to create a new Boolean column in that DataFrame that shows if that person was randomly selected.

Anyone have any idea how to do this?

Improve this question
3
  • 1
    pandas.pydata.org/pandas-docs/stable/reference/api/… If you collect the ids from the sample, you can join this back on to see if they were sampled. Commented Jul 20, 2020 at 20:38
  • df.sample(frac=0.27) or df['selected'] = np.random.choice([0,1], size=len(df), p=[0.73,0.27])? Commented Jul 20, 2020 at 20:43
  • Great, the sample function seems like a great way to do it. But how do I create a new column that shows if a person was in the sample? Commented Jul 20, 2020 at 20:46

3 Answers 3

Reset to default
1

The in-built sample function provides a frac argument to give the fraction contained in the sample.

If your DataFrame of people is people_df:

percent_sampled = 27
sample_df = people_df.sample(frac = percent_sampled/100)

people_df['is_selected'] = people_df.index.isin(sample_df.index)
Improve this answer
Sign up to request clarification or add additional context in comments.

2 Comments

This make sense, thank you. However I'm getting an error. "A value is trying to be set on a copy of a slice from a DataFrame. Try using .loc[row_indexer,col_indexer] = value instead"
This is a silly warning that pandas incessantly throws, for very little good reason. Please read the discussion here: stackoverflow.com/questions/20625582/… I would recommend simply suppressing the warning by using pd.options.mode.chained_assignment = None after you import pandas.
0 
n = len(df) 
idx = np.arange(n)
idx = random.shuffle(idx)
*selected_idx = idx[:int(0.27*n)] 
selected_df = df[df.index.isin(selected_idx)]
Improve this answer

Comments

0

Defining a dataframe with 100 random numbers in column 0:

import random
import pandas as pd
import numpy as np
a = pd.DataFrame(range(100))
random.shuffle(a[0])

Using random.sample to choose 27 random numbers from the list, WITHOUT repetition: (replace 27 with 0.27*int(len(a[0]) if you want to define this as percentage)

choices = random.sample(list(a[0]),27)

Using np.where to assign boolean values to new column in dataframe:

a['Bool'] = np.where(a[0].isin(choices),True,False)
Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.