How to index a dataframe using a condition on a column that is a column of numpy arrays?

Question

I currently have a pandas dataframe that has a column of values that are numpy arrays. I am trying to get the rows of the dataframe where the value of the column is an empty numpy array but I can't index using the pandas method. Here is an example dataframe.

data = {'Name': ['A', 'B', 'C', 'D'], 'stats': [np.array([1,1,1]), np.array([]), np.array([2,2,2]), np.array([])]}
df = pd.DataFrame(data)

I am trying to just get the rows where 'stats' is None, but when I try df[df['stats'] is None] I just get a KeyError: False. How can I filter by rows that contain an empty list?

Additionally, how can I filter by row where the numpy array is something specific? i.e. get all rows of df where df['stats'] == np.array([1, 1, 1])

Thanks

jezrael · Accepted Answer · 2021-08-24 10:52:06Z

1

You can check length by Series.str.len, because it working with all Iterables:

print (df['stats'].str.len())
0    3
1    0
2    3
3    0
Name: stats, dtype: int64

And then filter, e.g. rows with len=0:

df = df[df['stats'].str.len().eq(0)]
#alternative
#df = df[df['stats'].apply(len).eq(0)]
print (df)
  Name stats
1    B    []
3    D    []

If need test specific array is possible use tuples:

df =df[ df['stats'].apply(tuple) == tuple(np.array([1, 1, 1]))]
print (df)
  Name      stats
0    A  [1, 1, 1]

edited Aug 24, 2021 at 10:52

answered Aug 24, 2021 at 10:48

jezrael

868k103 gold badges1.4k silver badges1.3k bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Anonymous Over a year ago

This works! Thank you so much. Just one more thing how would I check if the numpy array is something specific i.e. get all rows of df where df['stats'] == np.array([1, 1, 1])

Niv Dudovitch Over a year ago

@Anonymous Answered your second question as another answer.

Niv Dudovitch · Accepted Answer · 2021-08-24 11:27:58Z

1

for this question: "Additionally, how can I filter by row where the numpy array is something specific? i.e. get all rows of df where df['stats'] == np.array([1, 1, 1])"

data = {'Name': ['A', 'B', 'C', 'D'], 'stats': [np.array([1,1,1]), np.array([]), np.array([2,2,2]), np.array([])]}
df = pd.DataFrame(data)
df = df[df['stats'].apply(lambda x: np.array_equal(x, np.array([1,1,1])))]

answered Aug 24, 2021 at 11:27

Niv Dudovitch

1,6789 silver badges15 bronze badges

Collectives™ on Stack Overflow

How to index a dataframe using a condition on a column that is a column of numpy arrays?

2 Answers 2

2 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related