filter pandas DataFrame column list with other list

Question

Input DataFrame :

data = { "id" : ['[1,2]','[2,4]','[4,3]'],
             "name" : ['a','b','c'] }
df = pd.DataFrame(data)

filterstr = [1,2]

Expected Output:

id     name
[1,2]   a      
[2,4]   b

Tried Code : df1 = df[df.id.map(lambda x: np.isin(np.array(x), [[ str([i]) for i in filter]]).all())]

This works for single value in id column but not for two values like '[1,2]' Not sure where i am going wrong.

The type of id column is a list or a string representation of a list? In the last case, is it safe to convert it as a list? Your filterstr is a list? — Corralien
– Corralien, Commented May 29, 2022 at 18:55
Not sure to understand, do you want partial or exact match? If exact why "b" is selected? If partial why "c" not selected (contains "2")? — Drakax
– Drakax, Commented May 29, 2022 at 22:15

BeRT2me · Accepted Answer · 2022-05-31 00:28:37Z

1

Taking exactly what you've given:

data = { "id" : ['[1,2]','[2,4]','[4,3]'],
             "name" : ['a','b','c'] }
df = pd.DataFrame(data)

filterstr = [1,2]

I do:

df['id'] = df['id'].apply(eval) # Convert from string to list.
output = df[df.id.map(lambda id: any(x for x in id if x in filterstr))]
print(output)

Output:

       id name
0  [1, 2]    a
1  [2, 4]    b

answered May 31, 2022 at 0:28

BeRT2me

13.3k2 gold badges18 silver badges39 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Zoe - Save the data dump · Accepted Answer · 2022-06-02 08:42:29Z

0

Creating DF:

data = { "id" : [[1,2],[2,4],[4,3]], #REMOVE STRING AROUND []!!!
         "name" : ['a','b','c'] }
df = pd.DataFrame(data)
df

Result:

Then let's create a variable which will be our "boolean" filter:

reg = [1,2]
filter = df.id.apply(lambda x: any(i for i in x if i in reg))

Intermediate result:

0     True
1     True
2    False
Name: id, dtype: bool

Then select only "True" values:

df = df[filter]
df

Final result:

index	id	name
0	1,2	a
1	2,4	b

28.4k22 gold badges130 silver badges163 bronze badges

answered May 31, 2022 at 0:17

Drakax

1,5735 silver badges11 bronze badges