How to delete rows from dataframe?

Question

I have dataframe df

   df.show()
   id|data|somedata
   1 |k   |v
   2 |s   |d
   3 |f   |k
   .
   .
   .

I tried to delete rows from df that id exist in lisst=List(4,9,200) so I used drop like this

   val df1=df.drop(col("id").isin(lisst:_*))

but does'nt work also I tried

   val df1=df.filter(col("id").isin(lisst:_*).drop("id"))

but df1 have same rows in df

akuiper · Accepted Answer · 2018-03-31 18:30:46Z

4

Simply using filter or where with the condition should work; no drop is needed if you don't plan to delete columns:

df.filter(!col("id").isin(lisst:_*))

or:

df.where(!col("id").isin(lisst:_*))

answered Mar 31, 2018 at 18:20

akuiper

216k33 gold badges362 silver badges379 bronze badges

Sign up to request clarification or add additional context in comments.

1 Answer 1