How to delete a row that contains a specific string in Python using numpy？

Question

Now I have a huge data ndarray as below, which contains 200 million rows.

[['5040' '5' 'load_video' '2015-03-30 12:31:27.727452'
['5040' '44' 'load_video' '2015-03-30 12:33:26.764407']
['5040' '34' 'load_video' '2015-03-30 12:31:26.102226']
 ..., 
['3076' '1' 'play_video' '2015-05-31 05:52:33.395859']
['3076' '1' 'seek_video' '2015-05-31 05:52:36.941808']
['1512' '8' 'load_video' '2015-05-31 07:19:56.715000']]]

What I want to do is to delete all rows that contain 'load_video" string. Is there any solution to do this?

PS: What I want to do next is to sort the rows according to the first column, but since it's string type and I found it impossible to use astype to only change the first row into int, what can I do?

These might be some very simple questions, but since I am new to python, your answers would help me a lot. Thanks!

Since the data set is quite huge, I think solution with np.apply_along_axis() would be better. — Vinnton
– Vinnton, Commented Nov 27, 2017 at 3:41
Aside: if you're working with non-numerical data you're probably going to have a much better time using pandas than pure numpy. — DSM
– DSM, Commented Nov 27, 2017 at 3:44
ndarray seems to be an odd choice for this data structure. pandas dataframe is more suitable. — skrubber
– skrubber, Commented Nov 27, 2017 at 3:49
@Vinnton: yep. For tabular data with different column types, using a frame will make life much easier. — DSM
– DSM, Commented Nov 27, 2017 at 4:21

Davis Allen · Accepted Answer · 2017-11-27 07:09:56Z

2

This is the python to remove any entry with 'load_video'.

new_list = [x for x in old_list if 'load_video' not in x]

edited Nov 27, 2017 at 7:09

answered Nov 27, 2017 at 4:12

Davis Allen

701 silver badge9 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Vinnton Over a year ago

But unfortunately I got an error AttributeError: 'numpy.ndarray' object has no attribute 'split' ?

Vinnton Over a year ago

I modify your solution to TrainData = [x for x in TrainData if x[2] != 'load_video'] and it works! Thanks a lot!

Davis Allen Over a year ago

Yes, you'd need to use your own list name instead of old_list, glad it works!!

Collectives™ on Stack Overflow

How to delete a row that contains a specific string in Python using numpy？

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related