downsample big numpy.ndarray on if condition in python

Question

I have a large numpy.ndarray and I need to downsample this array based on the value of one column. My solution works, but is very slow

data_table = data_table[[i for i in range(0, len(data_table)) if data_table[i][7] > 0.2 and data_table[i][7] < 0.75]]

does anybody know what the fastest way is to do this?

Divakar · Accepted Answer · 2017-09-27 22:34:45Z

2

Use column-slicing to select relevant columns and compare those against the thresholds in a vectorized manner to give us a mask of valid rows and then index into the rows for the rows filtered output -

out = data_table[(data_table[:,7] > 0.2) & (data_table[:,7] < 0.75)]

answered Sep 27, 2017 at 22:34

Divakar

222k19 gold badges273 silver badges374 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

downsample big numpy.ndarray on if condition in python

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related