Select DataFrame row by time range

Question

I need to select the rows +- 1sec from specific lines (marked as to_pick)

I can do it with loops, but I looking for more elegant way

import pandas as pd
import numpy as np
N = 1000
tidx = pd.date_range('2019-07-01 09:30:00', periods=N, freq='S')
np.random.seed(3)
data = np.random.randn(N)
ts = tidx + pd.to_timedelta(pd.np.random.randn(N), unit='s')

df = pd.DataFrame({'Time': ts, 'to_pick': pd.np.random.randn(N) > 0.98, 'start': ts - pd.to_timedelta(1, 's'), 'end': ts + pd.to_timedelta(1, 's')})
df.loc[~df['to_pick'], 'start'] = pd.np.nan
df.loc[~df['to_pick'], 'end'] = pd.np.nan

I am looking for something like

 df['Time'].between(df['start', df['end'])

that will work to combine the conditions

This might help stackoverflow.com/questions/29370057/…

Casper
– Casper

2019-08-29 04:47:44 +00:00
Commented Aug 29, 2019 at 4:47 — Casper
– Casper, Commented Aug 29, 2019 at 4:47
what is df[start] and df[end]

min2bro
– min2bro

2019-08-29 06:07:57 +00:00
Commented Aug 29, 2019 at 6:07 — min2bro
– min2bro, Commented Aug 29, 2019 at 6:07

Code Different · Accepted Answer · 2019-08-29 19:16:59Z

1

If I understand you correctly, you want to select rows whose Time are between start and end when to_pick = true:

start = df.query('to_pick')['start'].values
end = df.query('to_pick')['end'].values
t = df['Time'].values[:, None]

df.loc[np.any((start <= t) & (t <= end), axis=1)]

answered Aug 29, 2019 at 19:16

Code Different

93.4k16 gold badges154 silver badges175 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

user1889297 Over a year ago

this is great! I need to understand how these line works t = df['Time'].values[:, None] (start <= t)

Code Different Over a year ago

It's using numpy array broadcast. Let me know if you want me to elaborate

Collectives™ on Stack Overflow

Select DataFrame row by time range

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related