pandas reindex multiindex and shift values by second index

Question

I have a pandas DataFrame looking like this :

                        x1             x2            x3            x4
Date       Time                                                               
2017-01-03 09:00:00      0.000097      0.000259      0.000629      0.000142   
           09:20:00      0.000046      0.000044      0.000247      0.000134   
           09:40:00      0.000021      0.000032      0.000171      0.000105   
           10:00:00      0.000033      0.000040      0.000136      0.000178   
           10:20:00      0.000079      0.000157      0.000094      0.000083
           .....
           17:00:00      0.000032      0.000137      0.000024      0.000028

However, i want to reindex the second index, by one 20min bin and I would like it to look like this:

                        x1             x2            x3            x4
Date       Time                                                               
2017-01-03 09:20:00      0.000097      0.000259      0.000629      0.000142   
           09:40:00      0.000046      0.000044      0.000247      0.000134   
           10:00:00      0.000021      0.000032      0.000171      0.000105   
           10:20:00      0.000033      0.000040      0.000136      0.000178   
           10:40:00      0.000079      0.000157      0.000094      0.000083
           .....
           17:20:00      0.000032      0.000137      0.000024      0.000028

So all the values stay the same, only the second index is renamed, everything else stays the same.

I've tried following code:

x.reindex(pd.date_range(pd.Timestamp('09:20:00'), pd.Timestamp('17:20:00'), freq="20min").time, level=1)

But it just moves the index and the values stay at the same place.

                        x1             x2            x3            x4
Date       Time                                                               
2017-01-03 09:20:00      0.000046      0.000044      0.000247      0.000134   
           09:40:00      0.000021      0.000032      0.000171      0.000105   
           10:00:00      0.000033      0.000040      0.000136      0.000178   
           10:20:00      0.000079      0.000157      0.000094      0.000083
           .....
           17:00:00      0.000032      0.000137      0.000024      0.000028

It does not even ad the bin for 17:20:00.

However, if I also tried to shift the values after grouping them like this:

x.groupby(level=1).shift(1)

or:

x.groupby(level=1).shift(1, freq='20min')

but that did not work at all.

Peter Leimbigler · Accepted Answer · 2018-06-15 00:40:17Z

The fastest way I can think of is to overwrite the entire first level (innermost level) of the MultiIndex with a 20-minute-shifted version of itself:

x.index = x.index.set_levels(x.index.levels[1].shift(20, 'min'), level=1)

Example

x = pd.DataFrame(index=pd.MultiIndex.from_product([pd.date_range('2017-01-03', '2017-01-06', freq='1D'), 
                                                   pd.date_range('09:00', '17:00', freq='20min')]))
x.loc[:, 'x1'] = list(range(len(x)))

x
                                x1
2017-01-03 2018-06-14 09:00:00   0
           2018-06-14 09:20:00   1
           2018-06-14 09:40:00   2
           2018-06-14 10:00:00   3
           2018-06-14 10:20:00   4
    ...                         ..
2017-01-06 2018-06-14 15:40:00  95
           2018-06-14 16:00:00  96
           2018-06-14 16:20:00  97
           2018-06-14 16:40:00  98
           2018-06-14 17:00:00  99

x.index = x.index.set_levels(x.index.levels[1].shift(20, 'min'), level=1)

x
                                x1
2017-01-03 2018-06-14 09:20:00   0
           2018-06-14 09:40:00   1
           2018-06-14 10:00:00   2
           2018-06-14 10:20:00   3
           2018-06-14 10:40:00   4
    ...                         ..
2017-01-06 2018-06-14 16:00:00  95
           2018-06-14 16:20:00  96
           2018-06-14 16:40:00  97
           2018-06-14 17:00:00  98
           2018-06-14 17:20:00  99

Collectives™ on Stack Overflow

pandas reindex multiindex and shift values by second index

1 Answer 1

Example

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Example

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related