0

When using the map method to apply a custom function in Xarray resample, the dimension coordinates are lost, and the resulting Dataset has a sequence instead of the actual coordinate values.

The following MWE will properly demonstrate the issue:

import numpy as np
import pandas as pd
import xarray as xr

def custom_fn(ds):
    return ds

if __name__=='__main__':
    times=pd.date_range(
            '2000-01-01 00:00', '2000-01-05 23:59:59',
            freq='5min',
            )
    data1=np.arange(len(times))
    data2=data1+1
    data=xr.Dataset(
            data_vars={
                    'data1':(['time',], data1,),
                    'data2':(['time',], data2,),
                    },
            coords={'time':times}
            )

    new=data.resample(time='30min').map(custom_fn)
    print(data)
    print(new)

And the output is:

'''

<xarray.Dataset> Size: 35kB
Dimensions: (time: 1440)
Coordinates:
* time (time) datetime64[ns] 12kB 2000-01-01 ... 2000-01-05T23:55:00
Data variables:
data1 (time) int64 12kB 0 1 2 3 4 5 6 ... 1434 1435 1436 1437 1438 1439
data2 (time) int64 12kB 1 2 3 4 5 6 7 ... 1435 1436 1437 1438 1439 1440

<xarray.Dataset> Size: 23kB
Dimensions: (time: 1440)
Dimensions without coordinates: time
Data variables:
data1 (time) int64 12kB 0 1 2 3 4 5 6 ... 1434 1435 1436 1437 1438 1439
data2 (time) int64 12kB 1 2 3 4 5 6 7 ... 1435 1436 1437 1438 1439 1440

'''

I have kept the custom_fn very simple, notice that the result has no time coordinate values. I used breakpoint inside the custom function if it is not receiving the coordinates, but it is. So, I assume the issue is with how map function (or whatever) is merging the results.

Does anyone know what is happning and how ot fix it?

7
  • It is because custom_fn output still contains the time dimension. Commented Mar 9 at 14:11
  • @cyril It shouldn't? If I'm applying a function on the tim-dimension, it s possible that the output also contain a time-dimension on which the it is concatenated. Commented Mar 9 at 17:49
  • resample is a mapping from a group of time steps to ONE time step. So custom_fn should reflect that. I don't know your use case, but it may be more like a blockwise function you're looking to apply ? Commented Mar 9 at 18:23
  • In your example, what was the output you were expecting by chaining the resampling and the identity fuction ? Did you expect one time step very six ones, or something else ? Commented Mar 9 at 18:26
  • 1
    @cyril The example is a simplified version, but it is similar to what I want to do. The function will do csome calculations using the group data and the result will have the same number of time coordinates as the input. Yes, it sounds like a blockwise function is what I need. Commented Mar 11 at 7:58

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.