Error on using xarray open_mfdataset function

Question

I am trying to combine multiple netCDF files with the same dimensions, their dimensions are as follows:

OrderedDict([(u'lat', <type 'netCDF4._netCDF4.Dimension'>: name = 'lat', size = 720
), (u'lon', <type 'netCDF4._netCDF4.Dimension'>: name = 'lon', size = 1440
), (u'time', <type 'netCDF4._netCDF4.Dimension'>: name = 'time', size = 96
), (u'nv', <type 'netCDF4._netCDF4.Dimension'>: name = 'nv', size = 2
)])
OrderedDict([(u'lat', <type 'netCDF4._netCDF4.Dimension'>: name = 'lat', size = 720
), (u'lon', <type 'netCDF4._netCDF4.Dimension'>: name = 'lon', size = 1440
), (u'time', <type 'netCDF4._netCDF4.Dimension'>: name = 'time', size = 96
), (u'nv', <type 'netCDF4._netCDF4.Dimension'>: name = 'nv', size = 2
)])
OrderedDict([(u'lat', <type 'netCDF4._netCDF4.Dimension'>: name = 'lat', size = 720
), (u'lon', <type 'netCDF4._netCDF4.Dimension'>: name = 'lon', size = 1440
), (u'time', <type 'netCDF4._netCDF4.Dimension'>: name = 'time', size = 96
), (u'nv', <type 'netCDF4._netCDF4.Dimension'>: name = 'nv', size = 2
)])
OrderedDict([(u'lat', <type 'netCDF4._netCDF4.Dimension'>: name = 'lat', size = 720
), (u'lon', <type 'netCDF4._netCDF4.Dimension'>: name = 'lon', size = 1440
), (u'time', <type 'netCDF4._netCDF4.Dimension'>: name = 'time', size = 96
), (u'nv', <type 'netCDF4._netCDF4.Dimension'>: name = 'nv', size = 2
)])
OrderedDict([(u'lat', <type 'netCDF4._netCDF4.Dimension'>: name = 'lat', size = 720
), (u'lon', <type 'netCDF4._netCDF4.Dimension'>: name = 'lon', size = 1440
), (u'time', <type 'netCDF4._netCDF4.Dimension'>: name = 'time', size = 96
), (u'nv', <type 'netCDF4._netCDF4.Dimension'>: name = 'nv', size = 2
)])
OrderedDict([(u'lat', <type 'netCDF4._netCDF4.Dimension'>: name = 'lat', size = 720
), (u'lon', <type 'netCDF4._netCDF4.Dimension'>: name = 'lon', size = 1440
), (u'time', <type 'netCDF4._netCDF4.Dimension'>: name = 'time', size = 96
), (u'nv', <type 'netCDF4._netCDF4.Dimension'>: name = 'nv', size = 2
)])
OrderedDict([(u'lat', <type 'netCDF4._netCDF4.Dimension'>: name = 'lat', size = 720
), (u'lon', <type 'netCDF4._netCDF4.Dimension'>: name = 'lon', size = 1440
), (u'time', <type 'netCDF4._netCDF4.Dimension'>: name = 'time', size = 96
), (u'nv', <type 'netCDF4._netCDF4.Dimension'>: name = 'nv', size = 2
)])

However, on using open_mfdataset, I get this error:

xr.open_mfdataset(path_file, decode_times=False)

*** ValueError: cannot infer dimension to concatenate: supply the ``concat_dim`` argument explicitly

How to fix this error? My dimensions are the same in all the files

shoyer · Accepted Answer · 2016-09-19 01:14:22Z

3

This error message is probably arising because you have two files with the same variables and coordinate values, and xarray doesn't know whether it should stack them together along a new dimension or simply check to make sure none of the values conflict.

It would be nice if explicitly calling open_mfdataset with concat_dim=None disabled all attempts at concatenation. This change should make it into the next release of xarray (v0.9.0).

In the meantime, you can work around this by opening the files individually and merging them explicitly, e.g.,

def open_mfdataset_merge_only(paths, **kwargs):
    if isinstance(paths, basestring):
        paths = sorted(glob(paths))
    return xr.merge([xr.open_dataset(path, **kwargs) for path in paths])

Under the covers, this is basically all that open_mfdataset is doing.

answered Sep 19, 2016 at 1:14

shoyer

9,6831 gold badge41 silver badges57 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

user308827 Over a year ago

thanks @Stephan, will use this. looking forward to v0.9!

hpaulj · Accepted Answer · 2016-09-18 21:23:55Z

1

http://xarray.pydata.org/en/stable/generated/xarray.open_mfdataset.html

xarray.open_mfdataset(paths, chunks=None, concat_dim=None, preprocess=None, engine=None, lock=None, **kwargs)

It looks like it needs you to give a concat_dim parameter. It's having problems inferring it from your data.

Dimension to concatenate files along. This argument is passed on to xarray.auto_combine() along with the dataset objects. You only need to provide this argument if the dimension along which you want to concatenate is not a dimension in the original datasets, e.g., if you want to stack a collection of 2D arrays along a third dimension.

Are these 3d arrays that you want to stack along a new, 4th, dimension?

edited Sep 18, 2016 at 21:23

answered Sep 18, 2016 at 21:21

hpaulj

233k14 gold badges260 silver badges392 bronze badges

2 Comments

user308827 Over a year ago

thanks @hpaulj, I am not sure why it is having a problem. also not sure of format needed for concat_dim. Documentation says it should be string or dataarray, but I have 4 dimensions: lat, loon, time, nv

user308827 Over a year ago

these netCDF files have exactly the same dimensions, but different variables. I want the output netCDF file to contain all the different variables and of course the dimensions

Collectives™ on Stack Overflow

Error on using xarray open_mfdataset function

2 Answers 2

1 Comment

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related