How to deal with array of variable size while creating a HDF5 Dataset in Python?

Question

How to create a HDF5 dataset when size of one dimension of a multidimensional array is not fixed. I tried the following toy code, but it seems that I am missing some point here.

import numpy as np

import h5py


Polyline=h5py.special_dtype(vlen=np.float32)

f=h5py.File('dataset.hdf5', mode='w')

var_features=f.create_dataset('var_features', (10,), dtype=Polyline )

features = np.empty(shape=(10,), dtype=Polyline)

for i in range(10):

    a=10+i*2
    features[i]=np.arange(a).reshape(a/2,2) 

var_features[...]=features

print features[0].shape 

print var_features[0].shape

kakk11 · Accepted Answer · 2016-03-27 17:35:01Z

1

It's quite simple, just create dataset with maxsize attribute with one or more None values.

Something like this:

import h5py
import numpy as np

fff = h5py.File('test1.h5','w')
fff.create_dataset('test_resize',(100,100),maxshape=(None,None),chunks=(10,10))
fff['test_resize'][:] = np.random.random((100,100))
fff.flush()
fff['test_resize'].resize((150,100))
fff['test_resize'][100:150,:] = np.ones((50,100))
fff.close()

answered Mar 27, 2016 at 17:35

kakk11

9339 silver badges21 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

How to deal with array of variable size while creating a HDF5 Dataset in Python?

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related