How to load multiple .mat files into a python script

Question

I want to load 38 .mat files into a dictionary to hold them all. the .mat files are named subject1 to subject38 The code I tried is a simple for loop

    import scipy.io as sio
    data = {}
    for i in range(1, 38):
        data["data{}".format(i)] = sio.loadmat('subject{}.mat'.format(i))

the error I'm getting is:

Traceback (most recent call last): File "D:/senior project/python/dataAqu.py", line 7, in data["data{0}".format(i)] = sio.loadmat('subject{0}.mat'.format(i)) File "C:\Users\mamdo\AppData\Roaming\Python\Python27\site-packages\scipy\io\matlab\mio.py", line 208, in loadmat matfile_dict = MR.get_variables(variable_names) File "C:\Users\mamdo\AppData\Roaming\Python\Python27\site-packages\scipy\io\matlab\mio5.py", line 292, in get_variables res = self.read_var_array(hdr, process) File "C:\Users\mamdo\AppData\Roaming\Python\Python27\site-packages\scipy\io\matlab\mio5.py", line 252, in read_var_array return self._matrix_reader.array_from_header(header, process) File "mio5_utils.pyx", line 675, in scipy.io.matlab.mio5_utils.VarReader5.array_from_header File "mio5_utils.pyx", line 705, in scipy.io.matlab.mio5_utils.VarReader5.array_from_header File "mio5_utils.pyx", line 778, in scipy.io.matlab.mio5_utils.VarReader5.read_real_complex File "mio5_utils.pyx", line 450, in scipy.io.matlab.mio5_utils.VarReader5.read_numeric File "mio5_utils.pyx", line 355, in scipy.io.matlab.mio5_utils.VarReader5.read_element File "streams.pyx", line 194, in scipy.io.matlab.streams.ZlibInputStream.read_string File "pyalloc.pxd", line 9, in scipy.io.matlab.pyalloc.pyalloc_v MemoryError

A comment on the loop: to load the matrices with suffix 1...38, the loop should be for i in range(1,39):. Regarding the error message: how big are the .mat files? — Pete
– Pete, Commented Oct 5, 2019 at 21:41
Each .mat file is approximately 250 MB. I restarted my pc and this error stopped occuring but now because of the file size the python script runs into a memory problem. — jo Mamdouh
– jo Mamdouh, Commented Oct 5, 2019 at 22:08
Why must you load them all at the same time? Can you loop through them one at a time? — cwalvoort
– cwalvoort, Commented Oct 5, 2019 at 22:13
I changed the code to load the mat file inside the loop so the mat files are not loaded all at once and it worked. Thanks — jo Mamdouh
– jo Mamdouh, Commented Oct 5, 2019 at 22:44

jo Mamdouh · Accepted Answer · 2019-10-05 22:46:40Z

1

So I found the problem. The mat files shouldnt be opened by any other program - like matlab - if there is an error restart the computer.

Also if there is a memory problem try to integrate the mat files seperatly and perform whatever code you need and then load the next file.

answered Oct 5, 2019 at 22:46

jo Mamdouh

214 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

How to load multiple .mat files into a python script

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related