Memory error reading a zip file in python

Question

I have a thread in which I am reading a zip file with zipfile.ZipFile().read(), where I am getting a memory error.

I am aware that read() loads the entire file into memory. The size of file after unzipping is more than 100MB. I also tried with zipfile.ZipFile().open().readlines(), but it takes too much time.

Is there any way that I can read the file with speed without getting memory error?

readlines() with no sizehint argument also reads the entire file into memory and builds a list of the lines. So it wouldn't reduce memory requirements, but rather increase them slightly. See Aya's answer. — msw
– msw, Commented Jun 20, 2013 at 19:29

Aya · Accepted Answer · 2013-06-20 18:11:20Z

7

Assuming you're trying to read a zipped text file, you can treat the file-like object returned by ZipFile.open() as an iterator, and process it line-by-line...

from zipfile import ZipFile

zip = ZipFile('myzip.zip')
stream = zip.open('myfile.txt')
for line in stream:
    do_something_with(line)

answered Jun 20, 2013 at 18:11

Aya

42.5k6 gold badges60 silver badges56 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Memory error reading a zip file in python

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related