Zip code not working - memory error

Question

Can someone point out any errors I might have included in my code. I wanted to keep the code as simple as possible but it's failing with a memory error. Some of the compressed files do extract fine though...

import zipfile
import from os.path isdir, join, normpath, split
print "Unzipping data"
z = zipfile.ZipFile("C:\\Incoming\\MyZipFile.zip", 'r')
print z.namelist()
for each in z.namelist():
   if not each.endswith('/'):
      root, name = split(each)
      print name
      file(join("C:\\Incoming\\", name), 'wb').write(z.read(each))
z.close()

The actual error message generated is as follows:

Traceback (most recent call last):
File "C:\\Scripts\\Zip_import_test.py", line 30, in <module>
   file(join("C:\\Incoming\\", name), 'wb').write(zip.read(each))
File "C:\\Python25\lib\zipfile.py, line 501, in read
   bytes = dc.decompress(bytes)
Memory Error

Thanks for any suggestions. Frank Ogiamien

what is the size of the zip objects you're trying to unzip ? — Cédric Julien
– Cédric Julien, Commented Jul 27, 2011 at 18:41
The .zip files I'm hoping to extract from range in size up to 500meg. My test example is 73meg. — User Error
– User Error, Commented Jul 27, 2011 at 21:52

Community · Accepted Answer · 2017-05-23 12:26:52Z

5

Don't call it zip! You're masking the builtin.

Also, is zip.close() really inside your for loop? It shouldn't be.

You should use the extract method of the ZipFile object, so you don't need to read the whole file into memory.

Instead of

file(join("C:\\Incoming\\", name), 'wb').write(zip.read(each))

do

zip.extract(each, "C:\\Incoming\\")

Edit: This was added in 2.6, as was extractall if you just want to extract the whole thing to a directory.

If you can't upgrade, the code in How to simulate ZipFile.open in Python 2.5? (in the question, not an answer) will let you use parse the zipfile and extract the data using zlib without reading it in to memory.

edited May 23, 2017 at 12:26

CommunityBot

11 silver badge

answered Jul 27, 2011 at 18:52

agf

178k45 gold badges300 silver badges241 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

User Error Over a year ago

Hi, I changed my "zip" to "z" and tried your suggestion of using z.extract(each, "C:\\Incoming\\") however I get a new error: AttributeError: ZipFile instance has no attribute 'extract'" Is this because my version 2.5 of py is too old? Oh, my zip.close() wasn't really inside the for loop. typo.

User Error Over a year ago

agf, thanks very much for your time. Your comments were very helpful. It looks like I'm better off upgrading to 2.6 if I want to keep the code short and simple. Frank

Collectives™ on Stack Overflow

Zip code not working - memory error

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related