Is it possible to create multiple files in memory, and then zip them up and write them to disk?

Question

Is it possible to create files in memory, and arrange them in a type of hierarchy before writing them to disk?

Can an open statement be redirected into some kind of in-memory representation?

My current technique for creating the zipped directories is this:

Write everything in memory to a temporary folder
Create a zipfile object
reload all of the previously made files
Add them to the zip and save
Delete all the temporary files.

Ultimately ending up with something like this:

Zipped_root 
     |
     |
     |---- file1.txt
     |
     |---- Image1.png
     |
     |---- Image2.png
     |
     |---- file...N.txt
     |

Is there a way to do this all in memory?

Have you read the ZipFile docs? There's a ZipFile.writestr() method that allows passing data directly. — Tim Peters
– Tim Peters, Commented Sep 27, 2013 at 2:13
@TimPeters I have. It works well for texts files. However, I was hoping to find a way to write images as well. — Zack Yoshyaro
– Zack Yoshyaro, Commented Sep 27, 2013 at 2:20
Have you looked at the StringIO library? It sounds like something that will be helpful. — SethMMorton
– SethMMorton, Commented Sep 27, 2013 at 2:20
Data is data - there's no difference, at the bit level, between text files and image files. What problem(s) did you have? — Tim Peters
– Tim Peters, Commented Sep 27, 2013 at 2:23
@TimPeters It was throwing an error when I tried to pass it the PIL Image object. However! Using SethMMorton's suggestion from above, I was able to use the PIL object with the StingIO module, and then pass that to the writestr function of the zipfile library. Works like a charm now! Thanks! — Zack Yoshyaro
– Zack Yoshyaro, Commented Sep 27, 2013 at 2:28

Robert Kajic · Accepted Answer · 2013-09-27 02:23:35Z

A while ago I implemented a small python library (https://github.com/kajic/vdir) for creating virtual directories, files and even compressing them if need be. From the README (the virtual directory is compressed at the end):

from vdir import VDir

vd = VDir()

# Write to file
vd.open("path/to/some/file").write("your data")

# Create directory, go inside it, and write to some other file
vd.mkdir("foo")
vd.cd("foo")
vd.open("bar").write("something else") # writes to /foo/bar

# Read from file
vd.open("bar").read()

# Get the current path
vd.pwd()

# Copy directory and all its contents
vd.cp("/foo", "/foo_copy")

# Move the copied directory somewhere else
vd.mv("/foo_copy", "/foo_moved")

# Create a file, then remove it
vd.open("unnecessary").write("foo")
vd.rm("unnecessary")

# Walk over all directories and files in the virtual directory
vd.cd("/")
for base, dirnames, dirs, filenames, files in vd.walk():
  pass

# Recursively list directory contents
vd.ls()

# Create a zip from the virtual directory
zip = vd.compress()

# Get zip data
zip.read()

I just did it for fun and haven't tested it extensively, but perhaps it can be of use to you anyway.

Craig · Accepted Answer · 2013-09-27 02:10:39Z

-1

Yes. Take a look at the zlib module documentation on compressing objects. There is also an archiving module that you can used to create the archive object to be compressed. You can access the documentation at any time:

$ python
>>> import zlib
>>> help(zlib)

answered Sep 27, 2013 at 2:10

Craig

4,4194 gold badges40 silver badges57 bronze badges

Collectives™ on Stack Overflow

Is it possible to create multiple files in memory, and then zip them up and write them to disk?

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related