Python decompress gzip data in memory without file

Question

I have gzipped data from HTTP reply. I have following code:

def gzipDecode(self, content):
    import StringIO
    import gzip

    outFilePath = 'test'

    compressedFile = StringIO.StringIO(content)
    decompressedFile = gzip.GzipFile(fileobj=compressedFile)
    with open(outFilePath, 'w') as outfile:
        outfile.write(decompressedFile.read())

    data = ''
    with open(outFilePath, 'r') as myfile:
        data=myfile.read().replace('\n', '')

    return data

which decompress input gzipped content and return string (http reply is gzipped json). - It works.

But I need it without creating test file - all in memory.

I modified it to:

def gzipDecode(self, content):
    import StringIO
    from io import BytesIO
    import gzip

    outFile = StringIO.StringIO()

    compressedFile = StringIO.StringIO(content)
    decompressedFile = gzip.GzipFile(fileobj=compressedFile)

    outFile.write(decompressedFile.read())
    outFile.flush()

    data = outFile.read().replace('\n', '')
    print "_" + data + "_"
    return data

but it crash (gzipDecode produce empty output) in parsing json:

Traceback (most recent call last):
__
  File "/usr/lib/python2.7/SocketServer.py", line 593, in process_request_thread

    self.finish_request(request, client_address)
----------------------------------------
  File "/usr/lib/python2.7/SocketServer.py", line 334, in finish_request
Exception happened during processing of request from ('10.123.66.3', 39853)
    self.RequestHandlerClass(request, client_address, self)
----------------------------------------
  File "/usr/lib/python2.7/SocketServer.py", line 649, in __init__
    self.handle()
  File "/usr/lib/python2.7/BaseHTTPServer.py", line 340, in handle
    self.handle_one_request()
  File "/usr/lib/python2.7/BaseHTTPServer.py", line 328, in handle_one_request
    method()
  File "/tmp/test_server.py", line 92, in do_POST
    data = json.loads(file_content)
  File "/usr/lib/python2.7/json/__init__.py", line 338, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python2.7/json/decoder.py", line 366, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python2.7/json/decoder.py", line 384, in raw_decode
    raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded

What I do bad?

but it crash in parsing json. How does it 'crash'? Can you please supply some sample input, expected output and actual output (including the complete traceback if there is one)? — Martijn Pieters
– Martijn Pieters, Commented Mar 29, 2016 at 10:47

Martijn Pieters · Accepted Answer · 2016-03-29 11:00:32Z

5

You need to seek back to the start before you can read:

outFile.write(decompressedFile.read())
outFile.flush()
outFile.seek(0)

data = outFile.read().replace('\n', '')

answered Mar 29, 2016 at 11:00

Martijn Pieters

1.1m326 gold badges4.2k silver badges3.4k bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Python decompress gzip data in memory without file

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related