OutOfMemoryError onhandling of large (JSON) strings [duplicate]

Question

Possible Duplicate:
Out of memory error when putting large JSON (InputStream) to String

I'm getting an encoded JSON String via an HttpResponse. Code looks like this:

    BasicResponseHandler brh = new BasicResponseHandler(); 
    String responseString = brh.handleResponse(response); 
    String decoded = new String(Base64.decode(responseString, Base64.NO_CLOSE));

I get an error like this:

09-03 12:57:09.664: E/AndroidRuntime(8309): FATAL EXCEPTION: Thread-272
09-03 12:57:09.664: E/AndroidRuntime(8309): java.lang.OutOfMemoryError
09-03 12:57:09.664: E/AndroidRuntime(8309):     at java.nio.CharArrayBuffer.<init>(CharArrayBuffer.java:43)
09-03 12:57:09.664: E/AndroidRuntime(8309):     at java.nio.ReadWriteCharArrayBuffer.<init>(ReadWriteCharArrayBuffer.java:47)
09-03 12:57:09.664: E/AndroidRuntime(8309):     at java.nio.CharBuffer.allocate(CharBuffer.java:54)
09-03 12:57:09.664: E/AndroidRuntime(8309):     at java.nio.charset.CharsetDecoder.allocateMore(CharsetDecoder.java:236)
09-03 12:57:09.664: E/AndroidRuntime(8309):     at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:195)
09-03 12:57:09.664: E/AndroidRuntime(8309):     at java.nio.charset.Charset.decode(Charset.java:487)
09-03 12:57:09.664: E/AndroidRuntime(8309):     at java.lang.String.<init>(String.java:174)
09-03 12:57:09.664: E/AndroidRuntime(8309):     at java.lang.String.<init>(String.java:141)
09-03 12:57:09.664: E/AndroidRuntime(8309):     at xxx.DataManager.doPost(DataManager.java:448)
09-03 12:57:09.664: E/AndroidRuntime(8309):     at xxx.DataManager.run(DataManager.java:185)

Sometimes I get it on

brh.handleResponse(response);

sometimes on

String decoded = new String(Base64.decode(responseString, Base64.NO_CLOSE));

So I take it the String is just too big to handle (I dont know how to determine it's size in bytes tbh), how can I work around this?

Do you also control the incoming string? If so, try to split up the requests in smaller json answers. — Tschallacka
– Tschallacka, Commented Sep 3, 2012 at 13:06
is whole response encoded with Base64? if not you can also use pull parser (JsonReader class on API > 11 or fx jackson parser) — Selvin
– Selvin, Commented Sep 3, 2012 at 13:09

Codo · Accepted Answer · 2012-09-03 13:18:45Z

You should switch to a streaming approach where only the final JSON data structure is completely held in memory. The other pieces, in particular the JSON encoded HTTP response, are processed chunk by chunk. So they will only require a minimal amount of memory.

That way you have several advantages:

The overall memory consumption is considerably reduced.
The number of large allocations (as for a large string) is considerably reduced. (Sometimes there is sufficient memory available but no single sufficiently large piece the allocate a big string.)
Since the parsing can start as soon as the first bytes from the answer arrive, the parsing can be done parallel to the data reception. Therefore, the app becomes quicker.

The org.json.JSONObject does not support streaming. You'll have to use a different class instead.

BTW: I don't see any reason why one would Base 64 encode JSON data. JSON is a text based format while Base 64 encoding is required if binary data is invovled.

Collectives™ on Stack Overflow

OutOfMemoryError onhandling of large (JSON) strings [duplicate]

1 Answer 1

Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Linked

Related