Deserializing large JSON Objects from Web Service (Out of Memory)

Question

I have a program that deserializes large objects from a web service. After a webservice call and a 200, the code looks like this.

JsonConvert.DeserializeObject<List<T>>(resp.Content.ReadAsStringAsync().Result).ToList()

Sometimes while running this process I will get an aggregate exception which shows an inner exception as out of memory. I can't determine if it is the process of reading in the string of JSON data (which is probably awfully large) or the Deserializing that is causing this issue. What I would like to do is break out the string and pull each JSON object back individually from the response and then deserialize it. I am just having trouble finding a way to only bring out one JSON object at a time from the response. Any suggestions are greatly appreciated!

Serialization is failing because, and this an example, you try to assign a string to a number. It happen with booleans or any other type — Dalorzo
– Dalorzo, Commented Jan 16, 2014 at 0:31
The serialization only fails sometimes and it throws an out of memory exception. Since it it intermittent with the same JSON data I wouldn't believe this is anything to do with type. — user3170736
– user3170736, Commented Jan 16, 2014 at 0:33

user3170736 · Accepted Answer · 2014-01-16 00:46:29Z

4

HttpClient client = new HttpClient();

using (Stream s = client.GetStreamAsync("http://www.test.com/large.json").Result)
using (StreamReader sr = new StreamReader(s))
using (JsonReader reader = new JsonTextReader(sr))
{
    JsonSerializer serializer = new JsonSerializer();

    // read the json from a stream
    // json size doesn't matter because only a small piece is read at a time from the HTTP request
    Person p = serializer.Deserialize<Person>(reader);
}

answered Jan 16, 2014 at 0:46

user3170736

5415 silver badges25 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

spender Over a year ago

Why use GetStreamAsync(...).Result? Waiting for an uncompleted task result blocks, so you might as well use the synchronous version.

nick_w Over a year ago

You should probably reference where you got the code from as well.

user3170736 Over a year ago

Code came from james.newtonking.com/json/help/index.html?topic=html/…. As for the async versus sync versions the site says to use async.

Akira Yamamoto Over a year ago

The user example returns a List. If the list is too large, this won't help. See this answer: stackoverflow.com/a/24115672/475876

Sjors Miltenburg · Accepted Answer · 2022-05-17 07:52:18Z

0

https://learn.microsoft.com/en-us/xamarin/xamarin-forms/data-cloud/web-services/rest contains a warning:

Using the ReadAsStringAsync method to retrieve a large response can have a negative performance impact. In such circumstances the response should be directly deserialized to avoid having to fully buffer it.

answered May 17, 2022 at 7:52

Sjors Miltenburg

2,8255 gold badges37 silver badges64 bronze badges

Collectives™ on Stack Overflow

Deserializing large JSON Objects from Web Service (Out of Memory)

2 Answers 2

4 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

4 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related