Newtonsoft JSON.NET Deserialization error

Question

I am have a bunch of long json output in individual files. I need to read these files and deserialize them into the entities that originally generated the json (I have access to the original entities). Each file has the json output that was generated by serializing an object of type IEnumerable<Response<MyEntity>>.

I am getting an exception when attempting to deserialize, but I don't understand why. Here is my attempt to deserialize:

List<string> jsonOutputStrings;
// Read json from files and populate jsonOutputStrings list
List<Response<MyEntity>> apiResponses = new List<Response<MyEntity>>();

foreach (string json in jsonOutputStrings)
{
    apiResponses.AddRange(JsonConvert.DeserializeObject<List<Response<MyEntity>>>(json, new JsonSerializerSettings { TypeNameHandling = TypeNameHandling.All }).ToList());
}

I also tried deserializing to an IEnumerable instead of a list:

apiResponses.AddRange(JsonConvert.DeserializeObject<IEnumerable<Response<MyEntity>>>(json, new JsonSerializerSettings { TypeNameHandling = TypeNameHandling.All }).ToList());

I get the following exception:

A first chance exception of type 'Newtonsoft.Json.JsonSerializationException' occurred in Newtonsoft.Json.dll

Additional information: Cannot create and populate list type System.Linq.Enumerable+WhereSelectListIterator`2[Entities.Requirement,Entities.RequirementEntity]. Path '$values[0].ReturnedEntity.Summaries.$values[0].Requirements.$values', line 1, position 715.

The entire json is too long to post (and also contains some confidential data) but I did find a place in the json that has this:

"Requirements":{"$id":"7","$type":"System.Linq.Enumerable+WhereSelectListIterator`2[[Entities.Requirement, Entities],[Entities.RequirementEntity, API.Entities]], System.Core","$values":[...]}

In the entity that I'm trying to deserialize into (same one that was originally serialized), Requirements is of type IEnumerable<Entities.RequirementEntity>.

It doesn't make sense to me how serialization from MyEntity works but deserializing to the same type doesn't. How do I solve this problem?

Did you try deserializing into an IEnumerable<ApiResponse<StudentReportApiEntity>>> instead? — Andrew Whitaker
– Andrew Whitaker, Commented Dec 17, 2014 at 21:06
Yes, that was what I did first. I still get the same error. It seems to have an issue with the IEnumerable<Entities.RequirementEntity> property that's nested several levels into the parent entity, but I don't know why... — mayabelle
– mayabelle, Commented Dec 17, 2014 at 21:09
This may be a silly question, but what is the namespace of the List type you are deserializing into? System.Collections...? — danludwig
– danludwig, Commented Dec 17, 2014 at 21:11
If you exclude the type from the serialized result, does it work for you then? (check for the TypeNameHandling setting) — Andrew Whitaker
– Andrew Whitaker, Commented Dec 17, 2014 at 21:25
@AndrewWhitaker - thank you! It worked when I set TypeNameHandling to None when deserializing. — mayabelle
– mayabelle, Commented Dec 17, 2014 at 21:39

Krzysztof · Accepted Answer · 2014-12-18 08:16:01Z

6

You must browse through Response<> and MyEntity and see how collections are initialized. This tells us that one of the collections in some class is created using Where method from linq. You can reproduce this error by executing this code:

class MyEntity
{
    public MyEntity()
    {
        Data = new List<string>().Where(x => true);
    }

    public IEnumerable<string> Data { get; set; }

}

class Program
{
    static void Main(string[] args)
    {
        string data = @"[{""Data"":[""a"",""b""]}]";
        var j = JsonConvert.DeserializeObject<IEnumerable<MyEntity>>(data);
    }
}

Another possibility is to have metadata in json. Then you have 2 solutions:

Set TypeNameHandling to TypeNameHandling.None (as somebody mentioned in comment);
Replace not working types in string with working ones

Using TypeNameHandling.None may lead to wrong deserialization for exmaple when you have IEnumerable<BaseType> and that list contains subtype of BaseType.

In that case you should go for second option. Basically you should replace any type that is not deserializing and replace it for example with List.

Sample code:

class MyEntity
{
    public IEnumerable<string> Data { get; set; }
}

class Program
{
    static void Main(string[] args)
    {
        IList<MyEntity> entities = new MyEntity[] {
            new MyEntity { Data = new [] { "1", "2" }.Where(x => x != string.Empty) },
            new MyEntity { Data = new [] { "A", "B" }.AsQueryable().Where(x => x != string.Empty) },
            new MyEntity { Data = new List<string> { "A", "B" } },
        };

        string data = JsonConvert.SerializeObject(entities, Formatting.Indented, new JsonSerializerSettings { TypeNameHandling = TypeNameHandling.All });
        data = Regex.Replace(data, "\"\\$type\":\\s+\"System.Linq.Enumerable\\+WhereArrayIterator(.+?), System.Core\",", "\"$type\": \"System.Collections.Generic.List$1, mscorlib\",", RegexOptions.Singleline);

        var j = JsonConvert.DeserializeObject<IEnumerable<MyEntity>>(data, new JsonSerializerSettings { TypeNameHandling = TypeNameHandling.All });
    }
}

edited Dec 18, 2014 at 8:16

answered Dec 17, 2014 at 21:20

Krzysztof

16.2k2 gold badges52 silver badges79 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

mayabelle Over a year ago

Thank you, this is helpful. Most likely this is what's happening, and I will have to look for it to prevent this from happening in the future. The problem that I have now is that we already have about 1000 of these json files containing many months of data, and we don't have a way to regenerate them. Is there some way I can manipulate the json strings / json files to make them look like the collection wasn't created with linq as you describe, so that I can deserialize them?

Krzysztof Over a year ago

This problem is not related with json itself, but with .net classes. While deserializing collection, deserializer tries to add element to it. If collection is read only (eg array or linq result) then exception is thrown. You should search in your code for collection initializers. For example if you remove 'where' from my code, it'll work.

mayabelle Over a year ago

In my case I did not have a collection initializer on the list causing the issue, but linq was used to populate the IEnumerable when the json was originally created and serialized (not as part of the class, just using a setter).

Krzysztof Over a year ago

I see now that you have metadata in that, and that causes issues. You may also instead of setting TypeNameHandling to None, replace unsupported types names in string to supported ones. I've added sample code in my answer.

John · Accepted Answer · 2014-12-17 21:21:24Z

2

Is there a Linq result in the serialized entity? Maybe defined as an IEnumerable and filled with a Linq Where result? It seems that's where your problem lies. You should convert it to a List or Array before serializing.

answered Dec 17, 2014 at 21:21

John

3,7121 gold badge14 silver badges13 bronze badges

1 Comment

mayabelle Over a year ago

Thank you, this is helpful. Most likely this is what's happening, and I will have to look for it to prevent this from happening in the future. The problem that I have now is that we already have about 1000 of these json files containing many months of data, and we don't have a way to regenerate them. Is there some way I can manipulate the json strings / json files to make them look like the collection wasn't created with linq as you describe, so that I can deserialize them?

Collectives™ on Stack Overflow

Newtonsoft JSON.NET Deserialization error

2 Answers 2

4 Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

4 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related