dotnet core System.Text.Json unescape unicode string

Question

Using JsonSerializer.Serialize(obj) will produce an escaped string, but I want the unescaped version. For example:

using System;
using System.Text.Json;

public class Program
{
    public static void Main()
    {
        var a = new A{Name = "你好"};
        var s = JsonSerializer.Serialize(a);
        Console.WriteLine(s);
    }
}

class A {
    public string Name {get; set;}
}

will produce a string {"Name":"\u4F60\u597D"} but I want {"Name":"你好"}

I created a code snippet at https://dotnetfiddle.net/w73vnO
Please help me.

Aside from making the data less readable, the default escaping also bloats the size of the json by 40 percent. And that is a significant change when you are caching or sending large json payloads. — Yogi
– Yogi, Commented Dec 19, 2021 at 7:46

micka190 · Accepted Answer · 2024-03-13 10:15:25Z

146

You need to set the JsonSerializer options not to encode those strings.

JsonSerializerOptions jso = new JsonSerializerOptions();
jso.Encoder = System.Text.Encodings.Web.JavaScriptEncoder.UnsafeRelaxedJsonEscaping;

Note that using UnsafeRelaxedJsonEscaping comes with security risks such as XSS attacks.

Then you pass this options when you call your Serialize method.

var s = JsonSerializer.Serialize(a, jso);

Full code:

JsonSerializerOptions jso = new JsonSerializerOptions();
jso.Encoder = System.Text.Encodings.Web.JavaScriptEncoder.UnsafeRelaxedJsonEscaping;

var a = new A { Name = "你好" };
var s = JsonSerializer.Serialize(a, jso);        
Console.WriteLine(s);

Result:

If you need to print the result in the console, you may need to install additional language. Please refer here.

edited Mar 13, 2024 at 10:15

micka190

1,0132 gold badges12 silver badges32 bronze badges

answered Sep 19, 2019 at 3:38

rcs

7,32713 gold badges61 silver badges88 bronze badges

Sign up to request clarification or add additional context in comments.

9 Comments

arkod Over a year ago

I could not believe my eyes when I found this: learn.microsoft.com/en-us/dotnet/api/… This is extremely surprising behavior by the default encoder.

ahsonkhan Over a year ago

It's important to understand the potential concerns with using this in your scenario and I would recommend safer alternatives if feasible. See learn.microsoft.com/en-us/dotnet/standard/serialization/…

gregsdennis Over a year ago

Those docs never mention why they avoid serializing those. Why was the decision made to encode everything when characters like the double-quote " and control chars have specific escape sequences for them?!

Karrde Over a year ago

Using an "unsafe" encoding is not the answer, the answer from ahsonkhan is correct

micka190 Over a year ago

Be careful with this since it can result in XSS attacks! See @ahsonkhan's answer instead!

|

ahsonkhan · Accepted Answer · 2019-12-10 04:24:17Z

67

To change the escaping behavior of the JsonSerializer you can pass in a custom JavascriptEncoder to the JsonSerializer by setting the Encoder property on the JsonSerializerOptions.

https://learn.microsoft.com/en-us/dotnet/api/system.text.json.jsonserializeroptions.encoder?view=netcore-3.0#System_Text_Json_JsonSerializerOptions_Encoder

The default behavior is designed with security in mind and the JsonSerializer over-escapes for defense-in-depth.

If all you are looking for is escaping certain "alphanumeric" characters of a specific non-latin language, I would recommend that you instead create a JavascriptEncoder using the Create factory method rather than using the UnsafeRelaxedJsonEscaping encoder.

JsonSerializerOptions options = new JsonSerializerOptions
{
    Encoder = JavaScriptEncoder.Create(UnicodeRanges.BasicLatin, UnicodeRanges.CjkUnifiedIdeographs)
};

var a = new A { Name = "你好" };
var s = JsonSerializer.Serialize(a, options);
Console.WriteLine(s);

Doing so keeps certain safe-guards, for instance, HTML-sensitive characters will continue to be escaped.

I would caution against using System.Text.Encodings.Web.JavaScriptEncoder.UnsafeRelaxedJsonEscaping flippantly since it does minimal escaping (which is why it has "unsafe" in the name). If the JSON you are creating is written to a UTF-8 encoded file on disk or if its part of web request which explicitly sets the charset to utf-8 (and is not going to potentially be embedded within an HTML component as is), then it is probably OK to use this.

See the remarks section within the API docs: https://learn.microsoft.com/en-us/dotnet/api/system.text.encodings.web.javascriptencoder.unsaferelaxedjsonescaping?view=netcore-3.0#remarks

You could also consider specifying UnicodeRanges.All if you expect/need all languages to remain un-escaped. This still escapes certain ASCII characters that are prone to security vulnerabilities.

JsonSerializerOptions options = new JsonSerializerOptions
{
    Encoder = JavaScriptEncoder.Create(UnicodeRanges.All)
};

For more information and code samples, see: https://learn.microsoft.com/en-us/dotnet/standard/serialization/system-text-json-how-to?view=netcore-3.0#customize-character-encoding

See the Caution Note

answered Dec 10, 2019 at 4:24

ahsonkhan

2,6472 gold badges13 silver badges16 bronze badges

4 Comments

Ruben Bartelink Over a year ago

@joey I know this came later but it should become the accepted answer

ahsonkhan Over a year ago

Here's the updated doc page link: learn.microsoft.com/en-us/dotnet/standard/serialization/…

imsan Over a year ago

"If the JSON you are creating is written to a UTF-8 encoded file on disk or if its part of web request which explicitly sets the charset to utf-8 (and is not going to potentially be embedded within an HTML component as is), then it is probably OK to use this. [UnsafeRelaxedJsonEscaping]" to highlight this part

Luke Vo Over a year ago

So if I understand correctly, if I do not use the JSON for HTML (I still don't understand, you are supposed to sanitize it before putting into HTML if it's your use case anyway), I can safely use UnsafeRelaxedJsonEscaping?

Cyrus · Accepted Answer · 2021-10-28 10:29:06Z

27

Use:

JsonSerializerOptions options = new JsonSerializerOptions
{
    Encoder = JavaScriptEncoder.Create(UnicodeRanges.All)
};

answered Oct 28, 2021 at 10:29

Cyrus

2,4793 gold badges26 silver badges39 bronze badges

2 Comments

mkb Over a year ago

that's the way, thx

Diwakar Padmaraja Over a year ago

This worked perfectly for all Unicode character ranges. This is should have been the highest rank answer. Thank you.

Steven Peirce · Accepted Answer · 2021-06-11 11:24:49Z

12

You can use: System.Text.RegularExpressions.Regex.Unescape(string) to unescape the unicode characters. https://learn.microsoft.com/en-us/dotnet/api/system.text.regularexpressions.regex.unescape

Updating example from original question:

using System;
using System.Text.Json;

public class Program
{
    public static void Main()
    {
            var a = new A{Name = "你好"};
            var s = JsonSerializer.Serialize(a);
        
            var unescaped = System.Text.RegularExpressions.Regex.Unescape(s);

            Console.WriteLine(s);
            Console.WriteLine(unescaped);
        }
}

class A {
    public string Name {get; set;}
}

Output:

{"Name":"\u4F60\u597D"}
{"Name":"你好"}

edited Jun 11, 2021 at 11:24

answered Jun 11, 2021 at 10:55

Steven Peirce

5467 silver badges11 bronze badges

1 Comment

code5 Over a year ago

This works the best for all cases.

Collectives™ on Stack Overflow

dotnet core System.Text.Json unescape unicode string

4 Answers 4

9 Comments

4 Comments

2 Comments

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

9 Comments

4 Comments

2 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related