C# StreamReader Encoding.UTF8 not working

Question

I have a C# Project in Visual studio which download and parse XML file that contains Korean, Chinese and another unicode characters. For example for korean artist named Taeyang it produce XML like this :

<name>태양</name>

but it returns

<name>??</name>

I have tried StreamReader Encoding.Default but result is

<name>íƒœì–‘</name>

The code:

string address = String.Format("http://musicbrainz.org/ws/2/artist/{0}?inc=url-rels", mbids[ord]);
HttpWebRequest newRequest = WebRequest.Create(address) as HttpWebRequest;
               newRequest.Headers["If-None-Match"] = etagProf;
               newRequest.Headers[HttpRequestHeader.AcceptEncoding] = "gzip";
var response = newRequest.GetResponse();
// Reader
Stream stream = response.GetResponseStream();
StreamReader reader = new StreamReader(stream, Encoding.UTF-8);
string data = reader.ReadToEnd();

and the xml source:

<?xml version="1.0" encoding="UTF-8"?>
<metadata xmlns="http://musicbrainz.org/ns/mmd-2.0#">
    <artist type="Person" id="d84e5667-3cbe-4556-b551-9d7e4be95d71">   
        <name>태양</name>
        <sort-name>Taeyang</sort-name><gender>Male</gender>
        <country>KR</country>
        ...........
    </artist>
</metadata>

I'm confused, why it happens ? Any idea dude ?

Are you sure the source file is actually in UTF8?

Allan S. Hansen
– Allan S. Hansen

2015-02-20 07:58:45 +00:00
Commented Feb 20, 2015 at 7:58 — Allan S. Hansen
– Allan S. Hansen, Commented Feb 20, 2015 at 7:58
Sure, i checked its charset from response header

Mike
– Mike

2015-02-20 07:59:51 +00:00
Commented Feb 20, 2015 at 7:59 — Mike
– Mike, Commented Feb 20, 2015 at 7:59
Can you share one of your original input xml files?

netblognet
– netblognet

2015-02-20 08:15:42 +00:00
Commented Feb 20, 2015 at 8:15 — netblognet
– netblognet, Commented Feb 20, 2015 at 8:15
@netblognet : just updated my question, please check it :)

Mike
– Mike

2015-02-20 08:20:47 +00:00
Commented Feb 20, 2015 at 8:20 — Mike
– Mike, Commented Feb 20, 2015 at 8:20

Sagiv b.g · Accepted Answer · 2015-02-20 08:55:09Z

6

using the code below (notice I comment out 2 of your lines)

//newRequest.Headers["If-None-Match"] = "d84e5667-3cbe-4556-b551-9d7e4be95d71";
//newRequest.Headers[HttpRequestHeader.AcceptEncoding] = "gzip";

and changed your line: StreamReader(stream, Encoding.UTF-8);

to : StreamReader(stream, Encoding.UTF8);

I got a good result characters wise: enter image description here

string address = String.Format("http://musicbrainz.org/ws/2/artist/{0}?inc=url-rels","d84e5667-3cbe-4556-b551-9d7e4be95d71");
HttpWebRequest newRequest = WebRequest.Create(address) as HttpWebRequest;
//newRequest.Headers["If-None-Match"] = "d84e5667-3cbe-4556-b551-9d7e4be95d71";
//newRequest.Headers[HttpRequestHeader.AcceptEncoding] = "gzip";
var response = newRequest.GetResponse();
// Reader
Stream stream = response.GetResponseStream();
StreamReader reader = new StreamReader(stream, Encoding.UTF8);
string data = reader.ReadToEnd();
MessageBox.Show(data);

edited Feb 20, 2015 at 8:55

answered Feb 20, 2015 at 8:10

Sagiv b.g

31.1k10 gold badges72 silver badges106 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

Mike Over a year ago

I just did it and output íƒœì–‘ again

chouaib Over a year ago

@Sag1v: GetEncoding is not relied on to determine the real text encoding

netblognet Over a year ago

@Sag1v - came to the same result. The code runs fine on my machine, too. So if this doesn't solve the problem of Michael Antonio, maybe his OS has some problems handling the UTF-8 code.

Sagiv b.g Over a year ago

@netblognet could be, but I think the problem resist in the 2 lines I commented out newRequest.Headers["If-None-Match"] = "d84e5667-3cbe-4556-b551-9d7e4be95d71"; newRequest.Headers[HttpRequestHeader.AcceptEncoding] = "gzip";

Mike Over a year ago

hi all, i fixed this problem. please look at my answer. thanks for your time ... nice to share :)

chouaib · Accepted Answer · 2015-02-20 08:01:58Z

0

try UTF8 Encoding

StreamReader sr= new StreamReader(file_name, System.Text.Encoding.UTF8);

edited Feb 20, 2015 at 8:01

answered Feb 20, 2015 at 7:56

chouaib

2,8175 gold badges23 silver badges36 bronze badges

2 Comments

chouaib Over a year ago

@MichaelAntonio is Encoding.UTF-8 a typo?

netblognet Over a year ago

What's the computer's system language and OS you're working on?

nathanchere · Accepted Answer · 2016-06-16 11:46:57Z

0

I found that Console.WriteLine() can't output unicode clearly. Everything unicode (e.g. Korean, Chinese) and all characters except a-z and 0-9 can't output as expected cause Console.WriteLine() use single font Raster Font

But the main problem was about my DB CONNECTION, i forget to add charset=utf-8 in my connection string

edited Jun 16, 2016 at 11:46

nathanchere

8,10815 gold badges68 silver badges86 bronze badges

answered Feb 20, 2015 at 10:05

Mike

1,23112 silver badges17 bronze badges

Collectives™ on Stack Overflow

C# StreamReader Encoding.UTF8 not working

3 Answers 3

5 Comments

2 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

5 Comments

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related