JS and text encoding

Question

I have a text file with accented characters like é. I used File Encoding Checker, it appears that the file is encoded as windows-1252. I read the file with the following JS and node.js code:

fs.readFile('frenchVerbsList.txt','utf-8', function(err, data) {
    if (err) {
        return console.log("ERROR here!: " + err);
    }
    frenchWords = data.split('\r\n');
    console.log(frenchWords);
});

The output from the console.log statement shows a question mark instead of the accented characters. What has happened?

The file probably isn't UTF-8 encoded.

robertklep
– robertklep

2016-05-31 09:27:37 +00:00
Commented May 31, 2016 at 9:27 — robertklep
– robertklep, Commented May 31, 2016 at 9:27
Can you provide the file?

Marc Dix
– Marc Dix

2016-05-31 09:28:26 +00:00
Commented May 31, 2016 at 9:28 — Marc Dix
– Marc Dix, Commented May 31, 2016 at 9:28

Marc Dix · Accepted Answer · 2016-05-31 09:44:38Z

1

Node only supports some encodings and windows-1252 is not part of it. You need to convert the encoding with, for example, encoding to, for example, utf-8.

Similar to this, but haven't tested

var encoding = require("encoding");

fs.readFile('frenchVerbsList.txt', function(err, text) {
    if (err) return console.log("ERROR here!: " + err);

    var resultBuffer = encoding.convert(text, 'utf8', 'windows1252');
    frenchWords = resultBuffer.toString().split('\r\n');

    console.log(frenchWords);
})

edited May 31, 2016 at 9:44

answered May 31, 2016 at 9:33

Marc Dix

1,9891 gold badge17 silver badges29 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Gerard Over a year ago

It certainly looks good, and I duly installed the "encoding" module to this project's directory. However it didn't change the output. Then I used Notepad++ to convert the french file to utf-8, this produced correct output. Thanks for the info about the "encoding" module though, it led me in the right direction :-)

Collectives™ on Stack Overflow

JS and text encoding

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related