JSON.parse(): SyntaxError: Unexpected token � in JSON at position 0

Question

I have a json object which returns from the following PowerShell command:

Get-Service -Name "name" | ConvertTo-Json -Compress > "/to/path/name.json"

If i basically open the file in vscode it seems to be correctly formatted. After i read the file with

fs.readFile(file, 'utf8', (err, data)...

and then try JSON.parse(data) im receiving the error:

undefined:1
��{
^

SyntaxError: Unexpected token � in JSON at position 0

Then i tried to do the following: data.replace(/[^\x00-\x7F]/g, "") to only have ASCII characters, which basically seems to work at least with a console.log().

But JSON.parse then complains:

undefined:1
{
 

SyntaxError: Unexpected token  in JSON at position 1

Im not sure whats the problem there. Hopefully somebody can help me with that.

This is an example json file: As i think the format is correct. Only too much spaces which are removed by the -Compress PowerShell parameter.

{
    "CanPauseAndContinue":  false,
    "CanShutdown":  false,
    "CanStop":  false,
    "DisplayName":  "OpenSSH Authentication Agent",
    "DependentServices":  [

                          ],
    "MachineName":  ".",
    "ServiceName":  "ssh-agent",
    "ServicesDependedOn":  [

                           ],
    "ServiceHandle":  {
                          "IsInvalid":  false,
                          "IsClosed":  false
                      },
    "Status":  1,
    "ServiceType":  16,
    "StartType":  4,
    "Site":  null,
    "Container":  null,
    "Name":  "ssh-agent",
    "RequiredServices":  [

                         ]
}

You name.json file doesn't have correct syntax. Can you post name.json content? — mk23
– mk23, Commented Jul 23, 2021 at 11:48
That's probably the UTF-16 BOM that PowerShell commands tend to add. — Álvaro González
– Álvaro González, Commented Jul 23, 2021 at 11:59
@ÁlvaroGonzález How can i resolve this? Just read the file as utf-16? — tautf
– tautf, Commented Jul 23, 2021 at 12:02

Leo · Accepted Answer · 2021-07-23 12:11:57Z

3

Your JSON file seems to have a different encoding, maybe is utf16le instead of utf8.

I replicated your scenario and found help here: Strange unicode characters when reading in file in node.js app

fs.readFile(file, 'utf16le', (err, data)...

answered Jul 23, 2021 at 12:11

Leo

2,3602 gold badges15 silver badges20 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

tautf Over a year ago

Thank you @Leo. This at least removes the crazy icons at the beginning of the line. But still i receive the following error when trying JSON.parse(data): undefined:1 { ^ SyntaxError: Unexpected token in JSON at position 0

tautf Over a year ago

As i just found out opening the file with notepad++ the encoding is also not utf16le its: UCS-2-LE-BOM

Leo Over a year ago

@FlorianTaut did you try to use iconv? stackoverflow.com/a/14404102/1724128

Leo Over a year ago

I'm reading the answer above and found that "Unfortunately node.js only supports UTF-16 Little Endian or UTF-16LE", maybe there is a way to output powershell json in utf-8

tautf Over a year ago

Using iconv i could now finally solve the issue by converting it from UTF18-LE to UTF-8. Thank you Leo!

Raximjon Komiljonov · Accepted Answer · 2021-07-23 11:58:22Z

0

I think a problem occurred because of line breakers and tabs. I tried to parse your code and it was parsed successfully. Please, try to make your JSON data to be one line. Like here: How to remove all line breaks from a string.

answered Jul 23, 2021 at 11:58

Raximjon Komiljonov

764 bronze badges

3 Comments

tautf Over a year ago

I already did that. PowerShell's -Compress parameter returns the json object as a one liner. But this doens't resolve the issue.

Raximjon Komiljonov Over a year ago

Try to print JSON data to console (console.log) your JSON data, I think it is different than you expected. My test result: prnt.sc/1er9fvh

tautf Over a year ago

Komiljinov As mentioned, i tried it doenst work. This it whats printed when use the -Compress parameter to receive a minified one liner:

��{"CanPauseAndContinue":false,"CanShutdown":false,"CanStop":false,"DisplayName":"OpenSSH Authentication Agent","DependentServices":[],"MachineName":".","ServiceName":"ssh-agent","ServicesDependedOn":[],"ServiceHandle":{"IsInvalid":false,"IsClosed":false},"Status":1,"ServiceType":16,"StartType":4,"Site":null,"Container":null,"Name":"ssh-agent","RequiredServices":[]}

Igor Lima · Accepted Answer · 2021-07-23 12:21:48Z

0

Maybe its the same problem as when you're writing PHP scripts that use session objects.

In PHP, when a file is encoded with UTF8 with BOM, the PHP script that uses session completely break. To solve this, I generally open the file in Notepad++, goes to Format -> UTF8 (without BOM), and then save it again. Always works for me. This might be your case here, since these broken characters appears to be on the beginning of your file, that's exacly where BOM is.

This answer might clarify about UTF8 and BOM.

answered Jul 23, 2021 at 12:21

Igor Lima

3072 silver badges11 bronze badges

Comments

SridharKritha · Accepted Answer · 2021-10-28 17:22:03Z

0

Normally the error message having some strange characters (like �) gives hint that there maybe some problem in the encoding.

A few nodejs readFile api's with different encoding

fs.readFile(file, 'utf8', function(err, data) { ... });

fs.readFile(file, 'utf16le', function(err, data) { ... }); // le - little endian

fs.readFile(file, 'ucs2', function(err, data) { ... });   // kind  of 'utf16le'

edited Oct 28, 2021 at 17:22

answered Oct 28, 2021 at 17:16

SridharKritha

9,9192 gold badges60 silver badges45 bronze badges

Collectives™ on Stack Overflow

JSON.parse(): SyntaxError: Unexpected token � in JSON at position 0

4 Answers 4

5 Comments

3 Comments

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

5 Comments

3 Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related