30

I made a 250MB json file that should look like this:

[ {"A":"uniquevalue0", "B":[1,2,3]}, 
  {"A":"uniquevalue1", "B":[1]}, 
  {"A":"uniquevalue2", "B":[1,2,3,4]} ]

where the "B" value can be variable len >= 1. This says I have valid JSON.

I call

df = pandas.read_json('ut1.json', orient = 'records', dtype={"A":str, "B":list})

Here is the documentation. When reading into a pandas dataframe, I get the following traceback:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/.../pandas/io/json.py", line 198, in read_json     
    date_unit).parse()
  File "/.../pandas/io/json.py", line 266, in parse 
    self._parse_no_numpy()
  File "/.../pandas/io/json.py", line 496, in _parse_no_numpy
    loads(json, precise_float=self.precise_float), dtype=None)
ValueError: Unexpected character found when decoding 'true'

Can't think of what is going wrong. The python file that is throwing the error is not that helpful.

Improve this question
1
  • The code works perfectly with the given json file. Commented Dec 2, 2014 at 2:44

10 Answers 10

Reset to default
27

I had the same error message, and I solved it by using an absolute path.

import os
basePath = os.path.dirname(os.path.abspath(__file__))
df = pandas.read_json(basePath + '/ut1.json', orient = 'records', dtype={"A":str, "B":list})

That worked for me!

Improve this answer
Sign up to request clarification or add additional context in comments.

3 Comments

I tried doing this, but it is giving me a different error: "ValueError: Expected object or value". I have tried validating the JSON using jsonlint.com and it says that it's a valid json file. I don't understand what's the issue with this.
This worked well. But try using os.path.join(basePath, '/ut1.json') Instead of string concatenation.
@rodrigombs not "/ut1.json", that will cause a bug... "ut1.json" joined using os.path.join
9

In my case, the path was wrong.

Make sure you check your current working directory, by placing this just before the pandas.read_json:

import os
print(os.getcwd())
Improve this answer

Comments

4

After tried @learn2day's answer, I still cannot get a good result from there, but I do try the following code and everything works for me. (PS: I'm opening a JSON file where Chinese characters were UTF-8 characters appeared - Chinese characters)

pandas.read_json(open("ut1.json", "r", encoding="utf8"))

The encoding="utf8" is the key part of this code.

Improve this answer

Comments

4

I had the same issue and then realized that the issue was while copying and pasting text from the browser to my file. It introduced carriage returns so that each line, for a given key was broken up into multiple rows. THAT was the issue. hope this helps someone!

Improve this answer

Comments

2

For me, the issue was that the file has a UTF-8 BOM character at the beginning. Using the following encoding solved the problem:

df = pd.read_json(r'C:\temp\foo.jsonl', lines=True, encoding='utf-8-sig')
Improve this answer

Comments

1

I was getting the "Value Error: Expected object or value" today while calling pandas.read_json('my_file.json'). I have ran this code with the same file earlier, so was very worried to see it is not working today. Later, I found for some reason, the json file was not there in the same dir. Then, I downloaded the file from git by right clicking the file link. That was a bad idea :(. I guess the json file was not encoded properly, so I kept getting the same error even when the json file was there in the same dir. At the end, I deleted the json file, cloned the original git repo to get the json file and put it in the same dir again. Then, this pandas.read_json did work. So, first of all please make sure the json file exists in the proper dir and then make sure, the file is not corrupted.

Improve this answer

Comments

1

try this

df = pd.read_json('file.jsonl', lines=True) #if its a jsonl file
Improve this answer

Comments

0

Posting this because the answers above did not match my problem with this error: It just occurred for me when reading a very long string document I thought would be valid json format but contained nan as exported from python to string while it should be "null" for valid json. If the document was not created using a json package it might have faulty values indicated by this error message.

Improve this answer

Comments

0

I had the same error when running the code on linux. I realised the file name given to read_csv had .JSON instead of .json. Changing it to lower case worked for me.

Improve this answer

Comments

0

  1. First make sure your file is there, now ofcourse it is but what worked for me is deleting the file and recopying to the location and be cautious while renaming it and then read the file as :

    df_test = pd.read_json('test_file.json',lines=True)

Note that I have had encountered all different errors talked in this particular thread and finally this solution work which I have explained i.e. recopying the file without renaming it and then reading the file ( also making sure my code and file shares the same dir if I am not providing absolute path while reading the file).

Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.