How to remove comment lines from a JSON file in python

Question

I am getting a JSON file with following format :

// 20170407
// http://info.employeeportal.org

{
 "EmployeeDataList": [
{
 "EmployeeCode": "200005ABH9",
 "Skill": CT70,
 "Sales": 0.0,
 "LostSales": 1010.4
} 
 ]
}

Need to remove the extra comment lines present in the file.

I tried with the following code :

import json
import commentjson

with open('EmployeeDataList.json') as json_data:
            employee_data = json.load(json_data)
            '''employee_data = json.dump(json.load(json_data))'''
            '''employee_data = commentjson.load(json_data)'''
            print(employee_data)`

Still not able to remove the comments from the file and bring the JSON file in correct format.

Not getting where things are going wrong? Any direction in this regard is highly appreciated.Thanks in advance

// comments are not allowed in JSON. So what you have is not valid JSON. You will have to remove the comments before parsing. — Klaus D.
– Klaus D., Commented Apr 9, 2017 at 3:58
@Klaus D...This is a way JSON file is generated. Is there a way to remove the comment lines from the file and bring it into correct format? While searching on internet I also came across JSON5 but able to get how to use it? — Eupheus
– Eupheus, Commented Apr 9, 2017 at 4:19
@user4569636: Your file can't easily be turned into valid JSON. It contains not only comments, but references to variables: "Skill": CT70. — Blender
– Blender, Commented Apr 9, 2017 at 4:25
What you have is probably JSON5, which contains comments and variables, as @Blender mentions. This appears to be only parsable by Javascript, not Python. github.com/json5/json5 — OneCricketeer
– OneCricketeer, Commented Apr 9, 2017 at 4:37

Blender · Accepted Answer · 2017-04-09 04:22:16Z

6

You're not using commentjson correctly. It has the same interface as the json module:

import commentjson

with open('EmployeeDataList.json', 'r') as handle:
    employee_data = commentjson.load(handle)

print(employee_data)

Although in this case, your comments are simple enough that you probably don't need to install an extra module to remove them:

import json

with open('EmployeeDataList.json', 'r') as handle:
    fixed_json = ''.join(line for line in handle if not line.startswith('//'))
    employee_data = json.loads(fixed_json)

print(employee_data)

Note the difference here between the two code snippets is that json.loads is used instead of json.load, since you're parsing a string instead of a file object.

edited Apr 9, 2017 at 4:22

answered Apr 9, 2017 at 4:16

Blender

300k55 gold badges463 silver badges512 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

spicypumpkin Over a year ago

@user4569636 How so? His second proposed solution should fix the example you posted.

Blender Over a year ago

@Posh_Pumpkin: The "JSON" contains variables ("Skill": CT70), not just comments.

OneCricketeer Over a year ago

@Posh_Pumpkin See my answer

Eupheus Over a year ago

@Blender..Thanks. The second solution does solves it but there is an additional "u'EmployeeCode": u'200005ABH9" added and the JSON file sequence go changed.

Blender Over a year ago

@user4569636: The u prefix just denotes a unicode string, it's not a problem. As for the order, objects in JavaScript and dictionaries in Python are both unordered. You can use have json.loads use Python's OrderedDict if the order matters: stackoverflow.com/questions/6921699/…

xlm · Accepted Answer · 2017-04-09 05:01:26Z

1

Try JSON-minify:

JSON-minify minifies blocks of JSON-like content into valid JSON by removing all whitespace and JS-style comments (single-line // and multiline /* .. */).

edited Apr 9, 2017 at 5:01

xlm

7,72415 gold badges59 silver badges60 bronze badges

answered Apr 9, 2017 at 4:06

hailinzeng

9959 silver badges25 bronze badges

Comments

Tomas Ruiz · Accepted Answer · 2021-03-09 10:12:41Z

1

I usually read the JSON as a normal file, delete the comments and then parse it as a JSON string. It can be done in one line with the following snippet:

with open(path,'r') as f: jsonDict = json.loads('\n'.join(row for row in f if not row.lstrip().startswith("//")))

IMHO it is very convenient because it does not need CommentJSON or any other non standard library.

edited Mar 9, 2021 at 10:12

answered Feb 10, 2018 at 13:15

Tomas Ruiz

1504 silver badges15 bronze badges

1 Comment

Jim Oldfield Over a year ago

Interesting answer but it will fail on lines containing // other than at the start e.g. {"foo": "http://bar.com"}. Better to write if not row.lstrip().startswith("//"). Also, if you remove .readlines() and the square brackets then the lines will be iterated lazily (but still joined before passing to loads) which will be slightly more efficient.

spicypumpkin · Accepted Answer · 2017-04-09 04:05:10Z

0

Well that's not a valid json format so just open it like you would a text document then delete anything from// to \n.

with open("EmployeeDataList.json", "r") as rf:
    with open("output.json", "w") as wf:
        for line in rf.readlines():
            if line[0:2] == "//"
                continue
            wf.write(line)

answered Apr 9, 2017 at 4:05

spicypumpkin

1,2192 gold badges12 silver badges21 bronze badges

4 Comments

OneCricketeer Over a year ago

You can have two open on the same line

spicypumpkin Over a year ago

@cricket_007 huh? Where?

OneCricketeer Over a year ago

I'm just saying the second open() as wf can be moved to the first line

spicypumpkin Over a year ago

@cricket_007 Oh wait sorry I totally misread your comment, mb

OneCricketeer · Accepted Answer · 2017-04-09 04:53:19Z

0

Your file is parsable using HOCON.

pip install pyhocon

>>> from pyhocon import ConfigFactory
>>> conf = ConfigFactory.parse_file('data.txt')
>>> conf
ConfigTree([('EmployeeDataList',
             [ConfigTree([('EmployeeCode', '200005ABH9'),
                          ('Skill', 'CT70'),
                          ('Sales', 0.0),
                          ('LostSales', 1010.4)])])])

answered Apr 9, 2017 at 4:53

OneCricketeer

193k20 gold badges147 silver badges277 bronze badges

Comments

kpie · Accepted Answer · 2017-04-09 04:04:33Z

-1

If it is the same number of lines every time you can just do:

fh = open('EmployeeDataList.NOTjson',"r")
rawText = fh.read()
json_data = rawText[rawText.index("\n",3)+1:]

This way json_data is now the string of text without the first 3 lines.

answered Apr 9, 2017 at 4:04

kpie

11.2k7 gold badges33 silver badges53 bronze badges

Collectives™ on Stack Overflow

How to remove comment lines from a JSON file in python

6 Answers 6

5 Comments

Comments

1 Comment

4 Comments

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

6 Answers 6

5 Comments

Comments

1 Comment

4 Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related