Formatting JSON in Python

Question

What is the simplest way to pretty-print a string of JSON as a string with indentation when the initial JSON string is formatted without extra spaces or line breaks?

Currently I'm running json.loads() and then running json.dumps() with indent=2 on the result. This works, but it feels like I'm throwing a lot of compute down the drain.

Is there a more simple or efficient (built-in) way to pretty-print a JSON string? (while keeping it as valid JSON)

Example

import requests
import json

response = requests.get('http://spam.eggs/breakfast')
one_line_json = response.content.decode('utf-8')
pretty_json = json.dumps(json.loads(response.content), indent=2)

print(f'Original: {one_line_json}')
print(f'Pretty: {pretty_json}')

Output:

Original: {"breakfast": ["spam", "spam", "eggs"]}
Pretty: {
  "breakfast": [
    "spam", 
    "spam", 
    "eggs"
    ]
}

@SuperStew but then the output is a formatted Python object, not JSON (e.g. True vs true, None vs null). — Mike Scotty
– Mike Scotty, Commented Nov 6, 2018 at 15:59
You have to parse the string one way or another, and then format and print it, one way or another. I don't think json.loads and json.dumps is more expensive than any other approch, e.g. using pprint instead of dumps. Probably even faster. — tobias_k
– tobias_k, Commented Nov 6, 2018 at 15:59
@Idlehands and then OP invoked json.dumps on the Python object, turning it into JSON. The whole question is about JSON output. A string representation of a Python object just is not the same as JSON, though both of them are similar. — Mike Scotty
– Mike Scotty, Commented Nov 6, 2018 at 16:08
@WhiteHotLoveTiger - Can you post a testable example for the type of page you're trying to decode? The link 'spam.eggs/breakfast' does not resolve. If you provide usable URL I can update my answer to reflect the cost of decoding the HTTP request. — Scott Skiles
– Scott Skiles, Commented Nov 6, 2018 at 16:29

Scott Skiles · Accepted Answer · 2018-11-06 21:08:20Z

11

json.dumps(obj, indent=2) is better than pprint because:

It is faster with the same load methodology.
It has the same or similar simplicity.
The output will produce valid JSON, whereas pprint will not.

pprint_vs_dumps.py

import cProfile
import json
import pprint
from urllib.request import urlopen


def custom_pretty_print():
    url_to_read = "https://www.cbcmusic.ca/Component/Playlog/GetPlaylog?stationId=96&date=2018-11-05"
    with urlopen(url_to_read) as resp:
        pretty_json = json.dumps(json.load(resp), indent=2)
    print(f'Pretty: {pretty_json}')


def pprint_json():
    url_to_read = "https://www.cbcmusic.ca/Component/Playlog/GetPlaylog?stationId=96&date=2018-11-05"
    with urlopen(url_to_read) as resp:
        info = json.load(resp)
    pprint.pprint(info)


cProfile.run('custom_pretty_print()')
>>> 71027 function calls (42309 primitive calls) in 0.084 seconds

cProfile.run('pprint_json()')
>>>164241 function calls (140121 primitive calls) in 0.208 seconds

Thanks @tobias_k for pointing out my errors along the way.

edited Nov 6, 2018 at 21:08

answered Nov 6, 2018 at 16:00

Scott Skiles

3,8867 gold badges45 silver badges73 bronze badges

Sign up to request clarification or add additional context in comments.

14 Comments

tobias_k Over a year ago

But is this any faster than using json.dumps? I seriously doubt that. Also, the result may not be valid JSON.

Scott Skiles Over a year ago

Ah good question, he did specify more "efficient" didn't he. I may have incorrectly interpreted this as fewer lines of code. I'd have to profile to compare efficiency directly.

tobias_k Over a year ago

It's not even fewer lines of code, as you just replace one function call with another. If anything, you don't need the indent parameter, but as I said: Invalid JSON for things like True, False, None, etc.

Scott Skiles Over a year ago

Thanks @tobias_k. I see what you mean about the cost to decode. And the cProfile isn't a one-to-one comparison with OP's example. Unfortunately, 'spam.eggs/breakfast' does not resolve into a usable example.

tobias_k Over a year ago

Sorry, but you are comparing apples and oranges here. If you do a timing analysis, please do it properly. Why is loading the JSON from the URL part of the method you are timing? This will take most time and is irrelevant for the question. Also, you are using two different methods for loading the content, skewing the results even further. Also, in your first example you are pretty-printing just the raw JSON string, not the JSON dictionary. Then, using pprint you only print part of the full JSON object. If you load the JSON just once, then json.dumps will come out a bit faster.

|

r.ook · Accepted Answer · 2018-11-06 17:50:01Z

I think for a true JSON object print, it's probably as good as it gets. timeit(number=10000) for the following took about 5.659214497s:

import json
d = {   
        'breakfast': [
            'spam', 'spam', 'eggs', 
            {
                'another': 'level', 
                'nested': [
                    {'a':'b'}, 
                    {'c':'d'}
                ]
            }
        ], 
        'foo': True,
        'bar': None
    }
s = json.dumps(d)
q = json.dumps(json.loads(s), indent=2)
print(q)

I tried with pprint, but it actually wouldn't print the pure JSON string unless it's converted to a Python dict, which loses your true, null and false etc valid JSON as mentioned in the other answer. As well it doesn't retain the order in which the items appeared, so it's not great if order is important for readability.

Just for fun I whipped up the following function:

def pretty_json_for_savages(j, indentor='  '):
    ind_lvl = 0
    temp = ''
    for i, c in enumerate(j):
        if c in '{[':
            print(indentor*ind_lvl + temp.strip() + c)
            ind_lvl += 1
            temp = ''
        elif c in '}]':
            print(indentor*ind_lvl + temp.strip() + '\n' + indentor*(ind_lvl-1) + c, end='')
            ind_lvl -= 1
            temp = ''
        elif c in ',':
            print(indentor*(0 if j[i-1] in '{}[]' else ind_lvl) + temp.strip() + c)
            temp = ''
        else:
            temp += c
    print('')

# {
#   "breakfast":[
#     "spam",
#     "spam",
#     "eggs",
#     {
#       "another": "level",
#       "nested":[
#         {
#           "a": "b"
#         },
#         {
#           "c": "d"
#         }        
#       ]      
#     }    
#   ],
#   "foo": true,
#   "bar": null
# }

It prints pretty alright, and unsurprisingly it took a whooping 16.701202023s to run in timeit(number=10000), which is 3 times as much as a json.dumps(json.loads()) would get you. It's probably not worthwhile to build your own function to achieve this unless you spend some time to optimize it, and with the lack of a builtin for the same, it's probably best you stick with your gun since your efforts will most likely give diminishing returns.

Collectives™ on Stack Overflow

Formatting JSON in Python

2 Answers 2

14 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

14 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related