python: catch exception for invalid JSON in requests library

Question

I have pretty old system CentOS-7.4 and python package requests-2.26.0.

import requests
...

try:
   response = requests.post("http://localhost:8000", json={ '' })
except requests.exceptions.RequestException as e:
   print e
except requests.exceptions.InvalidJSONError:
   print "Invalid JSON Error."
...

Since the json passed to post method is invalid, I was expecting that exceptions would catch it, however I'm getting this:

Traceback (most recent call last):
  File "./cfg.py", line 219, in <module>
    response = requests.post(URL, json={ '' })
  File "/usr/lib/python2.7/site-packages/requests/api.py", line 117, in post
    return request('post', url, data=data, json=json, **kwargs)
  File "/usr/lib/python2.7/site-packages/requests/api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 528, in request
    prep = self.prepare_request(req)
  File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 466, in prepare_request
    hooks=merge_hooks(request.hooks, self.hooks),
  File "/usr/lib/python2.7/site-packages/requests/models.py", line 319, in prepare
    self.prepare_body(data, files, json)
  File "/usr/lib/python2.7/site-packages/requests/models.py", line 471, in prepare_body
    body = complexjson.dumps(json, allow_nan=False)
  File "/usr/lib64/python2.7/json/__init__.py", line 250, in dumps
    sort_keys=sort_keys, **kw).encode(obj)
  File "/usr/lib64/python2.7/json/encoder.py", line 207, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/usr/lib64/python2.7/json/encoder.py", line 270, in iterencode
    return _iterencode(o, 0)
  File "/usr/lib64/python2.7/json/encoder.py", line 184, in default
    raise TypeError(repr(o) + " is not JSON serializable")
TypeError: set(['']) is not JSON serializable

What am I doing wrong?

flakes · Accepted Answer · 2021-10-29 03:14:47Z

1

Requests uses simplejson or json libraries to encode the JSON data. Anything that raises ValueError is caught and cast to InvalidJSONError, but TypeError is not caught. See here:

try:
    body = complexjson.dumps(json, allow_nan=False)
except ValueError as ve:
    raise InvalidJSONError(ve, request=self)

Your test would work with either of these examples:

a = {}
a["a"] = a
requests.get("http://url", json=a)

requests.get("http://url", json={"a": float("nan")})

Easiest thing to do would be to catch TypeError as well in the except:

try:
    response = requests.post("http://localhost:8000", json={''})
except requests.exceptions.RequestException as e:
    print(e)
except (requests.exceptions.InvalidJSONError, TypeError):
    print("Invalid JSON Error.")

answered Oct 29, 2021 at 3:14

flakes

24.1k8 gold badges52 silver badges101 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

flakes Over a year ago

A note I wont put in this answer.. It might be worth suggesting this upstream to requests as a feature enhancement and have them do the catch converting TypeError to InvalidJSONError

Mark Over a year ago

thanks for your suggestion. Is TypeError a python's built-in exception?

flakes Over a year ago

@Mark Yes it is! See here docs.python.org/3/library/exceptions.html

Collectives™ on Stack Overflow

python: catch exception for invalid JSON in requests library

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related