python structure JSON serializing issue

Question

I've got a cache dictionary that stores a 3-element list for each key: [value, date_created, hits]. The cache communication is done via JSON. There is a items command in the cache that shall return all items. This is the set cache method:

@status
def handle_set(self, key, value):
    self.data[key] = [value, datetime.datetime.now(), 0]
    return

The problem occurs when I have a non-empty cache and I call items on it. Python datetime object is not serializable:

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/twisted/python/log.py", line 84, in callWithLogger
    return callWithContext({"system": lp}, func, *args, **kw)
  File "/usr/lib/python2.7/dist-packages/twisted/python/log.py", line 69, in callWithContext
    return context.call({ILogContext: newCtx}, func, *args, **kw)
  File "/usr/lib/python2.7/dist-packages/twisted/python/context.py", line 118, in callWithContext
    return self.currentContext().callWithContext(ctx, func, *args, **kw)
  File "/usr/lib/python2.7/dist-packages/twisted/python/context.py", line 81, in callWithContext
    return func(*args,**kw)
--- <exception caught here> ---
  File "/usr/lib/python2.7/dist-packages/twisted/internet/selectreactor.py", line 146, in _doReadOrWrite
    why = getattr(selectable, method)()
  File "/usr/lib/python2.7/dist-packages/twisted/internet/tcp.py", line 460, in doRead
    rval = self.protocol.dataReceived(data)
  File "./server.py", line 17, in dataReceived
    result = getattr(self.factory, command)(**request)
  File "./server.py", line 35, in execute
    return json.dumps(result)
  File "/usr/lib/python2.7/json/__init__.py", line 231, in dumps
    return _default_encoder.encode(obj)
  File "/usr/lib/python2.7/json/encoder.py", line 201, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/usr/lib/python2.7/json/encoder.py", line 264, in iterencode
    return _iterencode(o, 0)
  File "/usr/lib/python2.7/json/encoder.py", line 178, in default
    raise TypeError(repr(o) + " is not JSON serializable")
exceptions.TypeError: datetime.datetime(2013, 10, 26, 11, 38, 42, 348094) is not JSON serializable

I have found a similar SO question. But the thing I don't like in the accepted answer is that I have to provide a custom serializer.

in my cache I have different commands, but I'd like to use one JSON-formatting method for all cache commands. I'm afraid I'd have to do an if-cascade if I were to follow that answer.

Is there any way to override datetime.datetime to provide one simple method that will be used by JSON serializer? Or any better solution to this?

datetime.datetime JSON serialization could be as simple as str(d) for me (a string representation).

I don't understand "in my cache I have different commands". I don't think you want to serialize the complete object along with the various methods attached. In this case, I would recommend writing a MyClass.to_JSON method which converts the datetime object to string via d.isoformat() and then a @staticmethod MyClass.from_JSON which converts it back using dateutil.parser.parse(d_str). — musically_ut
– musically_ut, Commented Oct 26, 2013 at 10:11

Krishan Gupta · Accepted Answer · 2013-11-19 21:15:06Z

The generally accepted approach is to subclass the default encoder.

import json

class CustomJSONEncoder(json.JSONEncoder):
    def default(self, obj):
        if hasattr(obj, 'isoformat'): #handles both date and datetime objects
            return obj.isoformat()
        elif hasattr(obj, 'total_seconds'): #handles both timedelta objects
            return str(obj)
        if isinstance(obj, Decimal): #handles decimal timedelta objects
            return float(obj)
        else:
            return json.JSONEncoder.default(self, obj)

To use the new class:

context['my_json_data'] = json.dumps(my_python_data, cls=CustomJSONEncoder)

The issue is that there is no universally accepted way to represent a date, so Python forces you to choose. While it can be frustrating, Python is forcing you to accept the fact that any time you convert a date you're making a choice.

Collectives™ on Stack Overflow

python structure JSON serializing issue

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related