Best way to store JSON data in a Python Module

Question

I have a Python module I'm developing that's includes a big JSON file as part of the data it relies on. I want Python users to be able to import the JSON file as a Python variable and for users of other programming languages, to be able to use the JSON file directly.

So, what I'm trying to figure out is what's the best way to make the JSON object "importable". Right now, my solution is in __init__.py:

import json
import os

with open(os.path.join(os.path.dirname(os.path.realpath(__file__)), 'seals.json')) as f:
    seals_data = json.load(f)

Then Python devs can call:

from my_module import seals_data

And it more or less works, but this feels weird to me, and I want to make sure there isn't a cleaner way to make the json importable.

your solution seems good for what you're trying to get. but why can't they just do json.load themselves; i feel this is over complicating it — Kiarash
– Kiarash, Commented Sep 3, 2014 at 5:50
This is almost literally the code I use. I use the module load as a singleton. — ldgabbay
– ldgabbay, Commented Sep 3, 2014 at 5:50
@Kiarash, the advantage to doing it as a Python module is that the user can import it regardless of where the module is installed. With a json file, code has to know where it is in order to load it. — mlissner
– mlissner, Commented Sep 3, 2014 at 15:26
@mlissner, python only runs the code in a module once. by importing the code that parses the file, anyone who imports the module after the first call gets the already-parsed results (seals_data). also, if no one imports the module, the file never gets parsed. thus, seals_data is a singleton. — ldgabbay
– ldgabbay, Commented Sep 3, 2014 at 15:30

Luke Wahlmeier · Accepted Answer · 2014-09-03 16:06:11Z

2

Your option is probably the best way to store the data as a python structure in a module for someone to use. The only down side I see with it is anything that imports that module will have to wait for it to do IO, which is not ideal, esp if the module is ever used for things that do not involved that data.

I would use a class, or function to lazily load that data, allowing that IO hit to be offset to only when that data is requested.

Here is an example:

SEALS_DATA = None
def get_seals_data():
    global SEALS_DATA
    if SEALS_DATA is None:
        SEALS_DATA = json.load(open("%s/seals.json"%(os.path.dirname(__file__))))
    return SEALS_DATA

After they import the module they will have no deal until the first time they call get_seals_data(). Any call after that should already have the data loaded.

answered Sep 3, 2014 at 16:06

Luke Wahlmeier

1961 bronze badge

Sign up to request clarification or add additional context in comments.

2 Comments

mlissner Over a year ago

Hm, the IO during import isn't so bad. I'd almost rather have that than have IO during some random part of code execution. Not sure it's worth the extra complication and global var.

Luke Wahlmeier Over a year ago

Yeah it really all comes down to if this module will ever be imported by things that dont need the JSON object. If all the module does is provide the JSON object you can just assume no one will import it unless they want it.

Collectives™ on Stack Overflow

Best way to store JSON data in a Python Module

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related