How to store a file in memory with Python (django)?

Question

I have this code: NNModel.py

import pickle
from keras.models import load_model

def load_obj(name):
    with open('/home/user/' + name + '.pkl', 'rb') as f:
        return pickle.load(f)

def load_stuff():
    model = load_model("/home/user/model2.h5")
    voc = load_obj('voc')
    return (model,voc)

It loads my files when I use this function, I'd like to load them to some static or singleton ?class? on the first time and then access that file. How can I achieve that in Python/django?

This file is fairly big and right now I belive every request loads it into memory which is not efficient I guess...

Will Keeling · Accepted Answer · 2018-03-15 15:58:08Z

It sounds as though you want to cache the load of the expensive file so it doesn't need to be reloaded on each subsequent request. You could use Django's caching framework to achieve that.

Ensure that you have the cache configuration specified in your settings.py. For example, if you're using memcache:

CACHES = {
    'default': {
        'BACKEND': 'django.core.cache.backends.memcached.PyLibMCCache',
        'LOCATION': 'localhost:11211',
    }
}

And then in your code, you can ask the cache for the file before you try to load it:

from django.core.cache import cache

def load_obj(name):
    with open('/home/user/' + name + '.pkl', 'rb') as f:
        return pickle.load(f)

def load_stuff():
    model = load_model("/home/user/model2.h5")

    voc = cache.get('my_files_cache_key')
    if voc is None:
        voc = load_obj('voc')
        cache.set('my_files_cache_key', voc, 600)

    return (model,voc)

The above example will cache the value of voc for 10 minutes (600 seconds).

See https://docs.djangoproject.com/en/2.0/topics/cache/ for more details on setting up and using Django's cache framework.

Collectives™ on Stack Overflow

How to store a file in memory with Python (django)?

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related