Within the functools pacakge in Python3, there is a lru_cache() decorator that will memo-ize your function calls.
Is there a way for me to dump out this cache into a file and then load the file back into in-memory later?
I couldn't find this feature in the functools documentation. What would be the recommended way to achieve the above requirements, preferably with the solution involving only Python?
I don't know a standart way to resolve this problem. But you can write your annotation like this:
def diskcached(cachefile, saveafter=1):
def cacheondisk(fn):
try:
with open(cachefile, 'rb') as f:
cache = pickle.load(f)
except:
cache = {}
unsaved = [0]
@wraps(fn)
def usingcache(*args, **kwargs):
try:
key = hash((args, kwargs))
except TypeError:
key = repr((args, kwargs))
try:
ret = cache[key]
except KeyError:
ret = cache[key] = fn(*args, **kwargs)
unsaved[0] += 1
if unsaved[0] >= saveafter:
with open(cachefile, 'wb') as f:
pickle.dump(cache, f)
unsaved[0] = 0
return ret
return usingcache
return cacheondisk
and use with
@diskcached("filename_to_save")
def your_function():
...
Here is a different solution using the third party package joblib (pip install joblib), which has served me very well:
from joblib import Memory
memory = Memory("/usr/src/app/no_sync/tmp/", verbose=0)
@memory.cache
def args_and_results_of_this_function_are_cached_to_disk(a,b):
return a + b
