How can i write python decorator for caching?

Question

I'm trying to write python decorator for memoize. I have few questions.

How does @memoize translate to memoize class's call function?
Why does init expect an argument.
Where is the cache stored? Is it associated with each and every function or it's a global variable? i.e Will there be two cache objects if i use @memoize for multiple functions.

..

class memoize:
    def __init__(self):
        self.cache = {}

    def __call__(self, function):
        def wrapper(*args, **kwargs):
            key = str(function.__name__) + str(args) + str(kwargs)
            if key in cache:
                return cache[key]
            else:
                value = function(*args, **kwargs)
                cache[key] = value
                return value
        return wrapper

@memoize
def fib(n):
    if n in (0, 1):
        return 1
    else:
        return fib(n-1) + fib(n-2)

for i in range(0, 10):
    print(fib(i))

I'm getting compilation error.

Traceback (most recent call last):
  File "memoize.py", line 17, in <module>
    @memoize
TypeError: __init__() takes exactly 1 argument (2 given)

Ok, just double checking, as there is a lru_cache decorator. — Dair
– Dair, Commented Jan 26, 2015 at 6:40

myaut · Accepted Answer · 2015-01-26 06:52:29Z

5

You should remember that @decorator is just a syntactic sugar for func = decorator(func). So here comes a difference:

(1)

@decorator 
def func():
     ...

is same as

func = decorator(func)  # Just call of __init__
func(...)               # Call of decorator.__call__

but (2)

@decorator(some_param)
def func():
     ...

is similiar to

# Call of __init__ plus call of __call__
func = decorator(some_param)(func)  
# Call of closure returned by decorator.__call__
func(...)

You have implemented decorator accepting arguments for (2) syntax, but do not provide them when using them as in example (1). That is why __init__ complaining, it receives func as second argument.
You should write self.cache in a wrapper closure, so wrapper would reference corresponding decorator object. Writing just cache will cause global variable search and thus will fail.

UPD: I changed your code to approach (1):

class memoize:
    def __init__(self, function):
        self.cache = {}
        self.function = function

    def __call__(self, *args, **kwargs):        
        key = str(args) + str(kwargs)
        if key in self.cache:
            return self.cache[key]

        value = self.function(*args, **kwargs)
        self.cache[key] = value
        return value

@memoize
def fib(n):
    if n in (0, 1):
        return 1
    else:
        return fib(n-1) + fib(n-2)

for i in range(0, 10):
    print(fib(i))

print(fib.cache)

edited Jan 26, 2015 at 6:52

answered Jan 26, 2015 at 6:40

myaut

11.6k2 gold badges34 silver badges66 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

user1159517 Over a year ago

If you see this example, wiki.python.org/moin/PythonDecoratorLibrary#Memoize cache is a member variable of class memoize. No wrappers around

myaut Over a year ago

I've changed your approach to (1) and posted code, but I suggest you should carefully read my explanation.

user1159517 Over a year ago

Okay. How should add a ttl now? I would like to use @memoize(ttl=10). I tried adding a parameter to init function in the class and got error again

myaut Over a year ago

He-he, I'd expected you to ask this. Than you need to change code from approach (1) to (2). Use your original code (but change cache -> self.cache). See my answer to your second question.

sffc Over a year ago

This is an extremely useful Python decorator for lots of applications. Thanks for sharing!

|

Collectives™ on Stack Overflow

How can i write python decorator for caching?

1 Answer 1

6 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

6 Comments

Your Answer

Sign up or log in

Post as a guest

Related