Customize how a Python object is processed as a function argument?

Question

A Python class's __call__ method lets us specify how a class member should be behave as a function. Can we do the "opposite", i.e. specify how a class member should behave as an argument to an arbitrary other function?

As a simple example, suppose I have a ListWrapper class that wraps lists, and when I call a function f on a member of this class, I want f to be mapped over the wrapped list. For instance:

x = WrappedList([1, 2, 3])
print(x + 1) # should be WrappedList([2, 3, 4])

d = {1: "a", 2: "b", 3:"c"}
print(d[x]) # should be WrappedList(["a", "b", "c"])

Calling the hypothetical __call__ analogue I'm looking for __arg__, we could imagine something like this:

class WrappedList(object):
    def __init__(self, to_wrap):
        self.wrapped = to_wrap

   def __arg__(self, func):
       return WrappedList(map(func, self.wrapped))

Now, I know that (1) __arg__ doesn't exist in this form, and (2) it's easy to get the behavior in this simple example without any tricks. But is there a way to approximate the behavior I'm looking for in the general case?

Do you want it to work this way in every function, or just in a set of functions you define? For the former, you can do something kind of similar in some languages with variable types and implicit casts (C++), but not in Python. For the latter, yes, it's basically the same way operator overloading works in Python, and I can explain more if that's not enough. — abarnert
– abarnert, Commented Mar 21, 2018 at 22:48
The short answer is no. The much longer answer is yes, although you get to your syntax by defining special methods like __add__ ( for + 1) and quite a few intermediary classes... — thebjorn
– thebjorn, Commented Mar 21, 2018 at 22:48
For most of the operators, they already support this by looking up special methods with double underscores. That's how numpy does exactly your example out of the box. For a set of new functions you create, you can create the same kind of protocol—calling f on a function first calls its __arg__ method. (Although you're probably better off calling it arg or _arg.) — abarnert
– abarnert, Commented Mar 21, 2018 at 22:50

abarnert · Accepted Answer · 2018-03-21 23:03:49Z

4

You can't do this in general.^*

You can do something equivalent for most of the builtin operators (like your + example), and a handful of builtin functions (like abs). They're implemented by calling special methods on the operands, as described in the reference docs.

Of course that means writing a whole bunch of special methods for each of your types—but it wouldn't be too hard to write a base class (or decorator or metaclass, if that doesn't fit your design) that implements all those special methods in one place, by calling the subclass's __arg__ and then doing the default thing:

class ArgyBase:
    def __add__(self, other):
        return self.__arg__() + other
    def __radd__(self, other):
        return other + self.__arg__()
    # ... and so on

And if you want to extend that to a whole suite of functions that you create yourself, you can give them all similar special-method protocols similar to the builtin ones, and expand your base class to cover them. Or you can just short-circuit that and use the __arg__ protocol directly in those functions. To avoid lots of repetition, I'd use a decorator for that.

def argify(func):
    def _arg(arg):
        try:
            return arg.__arg__()
        except AttributeError:
            return arg
    @functools.wraps(func)
    def wrapper(*args, **kwargs):
        args = map(_arg, args)
        kwargs = {kw: _arg(arg) for arg in args}
        return func(*args, **kwargs)
    return wrapper

@argify
def spam(a, b):
    return a + 2 * b

And if you really want to, you can go around wrapping other people's functions:

sin = argify(math.sin)

… or even monkeypatching their modules:

requests.get = argify(requests.get)

… or monkeypatching a whole module dynamically a la early versions of gevent, but I'm not going to even show that, because at this point we're getting into don't-do-this-for-multiple-reasons territory.

You mentioned in a comment that you'd like to do this to a bunch of someone else's functions without having to specify them in advance, if possible. Does that mean every function that ever gets constructed in any module you import? Well, you can even do that if you're willing to create an import hook, but that seems like an even worse idea. Explaining how to write an import hook and either AST-patch each function creation node or insert wrappers around the bytecode or the like is way too much to get into here, but if your research abilities exceed your common sense, you can figure it out. :)

As a side note, if I were doing this, I wouldn't call the method __arg__, I'd call it either arg or _arg. Besides being reserved for future use by the language, the dunder-method style implies things that aren't true here (special-method lookup instead of a normal call, you can search for it in the docs, etc.).

_{* There are languages where you can, such as C++, where a combination of implicit casting and typed variables instead of typed values means you can get a method called on your objects just by giving them an odd type with an implicit conversion operator to the expected type.}

answered Mar 21, 2018 at 23:03

abarnert

368k54 gold badges626 silver badges691 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

user1604015 Over a year ago

Yah... I get the decorator approach, but was hoping for something cleaner, especially for cases like using my class instances as indices to dicts etc. But I guess this is the best I'm likely to do in Python - thanks! (I guess this should be possible in any language with procedural reflection, but as you point out, building on Python's structural reflection (AST etc.) is not nice in this case.)

abarnert Over a year ago

@user1604015 Once you get the hang of writing import hook hacks, it isn’t really all that hard to do something like replacing every function call and operator mode, or every CALL_FUNCTION bytecode, but you still have the problem that anything that gets called internally by a C function (not the outermost special method lookup for operators/abs/etc., but anything an extension module does—like, say, most of what happens inside any Cython function) will be a problem. So ultimately, I think you might have to fork CPython and hook the bytecode itself in ceval, and also all the relevant C API.

Collectives™ on Stack Overflow

Customize how a Python object is processed as a function argument?

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related