1

How does polymorphism work, under the hood, in python?

In python, if I have some function e.g.

def f(x):
    return x + 2*x + 3*x + 4*x + 5*x + 6*x

then according to dis.dis(f) python translates this to bytecode instructions which describe a cycle of:

  • loading the next constant value
  • loading x again
  • multiplying them together
  • adding the product (onto the accumulation of preceding terms)

But if x is a numpy array or python class, rather than a basic data type, then presumably the interpreter must do additional work (e.g. the binary multiply op-code must somehow lead other functions to be called, perhaps starting with some attribute lookups, which usually correspond to entirely different op-codes). This seems very different from ordinary assembly language, where a simple arithmetic operation would be atomic (and not cause the CPU to also execute extra instructions that aren't visible in the dissassembly listing).

Is there documentation for how the python interpreter operates, and what sequence of steps it actually performs when evaluating an expression involving polymorphism? (Ideally, at a lower level of detail then what a step-through python debugger would expose?)

Edit:

To support polymorphism, an arithmetic operation must also involve not only arithmetic but also type checking, attribute look-up, conditional jumps, and function calls. (All these things have their own op-codes.) Is it correct that cpython implements this by making the arithmetic op-code itself perform many complex actions in a single step of the interpreter (except for the instructions contained in the called function), instead of by stepping the interpreter through a sequence of separate op-codes to achieve the same result (e.g., LOAD_ATTR, CALL_FUNCTION, etc)?

Is there any documentation such as a table, for all op-codes, describing all of the actions each op-code may cause?

Improve this question
9
  • 2
    I think this is a duplicate. Will wait for others to comment. Commented Jul 19, 2017 at 3:09
  • 1
    But basically, if you want to know exactly what happens, you have to read the source code. This is implementation dependent. Here is a current CPython implementation. That is the big switch statement that actually evaluates the op-codes. Commented Jul 19, 2017 at 3:17
  • @juanpa.arrivillaga I think * for numpy array will never go to the CPython's op_code switch but will call a matrix multi function, check my update I think that is the real thing numpy has done undertook, again not sure whether it's the real file, but I'd be very surprised if numpy has done this other way :D Commented Jul 19, 2017 at 3:25
  • @armnotstrong no it certainly does not. For starters, * in numpy does not do matrix multiplication, it does vectorized/broadcasted multiplication. Second, that is implemented through the __mult__ magic methods, like everything else. There are no "basic types" in Python, anyway, that is not a useful distinction to make in a language like Python. As you see in the source code, everything except unicode-strings are treated the same, only an optimization is done for them. Commented Jul 19, 2017 at 3:30
  • @juanpa.arrivillaga I'm not a mod, but I think you're right. Each binary operator corresponds to an opcode, which is handled by the interpreter in a switch case. This is where Python dispatches to the relevant method, if need be. Commented Jul 19, 2017 at 3:41

1 Answer 1

Reset to default
0

You can define how an operator behaves for a custom class by implementing the corresponding magic method:

 >>> class MyClass(object):
...     def __add__(self, x):
...         return '%s plus %s' % (self, x)
...     def __mul__(self, x):
...         return "%s mul %s" % (self, x)

And I think that's how array in numpy does the job under the hood.

You could trace this for more information about the implementation of * for numpy's array

Improve this answer
Sign up to request clarification or add additional context in comments.

6 Comments

In this case, what op-codes does the interpreter actually execute? Isn't there a different pair of op-codes involved for looking up the __add__ attribute and calling the function method?
Have never used numpy before, as a math idiot, the only thing I could understand is the mutilation of matrix but I am checking numpy's code for more information to verify this. :D
I'm not asking what syntax can be used for operator overloading in python. I'm asking how it works at a lower level, at least for cpython.
It's nothing different from a normal function call once an override for the operator was done, maybe will load a C lib for performance reason, not sure what you are asking
You can't really 'override' operators since they're not defined for custom types to begin with. I edited the answer to reflect that. Hope you don't mind.
|

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.