swig/python: object does not support indexing

Question

Given this set of files:

foo.h:

#pragma once

#include <stdio.h>

template <class T0> class Foo {
  public:
    T0 m[3];

    Foo(const T0 &a, const T0 &b, const T0 &c) {
        m[0] = a;
        m[1] = b;
        m[2] = c;
    }
    void info() { printf("%d %d %d\n", m[0], m[1], m[2]); }
    // T0 &operator[](int id) { return ((T0 *)m)[id]; }
};

foo.cpp:

#include "foo.h"

foo.i (Attempt1):

%module foo

%{
#include "foo.h"
%}

%include "foo.h"

%template(intFoo) Foo<int>;

%extend Foo{
    T0& __getitem__(int id) { return ((T0 *)m)[id]; }
}

setup.py:

import os
import sys
from setuptools import setup, Extension

foo_module = Extension('_foo',
                           sources=[
                               'foo.i',
                               'foo.cpp'
                           ],
                           swig_opts=['-c++', '-py3', '-builtin'],
                           include_dirs=['.']
                           )

setup(name='foo',
      version='0.1',
      platforms=['Windows', 'Linux'],
      ext_modules=[foo_module],
      py_modules=["foo"],
      )

test.py:

from foo import intFoo

a = intFoo(10,20,30)
print(dir(a))
a.info()
print(a[2])

I build the extension running:

python setup.py build_ext --force -i

but when i try to run test.py i'll get:

TypeError: 'foo.intFoo' object does not support indexing

The statement extend in foo.i is the answer suggested on any other SO related threads, that means I'm using it incorrectly here. Could anyone explain how to fix this so when i run test.py is able to use the [] operator succesfully?

Another attempts:

Attempt2:

%module foo

%{
#include "foo.h"
%}

%include "foo.h"

%template(intFoo) Foo<int>;

%extend intFoo{
    T0& __getitem__(int id) { return ((T0 *)m)[id]; }
}

Throws this error TypeError: 'foo.intFoo' object does not support indexing

Attempt3

%module foo

%{
#include "foo.h"
%}

%include "foo.h"

%extend Foo{
    T0& __getitem__(int id) { return ((T0 *)m)[id]; }
}

%template(intFoo) Foo<int>;

Throws this error foo_wrap.cpp(3808): error C2065: 'm': undeclared identifier

Try %extend intFoo or move the %extend Foo before %template(intFoo). — Mark Tolonen
– Mark Tolonen, Commented Jun 1, 2017 at 17:30
@MarkTolonen Tried your suggestions and edited the question, unfortunately, no luck :( — BPL
– BPL, Commented Jun 1, 2017 at 18:33
As far as I know, the %extend feature does not work together with the -builtin option — Jens Munk
– Jens Munk, Commented Jun 2, 2017 at 23:30
It's nice to see a question with setup.py and all the files needed for efficient reproduction of your problem. — Flexo - Save the data dump
– Flexo - Save the data dump ♦, Commented Jun 5, 2017 at 8:38
FWIW, this is a duplicate of stackoverflow.com/questions/22736700/…. This question cannot closed as a duplicate due to the open bounty. — R Sahu
– R Sahu, Commented Jun 5, 2017 at 15:50

Flexo - Save the data dump · Accepted Answer · 2017-06-05 08:37:25Z

(Throughout this example I'm working with your first version of foo.i)

First up the %extend you've got needs to be specified before the %template directive to have any effect.

Once we fix that we now get a compiler error from your %extend code:

foo_wrap.cpp: In function 'int& Foo_Sl_int_Sg____getitem__(Foo<int>*, int)':
foo_wrap.cpp:3705:85: error: 'm' was not declared in this scope

This happens because the methods you add with %extend aren't really members of the class you're adding them to. To access m in this context we need to refer to it with $self->m instead. SWIG will replace $self with the appropriate variable for us. (It's worth a quick peek at the generated code to understand how this works)

One useful tip when debugging typemaps or extensions is to search for the code you wrote in the generated output from SWIG - if it's not there then it isn't being applied how you thought.

So once we fix the error about m not being declared we have another problem because you've compiled with -builtin:

In [1]: import foo

In [2]: f=foo.intFoo(1,2,3)

In [3]: f[0]
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-3-e71eec16918d> in <module>()
----> 1 f[0]

TypeError: 'intFoo' object does not support indexing

In [4]: f.__getitem__(0)
Out[4]: <Swig Object of type 'int *' at 0xa8807d40>

Even though you've added __getitem__, indexing with f[n] still doesn't work. That's happening because with pure C-API classes in Python operator overloading doesn't work the same way. You've added a __getitem__ method successfully, but Python is looking in the builtin type's slots (specifically mp_subscript) for a way to perform the operation. So we need to fix that too. With that done a working foo.i looks like:

%module foo

%{
#include "foo.h"
%}

%feature("python:slot", "mp_subscript", functype="binaryfunc") Foo::__getitem__;

%include "foo.h"

%extend Foo{
    T0& __getitem__(int id) { return ((T0 *)$self->m)[id]; }
}

%template(intFoo) Foo<int>;

So now we can do what you want:

In [1]: import foo

In [2]: f=foo.intFoo(1,2,3)

In [3]: f[0]
Out[3]: <Swig Object of type 'int *' at 0xb4024100>

(You don't actually have to call it __getitem__ any more, because the function gets registered in the slots, it should be possible to call the operator[] without %extend at all for instance)

Finally you probably want to change the return type to const T0&, or write a Python type to proxy objects for non-const int reference better.

It works great, but without the const in front of T0&, the return type becomes a Swig Object of type 'int *'. With the const added it becomes an integer.

Jens Munk · Accepted Answer · 2017-06-05 16:37:23Z

3

I have changed the wrong answer (stating that %extent could only be used without the -builtin option) to something that works, but again only without the 'built-in' option

setup.py

import os
import sys
from setuptools import setup, Extension

foo_module = Extension('_foo',
                           sources=[
                               'foo.i', 'foo.cpp'
                           ],
                           swig_opts=['-c++'],
                           include_dirs=['.']
                           )

setup(name='foo',
      version='0.1',
      platforms=['Windows', 'Linux'],
      ext_modules=[foo_module],
      py_modules=["foo"],
      )

foo.i

%module foo

%{
#include "foo.h"
%}

%include "carrays.i"
%array_functions(int, intArray);

%include "foo.h"

%extend Foo<int> {
%pythoncode %{
def __getitem__(self, id):
  return _foo.intArray_getitem(self.m,id)             
%}
};

%template(intFoo) Foo<int>;

Note, how the extension is generating Python code, which allows you to do many sophisticated things.

Old answer:

According to the SWIG documentation the python layer is stripped off and thereby the %extend feature is ignored (this is incorrect, the proxy objects are not created)

See http://www.swig.org/Doc3.0/SWIGDocumentation.html

36.4.2 Built-in Types

When -builtin is used, the pure python layer is stripped off. Each wrapped class is turned into a new python built-in type which inherits from SwigPyObject, and SwigPyObject instances are returned directly from the wrapped methods. For more information about python built-in extensions, please refer to the python documentation:

edited Jun 5, 2017 at 16:37

answered Jun 4, 2017 at 19:54

Jens Munk

4,7751 gold badge27 silver badges41 bronze badges

6 Comments

Flexo - Save the data dump Over a year ago

%extend applies at the C++ layer, not the Python layer so this isn't the answer here. The actual answer is in 36.4.2.2

Jens Munk Over a year ago

@Flexo I have several examples using %extend, where the only effect is injection of code in the Python layer. Removal of the built-in flag works fine for me and of course referencing the object using $self.

Flexo - Save the data dump Over a year ago

Can you share one on pastebin or something? %extend is generic SWIG and not from the Python module, so I don't understand how that can be. %pythoncode, %pythonprepend and %feature('shadow') are all Python specific and won't do anything with -builtin, but %extend is as though your wrote it in your C or C++ declaration/definition

Jens Munk Over a year ago

I write, %extend Foo<int> { %pythoncode %{ def __getitem__(int i): .... In this way, the extension is only affecting the Python code. I have never used %extend without %pythoncode. I learn every day

Flexo - Save the data dump Over a year ago

Heh, I'd never used %pythoncode inside %extend and had always added more pure Python methods by 'monkey patching' with module level %pythoncode

|

Collectives™ on Stack Overflow

swig/python: object does not support indexing

2 Answers 2

1 Comment

6 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

6 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related