2

I'm writing a python extension module in C. Python stops running when I declare an array of structs greater than 4 elements in a function of that module.

I'm writing the module to increase performance. I've declared 3 structs( "SKU", "Cromosoma", "Aptitud" ) and I want to create an array of Cromosoma, but when I try to create the array with more than 4 elements it breaks.

// need_for_speed.c   extension module code
#include <Python.h>
#include <stdlib.h>
#define MAX_GENES_SIZE  2000
typedef struct{
    char codigo[30];
    double venta;
    char nombre[100];
    double categoria;
    double peso;
    double ubicacion_especifica;
    double ubicacion_actual;
    double ubicacion_a_mover;
    double stock;
} SKU;
typedef struct{
    double ubicaciones_rompe_regla;
    double cercania_medio;
    double desv_std_picks_x_seccion;
    double peso_x_ubicacion;
} Aptitud;
typedef struct{
    SKU genes[MAX_GENES_SIZE];
    Aptitud aptitud;
    int genes_size;
    int edad;
}Cromosoma;

static PyObject* prueba(PyObject* self, PyObject* args){
    Cromosoma a;
    SKU s;
    strcpy(s.codigo,"1212");
    a.genes[0] = s;
    Cromosoma poblacion[] = {a,a,a,a,a};
    printf("codigo %s ", poblacion[0].genes[0].codigo);
    return PyLong_FromDouble(1);
}
static PyMethodDef Methods[] = {
    {"prueba", prueba, METH_NOARGS, "Prueba general"},
    { NULL, NULL, 0, NULL }
};

// Module Definition struct
static struct PyModuleDef need_for_speed = {
    PyModuleDef_HEAD_INIT,
    "need_for_speed",
    "Modulo para aumento de la velocidad de procesamiento para el algoritmo genético",
    -1,
    Methods
};

// Initialize module 
PyMODINIT_FUNC PyInit_need_for_speed(void)
{
    PyObject *m;
    m = PyModule_Create(&need_for_speed);
    return m;
}

the setup.py to build this module:

from distutils.core import setup, Extension
setup(name = 'need_for_speed', version = '1.0',ext_modules = [Extension('need_for_speed', ['need_for_speed.c'])])

command to build module:

python setup.py build

when I call the function prueba:

import need_for_speed
i = need_for_speed.prueba()

python stops running without printing or returning anything, but if modify the array named "poblacion" in the "prueba" function to have only 4 elements it runs perfectly, returning 1 and printing "codigo 1212".

I'm using Windows BTW.

2
  • 1
    I see no reason to believe that the code presented is responsible for the behavior described, even if we stipulate that it is corrected to make it valid C by inserting the missing } into the definition of the Cromosoma typedef. At least nothing related to the arrays. A bona fide minimal reproducible example is needed for us to determine what may be going on. Commented Jul 8, 2019 at 18:31
  • Sorry for the missing } and the missing code necessary for the minimal reproducible example. I just edited it. Commented Jul 8, 2019 at 18:41

1 Answer 1

3

It's probably a stack overflow.

Let's see how large your structs are assuming they only take the size of the individual members (neglecting padding, etc.):

  • SKU: 7 doubles and 130 chars -> 7 * 8 bytes + 130 bytes -> 186 bytes
  • Aptitud: 4 doubles -> 4 * 8 bytes -> 32 bytes
  • Cromosoma: 2 ints, 1 Aptitud and 2000 SKU -> 2 * 4 bytes + 32 bytes + 2000 * 186 bytes -> 372040 bytes

So one instance of Chromosoma will take ~370kB. And you create 5/6 of them; one with Cromosoma a; and one for each slot in the array: 4/5.

A typical stack will be only a few megabytes. With 6 * 370kB ~ 2.1MB it's at least reasonable that you exhausted your stack. For example MSVC (Windows Visual Studio C/C++ compiler) by default uses just 1 MB. Given that it fails with an array of size 5 but works with one of size 4 it seems that you have ~2 MB of stack.

To avoid this problem you could increase your stack size (how you do that depends on your compiler). However increasing the stack size will again lead to problems when you need more Chromosoma or change the number of SKUs.

Another alternative (probably better than increasing the stack size) would be to allocate all big arrays on the heap. For example you could allocate the poblacion on the heap - and/or make the Chromosoma.genes a pointer to an array of SKU.

Sign up to request clarification or add additional context in comments.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.