PyEval_CallObject and PyObject_CallObject return a null object

Question

This is a followup question concerning this one. As pointed in the former question comments, my initial attempt for importing the module was completely wrong, and instead I should have done this :

auto module = PyImport_ImportModule("Package1.Module1");

so I did so :

auto module = PyImport_ImportModule("FV.F_V");

After doing this, now I get much more meaningful output as you can see:

module: 0000021306FF9B38
size: 30
FaceVerification: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
Image: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
Path: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
__builtins__: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
__cached__: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
__doc__: FV.F_V
__file__: C:\Users\Master\Anaconda3\Lib\site-packages\FV\F_V.py
__loader__: FV
__name__: FV.F_V
__package__: FV
__spec__: FV
__warningregistry__ : C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
align_face: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
cv2: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
data_transforms: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
get_central_face_attributes: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
math: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
nn: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
np: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
os: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
pickle: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
plt: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
resnet101: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
resnet18: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
resnet50: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
scipy: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
time: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
torch: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
tqdm: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
transforms : C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc

However, when ever I try to use a class defined in the module (i.e FaceVerificationand instantiate it, it fails. The class seems OK, but the calls that try to instantiate the object, return a null object!: I tried both PyEval_CallObject and PyObject_CallObject:

auto python_class = PyDict_GetItemString(pdict, "FaceVerification");

if (PyCallable_Check(python_class)) 
{
    object = PyEval_CallObject(python_class, args);
    # or 
    //object = PyObject_CallObject(python_class, args);
    //Py_DECREF(python_class);
}
else 
{
    cout << "Cannot instantiate the Python class" << endl;
    //Py_DECREF(python_class);
    //return 1;
}

to no avail. This is strange, I therefore added a new CONSTANT field to the F_V.py and tried to see if I can access that (to see if the module was indeed valid). It actually successfully retrieved the constant. However, I noticed, on the re-running of the c++ application, now when I print the map which holds the output of PyModule_GetDict(module), I see a different result :

module: 000001F7E07E8C28
size: 31
CONSTANT: THIS IS A TEST. REMOVE IT
FaceVerification: THIS IS A TEST. REMOVE IT
Image: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
Path: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
__builtins__: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
__cached__: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
__doc__: FV.F_V
__file__: C:\Users\Master\Anaconda3\Lib\site-packages\FV\F_V.py
__loader__: FV
__name__: FV.F_V
__package__: FV
__spec__: FV
__warningregistry__: THIS IS A TEST. REMOVE IT
align_face: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
cv2: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
data_transforms: THIS IS A TEST. REMOVE IT
get_central_face_attributes: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
math: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
nn: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
np: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
os: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
pickle: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
plt: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
resnet101: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
resnet18: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
resnet50: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
scipy: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
time: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
torch: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
tqdm: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
transforms: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc

most notably the FaceVerification which should be a class, is not showing the CONSTANT field's value!

What am I doing wrong here?

I digged a bit trough your code (and more through the one of the previous question). Not sure whether this is helpful but: PyDict_GetItemString() returns a borrowed reference. Hence, the Py_DECREF(python_class); looks very questionable to me. — Scheff's Cat
– Scheff's Cat, Commented Mar 25, 2020 at 7:43
@Scheff Thanks, but that snippet where it is used is not executed (it was borrowed from other codes, and I forgot to comment that part). I looked at that link, it seems it uses PyObject_CallObject which I also used. this also returns a null object. — Hossein
– Hossein, Commented Mar 25, 2020 at 8:07
I agree concerning PyObject_CallObject(). I've no real idea what could be broken as well. In our code (C++ extensions for Python which is in turn used as "macro engine" in our C++ appl.), I call the "constructor" functions directly and never faced such problems. What if the constructor fails in your case. (Thrown exceptions are usually signaled in Python C code by returning NULL.) I would try with an even simpler MCVE if I was in your situation. (Something like this: SO: How to list all function names of a Python module in C++?.) ;-) — Scheff's Cat
– Scheff's Cat, Commented Mar 25, 2020 at 8:25
Thanks a lot. I'll have a look at it. really appreciate your kind help — Hossein
– Hossein, Commented Mar 25, 2020 at 10:46

Hossein · Accepted Answer · 2020-03-25 12:18:25Z

As it turns out(many thanks to @Scheff), the reason why the object was null, was because an exception was happening inside the constructor and thus resulted in the object being null. The reason for the exception was a malformed path and problematic (wrong) arguments.
basically I had set the arguments like this :

auto model_name = "r189";
auto model_checkpoint_path = PyUnicode_DecodeFSDefault("L:\\test_procs\\trained_models\BEST_checkpoint_r189.tar");
auto align_fn = Py_None;
auto img_bank_folder_root = PyUnicode_DecodeFSDefault("L:\\test_procs\\data\\facebank_2");
auto cache_folder = Py_None;
auto postfix = Py_None;
auto rebuild_cache = Py_False;
auto use_jit = Py_False;
auto threshold = 65; 
auto device = "cpu";

and the arguments thus like this :

auto args = Py_BuildValue("sOOOOONNiN", model_name, model_checkpoint_path, align_fn,
                                  img_bank_folder_root, cache_folder, postfix, 
                                  rebuild_cache, use_jit, threshold, device);

which was wrong. I changed them to this :

auto model_name = "r189";
auto model_checkpoint_path = "L:\\test_procs\\trained_models\BEST_checkpoint_r189.tar";
auto align_fn = "";
auto img_bank_folder_root = "L:\\test_procs\\data\\facebank_2";
auto cache_folder = "";
auto postfix = "";
auto rebuild_cache = 0;
auto use_jit = 0;
auto threshold = 65; 
auto device = "cpu";

and simplified the args like this :

auto args = Py_BuildValue("ssssssiiis", model_name, model_checkpoint_path, align_fn,
                          img_bank_folder_root, cache_folder, postfix, 
                          rebuild_cache, use_jit, threshold, device);

after doing all of this, I still faced the exception and found out : model_checkpoint_path needed a fixed as its path lacked a single \ in its path:

auto model_checkpoint_path = "L:\\test_procs\\trained_models\BEST_checkpoint_r189.tar";

fixing it :

auto model_checkpoint_path = "L:\\test_procs\\trained_models\\BEST_checkpoint_r189.tar";

thus solved everything.

Collectives™ on Stack Overflow

PyEval_CallObject and PyObject_CallObject return a null object

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related