How to embed a python code which sets flags in C++

Question

I couldn't find a way to embed a python code which sets flags in C++ code at https://docs.python.org/3.5/extending/embedding.html.

I have this python code test.py:

import tensorflow as tf

# Settings
flags = tf.app.flags
FLAGS = flags.FLAGS

#core params..
flags.DEFINE_string('model', 'gcn', 'model name')
flags.DEFINE_float('learning_rate', 0.01, 'initial learning rate')
flags.DEFINE_string("model_size", "small", "define model size")


def main(argv=None):
    print("Flags Set")
    print(FLAGS.learning_rate)

if __name__ == '__main__':
    tf.app.run()

when I execute the command:

python -m test --learning_rate 0.0002

The output is:

Flags Set
0.0002

How can I embed the above python code in C++ and invoke it?

Your first sentence is a bit strange as you link the exactly doc. page which explains how to do it. C++ code is usually compiled to binary (although it might be interpreted as well as I recently learnt). Python code is usually interpreted by a Python interpreter (although there might be compilers like e.g. IronPython). Hence, this is what the doc. describes: Embedding a Python interpreter in a C or C++ program. The C++ code may also extend the Python by additional types, variables, and functions for interop. of Python and C++. — Scheff's Cat
– Scheff's Cat, Commented Apr 8, 2019 at 5:35
According to the doc's description, the embedded python code functions need to be invoked via ./executable <python_module> <function> <arguments of the function>. But when I executing my python code separately I want to give flag values (eg:--learning_rate 0.0002). I couldn't find a way to set those flags when I embed the python code in C++ — Anuradha Karunarathna
– Anuradha Karunarathna, Commented Apr 8, 2019 at 6:02
For a start, have a look at this: SO: How to access a Python global variable from C?. (I googled with google "find a specific python variable c c++" to find this.) I must admit, I needed a while to combine C++ and Python properly (and a while longer to get it stable) but I used exclusively Python doc. or google research to master it. With a bit patience, it's possible. ;-) — Scheff's Cat
– Scheff's Cat, Commented Apr 8, 2019 at 6:08
May be, I mis-understood your question. If you ask how to pass command line arguments via PyRun_ functions -> I found this: SO: Pass argument to PyRun_File(***) linking to this Blog article: Passing Parameters in From C to Pyhton.. — Scheff's Cat
– Scheff's Cat, Commented Apr 8, 2019 at 6:19
docs.python.org/3.5/extending/embedding.html is about executing a function in python script not the python script itself. — user10704605
– user10704605, Commented Apr 8, 2019 at 7:46

score 1 · Accepted Answer · 2019-04-23 07:56:34Z

1

You have two options,

1. Simply execute the python script with system(),

system("python -m /path/to/test.py  --learning_rate 0.0002");

2. Use Python/C Api as follows,

#include <python3.6/Python.h>
#include<iostream>
using namespace std;

int main(int argc, char *argv[])
{
    FILE* file;
    wchar_t *program = Py_DecodeLocale(argv[0], NULL);
    wchar_t** _argv;
    for(int i=0; i<argc; i++){
        wchar_t *arg = Py_DecodeLocale(argv[i], NULL);
        _argv[i] = arg;
    }
    Py_SetProgramName(program);
    Py_Initialize();   
    PySys_SetArgv(argc, _argv);
    file = fopen("/path/to/test.py","r");
    PyRun_SimpleFile(file, "/path/to/test.py");
    Py_Finalize();
    return 0;
}

If you get an executable in a.out, you can run it like below,

./a.out --learning_rate 0.0002

Note:- I have Python.h in python3.6m folder and I used a flag -lpython3.6m for compilation.

edited Apr 23, 2019 at 7:56

answered Apr 8, 2019 at 10:14

user10704605

Sign up to request clarification or add additional context in comments.

3 Comments

Anuradha Karunarathna Over a year ago

Thanks a lot. I prefer the second option. However, I did a small change to the above code. I put Py_Initialize(); as the first line of the main method. Otherwise, I got ImportError: No module named 'tensorflow'

user10704605 Over a year ago

I would have tested it if I had tensorflow, glad it worked for you.

Anuradha Karunarathna Over a year ago

if I take this main method content into a separate function and invoke that function from two threads, I end up with a segmentation fault. Do you have an answer for that?

Collectives™ on Stack Overflow

How to embed a python code which sets flags in C++

1 Answer 1

3 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related