312 questions
0
votes
0
answers
68
views
How to run VS Code Python tests inside an external app’s embedded interpreter?
I'm working on Python code that must be executed inside an external application which embeds its own Python interpreter (not the system Python, nor a virtual environment).
Because of this, I can't ...
- 1
2
votes
1
answer
137
views
Correct way to embed and bundle Python in C++ to avoid "ModuleNotFoundError: encodings"?
I am trying to embed Python inside my C++ DLL.
The idea is that the DLL, once distributed, should be sufficient and not rely on other installations and downloads.
Interestingly, the below "sort ...
- 94
0
votes
0
answers
52
views
Error when Importing pandas in embedded python in C++
I'm working with Python 3.10 64-bit，win10，vs2013
I run the python script alone and there are no errors and it also imports the pandas package properly.However, when calling python scripts in c++ (...
- 1
0
votes
0
answers
49
views
PyImport_Import does not import functions
I'm doing some code that allows plugin authors for our software to embed python code in their plugins. And I'm stuck with some strange import behavior.
When I import this simple python file with ...
1
vote
1
answer
1k
views
How to develop a Generalized RAG Pipeline for Text, Images, and Structured Data [closed]
I'm trying to find a general solution for RAG to solve problems involving both text, images, chart, tables,.., they are in many different formats such as .docx, .xlsx, .pdf.
The requirement for the ...
- 515
4
votes
1
answer
7k
views
Ollama with RAG for local utilization to chat with pdf
I am trying to build ollama usage by using RAG for chatting with pdf on my local machine.
I followed this GitHub repo: https://github.com/tonykipkemboi/ollama_pdf_rag/tree/main
The issue is when I am ...
- 321
0
votes
1
answer
341
views
Why is embedding CSV file taking much longer than pdf embedding in LangChain?
I successfully embedded a 400-page PDF document within 1-2 hours. However, when I tried to embed a CSV file with about 40k rows and only one column, the estimated embedding time is approximately 24 ...
- 2,529
0
votes
0
answers
54
views
How to load a python module installed in the executable folder? (C++)
I have an install structure like this:
install_folder\
app.exe
lib.dll
python\
kitsu.py
And I am trying to execute some functions in my application from the file kitsu.py.
My C++ code is like this:
(...
- 21
1
vote
1
answer
2k
views
Chroma DB using embedding values
I tried creating a chromadb using embedding values but its not working. I have my own embedding model that I want to deploy on the server. So here if I pass text as list I am getting its embedded ...
- 11
-2
votes
1
answer
173
views
Embedded Python packages not recognised by Intellisense (Visual Studio Community 2022 Version 17.10.1)
I am creating a Unity project that has python embedded using this tutorial, here. For some reason Inillisense is warning that the installed packages, which used pip, were not reconised. I have made ...
0
votes
1
answer
806
views
sys.path not always/reliably set by sitecustomize -- how can it be reliably overridden?
I have an application using embedded Python, which includes python.exe. I have added a sitecustomize.py to set sys.path, something like:
import sys
sys.path = ['.','.\\python310.zip','.\\Lib','.\\Lib\\...
- 480
0
votes
1
answer
239
views
Does Pytorch automaticly use the GPU
I'm new to using torch and calculate on huge amount of data. I want to create embeddings for an large text corpus and write my embedding function, it works well but it seems pretty slow, so I'm not ...
3
votes
1
answer
200
views
How does this embedded Python packaging find its lib files?
Blender(*) for Windows is shipped with an embedded Python, like this:
blender-2.79b-windows64\
2.79\
python\
bin\
python.exe
python35.dll # ...
- 47.5k
1
vote
1
answer
295
views
Folder structure inside Python "embedded" packaging
I distribute a ready-to-run software for Windows written in Python by:
shipping the content of an embedded version of Python, say python-3.8.10-embed-amd64.zip
adding a myprogram\ package folder (= ...
- 47.5k
2
votes
0
answers
718
views
Python .pth and ._pth files (standard install vs. embedded Python)
I see two diffrent behaviours about .pth and ._pth files:
the official Python install for Windows uses .pth files in lib\site-packages as documented in https://docs.python.org/3.12/library/site.html#...
- 47.5k
6
votes
1
answer
1k
views
How to embed python 3.8+ in a C++ application while using a virtual environment?
This is a self-answered question. I've found that there aren't any good examples of this specific situation, and the seemingly related questions don't address this use case.
My C++ application ...
- 176
0
votes
0
answers
1k
views
How to upload document to pinecone index using langchain modules ask questions in document
I want to upload my document to pincone and ask the questions in the document. I want ask in terminal itself, I know there is streamlit videos are in youtube but i don't want to use Stream lit. I'm ...
- 25
1
vote
2
answers
3k
views
How to copy a collection from one instance to another instance with Qdrant?
In the process that i'm running, i need very low latency to be able to process a job, so i use a local instance of qdrant db to be able to insert everything very fast to it.
After finishing the job, i ...
- 1,979
3
votes
2
answers
182
views
How to retrieve the line number where a C++ function is called from python using pybind11
I'm trying to make a C++ logger class for embedded python script with pybind11. How can I retrieve the line number where a C++ function is called from python?
I have something like this in C++:
class ...
- 33
0
votes
1
answer
904
views
How do i filter and show response from latest file using my PGVector
In langchain embeddings using PGVector if the user asks question and it matched 2 files. I need to show the response with the latest uploaded file. In my embeddings when a file is uploaded i am ...
- 51
2
votes
0
answers
175
views
Embedding Process Stuck While Handling Large CSV
I wanted to use the script below for embedding. It worked fine on a small amount of data, but when I loaded a CSV with 300,000 records into it, the embedding has been running for 40 minutes and doesn'...
- 45
0
votes
0
answers
276
views
google.protobuf.message.DecodeError: Field number 0 is illegal while using tf.keras.layers.InputLayer(input_shape=(4,))
C++ code, I need to build model and use python embedding, but i got this:
int
main(int argc, char *argv[])
{
Py_Initialize();
PyRun_SimpleString("import tensorflow as tf");
...
1
vote
0
answers
208
views
ImportError with embedded python module when linked to larger shared library. undefined symbol: PyLong_AsLongLongAndOverflow
I wrote a C wrapper to access the ezdxf Python package from my C++ program using the embedded interpreter. When I compile the wrapper by itself it works as expected (loads the module below which loads ...
- 31
1
vote
1
answer
95
views
Access application data when extending embedded Python
I am writing an application that embeds a Python interpreter. The application also extends Python with a custom module. This is done along the lines of the "Extending and Embedding the Python ...
- 21
0
votes
2
answers
75
views
Word 2Vec pretrained embedding KeyError with Gensim==4
I'm traying to use a pretrained word2vec model for Arabic language
the code suppose to be written as following
unknownArray=[]
# load the whole embedding into memory
w2v_embeddings_index={}
...
- 11
1
vote
0
answers
126
views
PyRun_File and PyImport_Import not compatible while embedding Py into C++
I am making a C++ app with embedded Python.
I use *PyImport_Import *to import module and PyObject_GetAttrString to import function from it.
The problem is that if .py file was imported with *...
- 11
0
votes
1
answer
52
views
Running arbitrary Python code from Python and preserving return values
I am attempting to write an API in Python that will allow me to run arbitrary Python code. In particular, I would like to be able to call any function through the API and have the return value passed ...
2
votes
0
answers
352
views
Importing local module from an embedded installation of Python
I'm trying to make a little program that sets up a portable python 3 with specific packages, with no admin rights required.
(The point of this is to execute some specific scripts with more ease and in ...
- 1,354
1
vote
0
answers
464
views
Importing python file to c++ program - ModuleNotFoundError
Following this thread
Assume I have the following files in the directory prog:
main.cpp (located in the directory prog)
int main(int argc, char *argv[]){
Py_Initialize();
//Adding current path
...
- 119
1
vote
2
answers
1k
views
Importing modules in embedded python
I'm trying to get module imports to work in embeddable python, but it doesn't want to work
C:\Users\test\Desktop\winpy\python-3.10.10-embed-win32>type run_scripts\script.py
from module_test import ...
- 23
0
votes
1
answer
46
views
Can not run the code of this repository - NETL-Automatic-Topic-Labelling-
I am trying to run this code - Automatic Labelling of Topics with Neural Embeddings
The problem is that they did not mention what versions they used for the libraries and tools they used. Sadly, not ...
- 1,064
1
vote
1
answer
658
views
How to kill py::scoped_interpreter guard{} in pybind11
How to destroy python interpreter in pybind11 to call python function from c++ in loop
Here am getting output for the first time ,when it's come for 2nd time loop it's throwing Microsoft C++ exception:...
- 11
0
votes
1
answer
567
views
Python `multiprocessing` spawned process using base Python, not virtualenv Python
On a standard installation of python (e.g. via miniconda), I run this script (also pasted below) and I get the following output:
python test_python_multiprocessing.py
arg1: called directly
sys....
- 4,515
1
vote
1
answer
234
views
Embed Python source code in C++ as string
I'm writing a C++ program that requires Python (3.11) code to be embedded into it and am using Python.h to try and accomplish this. The general idea is that my a python script, which will be stored by ...
- 150
0
votes
1
answer
589
views
Embedded Python: fatal error in sub-interpreter with C++ multi-thread when cleanup
I'm trying to use Sub-interpreter for having distinct environment, and having multi-thread on same environment(interpreter).
However, when I tried to cleanup sub-interpreter, Python raise Fatal error ...
- 33
3
votes
1
answer
1k
views
How can I check if a thread holds the GIL with sub-interpreters?
I am working on some changes to a library which embeds Python which require me to utilize sub-interpreters in order to support resetting the python state, while avoiding calling Py_Finalize (since ...
- 4,831
3
votes
2
answers
868
views
Prevent a Python-embedded to look in my default path C:\Python38 for modules
I'm using Cython in --embed mode to produce a .exe. I'm evaluating the Minimal set of files required to distribute an embed-Cython-compiled code and make it work on any machine. To do this, I only ...
- 47.5k
2
votes
0
answers
204
views
Python: Is there a way to bundle an application (with PyInstaller or similar) and after instruct the bundle to use a venv?
so I have a kind of pretty special problem.
I want to bundle an application, for Windows for now. PyInstaller seems best so far.
BUT when delivering the bundle, a persistent venv shall be created by ...
- 21
0
votes
1
answer
206
views
Segmentation Fault while using Python lib in C with Python.h
This is my first time asking a question on stack overflow so please bear with me
I am trying to create a calculator in c as a project but I am getting a segmentation fault when evaluating a algebraic ...
0
votes
1
answer
695
views
Py_InitializeEx(0) fails after Py_Finalize (after previous initialization) [duplicate]
The question is about Python.h (embedded Python 3.10). Here in this minimal example where I Py_Initialize, import numpy and then Py_DecRef it. After that I Py_Finalize and repeat the process one more ...
- 489
1
vote
1
answer
805
views
Memory Management while embedding python in C# using python.net
I am trying to understand how memory is managed when python is embedded in C# using python.net. Python is executed in C# using the py.GIL (global interpreter lock), essentially C# calls the python ...
- 927
0
votes
2
answers
2k
views
Is it possibile to embed python libraries in C++?
Is it possibile to use python libraries in C++ like selenium , django etc ....? If yes are there any docs that explain this well like fully embedding a python library in C++ not just some run script ...
- 259
-2
votes
1
answer
154
views
How can i call a Python Function from C++ in the same instance succesfully?
hey so im trying to embed Python Selenium in C++ , but i managed it to just open the browser and get a item. How can i succesfully open the browser and search HELLO inside the google searchbar?
#...
- 259
1
vote
1
answer
534
views
Which process runs python script running through python.net in C#
I have the following script.
try
{
PythonEngine.Initialize();
using (Py.GIL())
{
using (PyScope scope = Py....
- 927
1
vote
1
answer
2k
views
How to release GIL after Py_Initialize?
I am trying to embed python in a GUI application, and so I have a the python interpreter initialized in the main thread of the application, and I may launch python scripts from another one.
I read ...
- 1,381
0
votes
1
answer
1k
views
PyObject_CallMethod: call static member function from c++
I've embedded Python (3.6) code into my C++ application successfully. I use the Python/C API to call methods from this library. An overview of the python module is as follows:
class MyClass(object):
...
- 21
1
vote
1
answer
1k
views
Py.GIL() is giving error pythonnet embedded python in C#
I have the following C# code:
static void Main()
{
string pythonpath1 = @"C:\Users\user\Documents\pynet_test\Python\Python37";
string pythonpath2 = @"C:\...
- 927
0
votes
0
answers
181
views
embedding python interpreter fails with lots of linker errors
After building cpython:
./configure --with-openssl=$(brew --prefix openssl) --enable-optimizations --prefix=/path/to/build
make
make install
I want to compile an application with an embedded python ...
- 388
0
votes
1
answer
2k
views
Using PyBind11 with a virtual environment created at runtime
My goal is to embed the python interpreter using PyBind11, but to use the interpreter from a virtual env, such that installing dependencies using pip does not clutter the system paths.
There is not ...
- 16.2k
1
vote
1
answer
556
views
Why PyCallable_Check() returns 0 on global class instances?
Rigth now I'm working on embedding python into a larger C++ application. Despite I'm not a python specialist, I understand that with the builtin PyCallable_Check() I can check if a python object is ...
- 35