How to have a Python Script run constantly in the background connected to a Node.JS server

Question

Currently I have a Node.JS server that needs to run a python script. Part of this script imports some data from a server that takes time which is detrimental to the experience as this Python script needs to be ran frequently.

What I want is for the Node.JS server to run this Python script when the server is ran, then in the background constantly have it keep running, with the ability to (from the Node.JS server) call a python function that returns data.

So far I have this on the Node.JS server that runs a python script and outputs the response. This is repeated every time data is needed to be retrieved:

const util = require('util'); var options = { mode: 'text', args: [sentenceToUse] };

const readPy = util.promisify(PythonShell.run);
const test = await readPy("spacyTest.py", options);


    var response = test.toString();
    response = response.toString();
    response = response.replace(/'/g, '"');
    response = JSON.parse(response);
return(response);
console.log(test);

'''

How can I keep this running in the background without restarting the python script every time data is needed?

It seems you need to change the python script itself to keep running and respond to requests from its parent. If the python script now runs and exits, there's nothing you can do from node.js to stop that. You need to change the python script. Yhe python script can regularly send data back to its parent on stdout or you can put a little local http server in the python script and have your nodejs parent communicate with that to make requests and get responses. The Python-shell module also shows you how to communicate between node.js parent and the python script, so that is an option too. — jfriend00
– jfriend00, Commented May 31, 2020 at 15:14
I had the idea of having the Python script communicate with it using Socket.IO like the clients do. However I don't want to have to hard code any addresses that I will need to change since I am constantly changing from LocalHost to other external permanent hosting. Would running the Python script as a http server allow me to do so in such a way that I wouldnt have to either the node or python script when switch hosting? — Max Kenney
– Max Kenney, Commented May 31, 2020 at 15:20
Since the python script and your nodejs server are running on the same host, you can just use localhost, no need to hard-code any IP addresses. That would work for a socket.io connection or a web server connection. And, that will work no matter where you move it as long as they remain running on the same host. Since there are two built-in communication mechanisms already `stdio/stdout' and built-n intperprocess messaging, I would start with one of those until they aren't sufficient. — jfriend00
– jfriend00, Commented May 31, 2020 at 15:25
Do you need Node? Could you not use Flask and just have everything in Python? — OneCricketeer
– OneCricketeer, Commented May 31, 2020 at 15:27
Its a Node.JS socket.io application. Part of it just needs to use a Python API which currently is taking about 8 seconds per call which, while it works, isnt ideal so I want to improve it. But the responses on this post have been really helpful and I think I can get it done now! Thanks a lot :) — Max Kenney
– Max Kenney, Commented May 31, 2020 at 15:39

jfriend00 · Accepted Answer · 2020-05-31 15:26:26Z

1

It seems you need to change the python script itself to keep running and respond to requests from its parent. If the python script now runs and exits, there's nothing you can do from node.js to stop that. You need to change the python script.

Here are some options:

The python script can regularly send data back to its parent on stdout and you can read that data from nodejs as it arrives.
You can put a little local http server or socket.io server in the python script on a known port and have your nodejs parent communicate with that to make requests and get responses.
The Python-shell module also shows you how to communicate between node.js parent and the python script here in the doc, so that is an option too.

Since options 1 and 3 are built-in already, I would probably start with one of those until you find some reason they aren't sufficient.

edited May 31, 2020 at 15:26

answered May 31, 2020 at 15:21

jfriend00

711k104 gold badges1.1k silver badges1k bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Max Kenney Over a year ago

This and the response from Jfriend00 have been exactly what I am looking for. Thanks a lot!

Collectives™ on Stack Overflow

How to have a Python Script run constantly in the background connected to a Node.JS server

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related