Collecting variables from multiple python scripts in one parent script

Question

I try to write a python-parent-script, that collects data from 4 child-scripts. What I got:

Every child-script reads data from a different sensor and they have to be read continiously. So what I do right now is read them in while True loops.
Different sensors have different response times, so one child-script is reading with a, lets say, once a second rate while another will read 100 times faster.

My goal/my struggle:

Collect all the child-generated data in one script, in 4 different variables

What I acheved yet:

Child-scripts are doing there work fine and reading the data with no issues
I could start all 4 child-scripts from terminal as subprocesses but no idea how to collect ther generated data
Pass data between scripts but never from two scripts at the same time and way to slow, since the 'from script import variable' is as fast as the reading of the sensor.

Later plans are sending those 4 variables via Bluetooth to my phone, whitch I successed allready with only one sensor.

Since I am quite new to the whole Raspberry/Python community I would firstly say sorry for unspecific explenation. Please feel free to ask for further informations or suggest to solve things differently. And secondly I would appreceate it a lot if you could help me with code-snippets if you like, because again I am quite new and that helps me way more then links to librarys or documenturies that create more questions than answearing them.

Thank you a lot in advance

Joel · Accepted Answer · 2017-08-18 20:10:03Z

0

The easiest way to accomplish this would be using threads, available through the threading class. Collecting data would be as simple as writing to a shared variable (with thread safety of course).

import threading
import time

data1 = []
lock1 = threading.Lock() # Use a lock, mutex, or semaphore to ensure thread safety

def foo():
    while True:
        lock1.acquire()
        data1.append(bar)
        lock1.release()
        time.sleep(10)

 foo_thread = threading.thread(target=foo)
 foo_thread.start()

 while True:
     lock1.acquire()
     # If all values don't need to be held in memory the parent
     # script can implement some kind of queue and delete values 
     # after processing
     do_stuff_with(data)
     lock1.release()
     # Queuing also allows the parent script to run at a much slower rate
     time.sleep(100)

The example above is a very quick demonstration of threading. There are better data types to implement queuing in, such as collection.deque.

Threading in python does have some caveats though, namely python runs scripts in one single process and there is a Global Interpreter Lock, meaning the threading module provides concurrency, but not parallelism. For only 4 data collections it probably would cause any issues, but if better performance is needed multiprocessing provides tools for spawning multiple python processes allowing for true parallelism.

answered Aug 18, 2017 at 20:10

Joel

3582 silver badges10 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Herr Nilbuil Over a year ago

Thank you so mutch for the detailled answear an for your effort, I will try that and get back to you if it worked.

Collectives™ on Stack Overflow

Collecting variables from multiple python scripts in one parent script

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related