How to write data to a file continuously without blocking main loop in Python

Question

I am new to Python and I have a scenario where I want to write measurement data to 2 files, one file within the main() function and another file within a function WriteToFile() which writes a new line of data every 10 seconds. How can I make WriteFile() get fired once in the main() loop and then run until 2 minutes after the main() loop iteration completes? How can I then exit WriteToFile() gracefully in CleanUp(), ensuring that FastFile.txt has closed properly? Here is a simplified program example:

from datetime import datetime
import os
import time
import sys
import random

M_Max = 2000
M_Array = [740] 
D_Array = [5, 10, 20, 100, 150, 200, 400,600, 740]  



M=random.random()*20000
D=random.random()*7500


#write data to a file every ten seconds  
def WriteToFile():  
    while True:
        h = open("FastFile.txt", "a")
        current_time_t=datetime.now().strftime("%d.%m.%y %H:%M:%S")
        p = random.random()*1000
        c = random.random()*100
        
        print("Fast_Time :", '{} {:9.0f} {:9.0f} {:9.3f} {:9.1f}  '.format(current_time_t, M, D, p, c))

        h.write("\n %s , %s , %s , %s, %s  " % (current_time_t, str('{:.1f}'.format(M)), str('{:.1f}'.format(D)), str('{:.3f}'.format(p)), str('{:.2f}'.format(
        c))))
        h.close()
        time.sleep(10)

    
        
    

def CleanUp():
    print("Cleaning up")
    
    #wait two minutes then stop WriteToFile()
    time.sleep(120)
    sys.exit()
    
    print("Done")
    


def main():
    global  M, D
   
    WriteToFile()  # call this function and start writing data including the global variables M and N generated here

    print("Starting measurement")
    time.sleep(3)
    
    for M in M_Array:

        for D in D_Array:  
            
            p = random.random()*1000
            c = random.random()*100

            g = open("SlowFile.txt", "a")
            

            current_time_t = datetime.now().strftime("%d.%m.%y %H:%M:%S")

            print("Main_Time :", '{} {:9.0f} {:9.0f} {:9.3f} {:9.1f}  '.format(current_time_t, M, D, p, c))
            
            #write the result of each iteration to a file
            g.write("\n %s , %s , %s , %s, %s  " % (current_time_t, str('{:.1f}'.format(M)), str('{:.1f}'.format(D)), str('{:.3f}'.format(p)), str('{:.2f}'.format(
                c))))
            g.close()


    CleanUp()


if __name__ == "__main__":
    main()

I am using Python 3.5.3 but can upgrade to accommodate solutions that require higher versions of Python.

you can use multiprocessing

Ευάγγελος Γρηγορόπουλος
– Ευάγγελος Γρηγορόπουλος

2021-10-21 12:01:51 +00:00
Commented Oct 21, 2021 at 12:01 — Ευάγγελος Γρηγορόπουλος
– Ευάγγελος Γρηγορόπουλος, Commented Oct 21, 2021 at 12:01

Nima · Accepted Answer · 2021-10-21 12:45:25Z

1

You need threading! Your if __name__="__main__" has to be like below:

if __name__ == "__main__":
    t1 = threading.Thread(target=WriteToFile)
    t2 = threading.Thread(target=loop) # put the loop in a new function and call it using this
    t1.start()
    t2.start()
    t1.join()
    t2.join()

Your main() function has to be renamed to loop or anything you like, without the call to WriteToFile(). Remember to import threading at the beginning of the code.

answered Oct 21, 2021 at 12:45

Nima

3894 silver badges14 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Buzz Over a year ago

Thanks very much. This solved my problem.

Collectives™ on Stack Overflow

How to write data to a file continuously without blocking main loop in Python

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related