how to stop the current execution of a system call in python

Question

From python I am calling a java function:

os.system("java -jar example.jar run myFunction 'inFile.txt' 'outFile.txt' " )

This function is processing a file and the output is written into 'outFile.txt'. The output is dependent on the information in 'inFile.txt'. While processing the input file and writing into out file, sometimes the 'outFile.txt' grows too large (tens of GBs) and at that time, i want to quit and the current processing and move on to process another inFile.txt

Is there is way to know that my outFile.txt that is being written has grown more than say 10GB.

Edit: As suggested by Maksym, I am using the following code and seems to be working. Thanks

import subprocess
from time import sleep
p = subprocess.Popen(["java", "-jar", "example.jar", "run", "myFunction", "'inFile.txt'", "'outFile.txt'")
rc = p.poll()  #returncode
while (rc == None):
    sleep(1)
    if(os.path.getsize(outFileName) < 1000000000):
        rc = p.poll()
        continue
     else:
        p.kill()
        break

Maksym Polshcha · Accepted Answer · 2013-03-04 20:05:22Z

2

Have a look at subprocess module. Using Popen you can fork a process and kill it when you need this:

import subprocess
from time import sleep
p = subprocess.Popen(["java", "-jar", "example.jar", "run", "myFunction", "'inFile.txt'", "'outFile.txt'")
while not check_my_conditions():
    sleep(my_timeout)
p.kill()

Then, you can rotate your files and restart the process.

edited Mar 4, 2013 at 20:05

answered Mar 4, 2013 at 19:50

Maksym Polshcha

18.4k8 gold badges54 silver badges78 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

user1140126 Over a year ago

I edit my OP. works but the exit condition is not working properly

jeffknupp · Accepted Answer · 2013-03-04 19:36:08Z

1

Instead of directly calling os.system, you should strongly considering using the multiprocessing.Process built-in class. It handles dealing with spawned processes much more gracefully.

You need to watch the output file periodically, either using something like os.stat to check the file size. You can then kill the original process (or whatever you want to do) when the threshold is exceeded.

answered Mar 4, 2013 at 19:36

jeffknupp

6,3643 gold badges30 silver badges29 bronze badges

3 Comments

LAK Over a year ago

Windows, at least, doesn't seem to update the file's 'size' until it's closed. Not sure os.stat would help, if he's on Windows - the submitter does say he couldn't see the file size. Or rather he did; the question seems to have been edited.

jeffknupp Over a year ago

os.stat is available on Windows. See the docs

LAK Over a year ago

I know about os.stat(), but the problem is that st_size is 0 for a file which is currently open and still being written to (again, under Windows; I'm not sure what OS the question applies to).

LAK · Accepted Answer · 2013-03-04 19:35:49Z

0

Does the java application provide any output (for example, a count of records processed) to stdout or stderr while it runs? If so, you could invoke it using Python's Popen class (in the subprocess module) and estimate when it has processed 'too much'.

answered Mar 4, 2013 at 19:35

LAK

9707 silver badges18 bronze badges

1 Comment

user1140126 Over a year ago

it writes output to outFile.txt. It does write to stdout when the processing completes.

Collectives™ on Stack Overflow

how to stop the current execution of a system call in python

3 Answers 3

1 Comment

3 Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

3 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related