Efficient execution and output stream redirection of process spawned with Runtime.exec()

Question

I have a script which executes a program several times, producing about 350 lines of output to both STDERR and STDOUT. Now, I need to execute the script in Java, thereby printing the output streams to their original destinations. So, basically, I execute the script from inside a Java class, maintaining the original behavior for the user.

The way I do this is inspired from suggestions like Reading streams from java Runtime.exec and, functionally, works fine.

Process p = Runtime.getRuntime().exec(cmdarray);
new Thread(new ProcessInputStreamHandler(p.getInputStream(), System.out)).start();
new Thread(new ProcessInputStreamHandler(p.getErrorStream(), System.err)).start();
return p.waitFor();

And the class ProcessInputStreamHandler:

class ProcessInputStreamHandler implements Runnable {
    private BufferedReader in_reader;
    private PrintStream out_stream;

    public ProcessInputStreamHandler(final InputStream in_stream, final PrintStream out_stream) {
        this.in_reader  = new BufferedReader(new InputStreamReader(in_stream));
        this.out_stream = out_stream;
    }

    @Override public void run() {
        String line;
        try {
            while ((line = in_reader.readLine()) != null) {
                out_stream.println(line);
            }
        } catch (Exception e) {throw new Error(e);}

        out_stream.flush();
    }
}

Now regarding my problem statement: While the execution of the script takes about 17 seconds, the "encapsulated" execution takes at least 21 seconds. Where do I lose these 4 or more seconds?

I already tried using a ProcessBuilder with redirection of STDERR to STDOUT, using POSIX vfork with libraries like https://github.com/axiak/java_posix_spawn, using a byte buffer instead of a BufferedReader... everything with no positive result at all.

Are there any suggestings? I understand that there will be some performance loss, but 4 seconds seem to be a bit much to me...

Appreciate any suggestions!

Best Regards and Thanks in Advance.

I daresay 4 seconds would be the JVM warmup time: e.g. loading jvm executable into memory, loading standard library, loading classes, etc. — Tadas S
– Tadas S, Commented Oct 29, 2013 at 10:37
I would guess it is the the additional scheduling that you introduced using your threads which might be slowing you down. Did you try having your script write to a file instead of STDERR and STDOUT? Then you could run the Java script wrapper without the ProcessInputStreamHandler objects to check whether that still takes an additional 4 seconds. If this is the case non-blocking I/O (nio) might help you. — cyon
– cyon, Commented Oct 29, 2013 at 10:48

Holger · Accepted Answer · 2013-10-29 12:24:56Z

5

The fastest way for your task is to use Java 7 and

return new ProcessBuilder(cmdarray).inheritIO().start().waitFor();

If that doesn’t help, I think there’s nothing you can do as every other approach would add even more code to your runtime environment that has to be processed.

answered Oct 29, 2013 at 12:24

Holger

301k43 gold badges483 silver badges827 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

dsteinhoefel Over a year ago

Nice one-liner, certainly good to clean up the code. Unfortunately, my test revealed that it does not provide any performance benefits...

Holger Over a year ago

The important point is the inheritIO() as it does not require you to execute any Java code for the output redirection but let the operating system handle this. If this feature does not improve the performance, the reason for the performance problem does not lie in this code at all.

dsteinhoefel Over a year ago

The delay probably stems from loading the "outer" program into the JVM. I made further measurements which support this view. Thanks for all suggestions and the Java 7 hint which was new to me.

brettw · Accepted Answer · 2013-11-20 03:34:23Z

1

Don't know if it will improve performance or not, but you can try the NuProcess library which while also providing non-blocking (asynchronous) I/O will also use vfork on Linux, which does decrease process launch times (and memory overhead) quite a bit.

answered Nov 20, 2013 at 3:34

brettw

11.1k2 gold badges47 silver badges62 bronze badges

Collectives™ on Stack Overflow

Efficient execution and output stream redirection of process spawned with Runtime.exec()

2 Answers 2

3 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

3 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related