Why can't the Python command "subprocess.Popen" find the jar file to run?

Question

I'm trying to run code from this repo: https://github.com/tylin/coco-caption, specifically from https://github.com/tylin/coco-caption/blob/master/pycocoevalcap/tokenizer/ptbtokenizer.py, line 51-52:

p_tokenizer = subprocess.Popen(cmd, cwd=path_to_jar_dirname, \
            stdout=subprocess.PIPE)

The error I get running this is

OSError: [Errno 2] No such file or directory

I can't figure out why the file can't be found.

The jar I'm trying to run is:

stanford-corenlp-3.4.1.jar

You can see the structure of directory by going to https://github.com/tylin/coco-caption/tree/master/pycocoevalcap/tokenizer. For more specificity into what my actual arguments are when I run the line of code:

cmd= ['java', '-cp', 'stanford-corenlp-3.4.1.jar', 'edu.stanford.nlp.process.PTBTokenizer', '-preserveLines', '-lowerCase', 'tmpWS5p0Z'],

and

path_to_dirname =abs_path_to_folder/tokenizer

I can see the jar that needs to be run, and it looks to be in the right place, so why can't python find it. (Note: I'm using python2.7.) And the temporary File 'tmpWS5p0Z' is where it should be.

Edit: I'm using Ubuntu

ralf htp · Accepted Answer · 2017-10-05 20:13:36Z

3

try an absolute path ( meaning the path beginning from root / )

https://en.wikipedia.org/wiki/Path_(computing)#Absolute_and_relative_paths

for relative paths in python see i.e. Relative paths in Python , How to refer to relative paths of resources when working with a code repository in Python

UPDATE:

As a test try subprocess.Popen() with the shell=True option and give an absolute path for any involved file, including tmpWS5p0Z

in this subprocess.Popen() call are involved two paths :

1) the python path, python has to find the java executable and the stanford-corenlp-3.4.1.jar which is essentially a java program with its own path

2) the java path of stanford-corenlp-3.4.1.jar

as this is all too complicated try

p_tokenizer = subprocess.Popen(['/absolute_path_to/java -cp /absolute_path_to/stanford-corenlp-3.4.1.jar /absolute_path_to/edu.stanford.nlp.process.PTBTokenizer -preserveLines -lowerCase /absolute_path_to/tmpWS5p0Z' ], shell=True)

Python specify popen working directory via argument

Python subprocess.Popen() error (No such file or directory)

edited Oct 5, 2017 at 20:13

answered Oct 2, 2017 at 21:15

ralf htp

9,4805 gold badges25 silver badges35 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

lars Over a year ago

I've updated my question, but my path_to_dirname is already on the absolute path. Updated question to reflect this.

lars Over a year ago

Thanks, your answer helped. Basically, the problem was that I assumed the computer I'm using had java installed. It did not.

too honest for this site Over a year ago

root is /, not `\`

StopReadingThisUsername Over a year ago

Where was your temporary file?

Moir Over a year ago

shell=True was the missing part for me, the relative paths did not seem to resolve without it.

rubencart · Accepted Answer · 2019-06-27 09:07:16Z

Just in case it might help someone:

I was struggling with the same problem (same https://github.com/tylin/coco-caption code). Might be relevant to say that I was running the code with python 3.7 on CentOS using qsub. So I changed

cmd = ['java', '-cp', 'stanford-corenlp-3.4.1.jar', 'edu.stanford.nlp.process.PTBTokenizer', '-preserveLines', '-lowerCase', 'tmpWS5p0Z']

to

cmd = ['/abs/path/to/java -cp /abs/path/to/stanford-corenlp-3.4.1.jar edu.stanford.nlp.process.PTBTokenizer -preserveLines -lowerCase ', ' /abs/path/to/temporary_file']

Using absolute paths fixed the OSError: [Errno 2] No such file or directory. Notice that I still put '/abs/path/to/temporary_file' as second element in the cmd list, because it got added later on. But then something went wrong in the tokenizer java subprocess, I don't know why or what, just observing because:

p_tokenizer = subprocess.Popen(cmd, cwd=path_to_jar_dirname, stdout=subprocess.PIPE, shell=True)
token_lines = p_tokenizer.communicate(input=sentences.rstrip())[0]

Here token_lines was an empty list (which is not the wanted behavior). Executing this in IPython resulted in the following (just the subprocess.Popen(..., not the communicate).

Exception in thread "main" edu.stanford.nlp.io.RuntimeIOException: java.io.IOException: Input/output error
    at edu.stanford.nlp.process.PTBTokenizer.getNext(PTBTokenizer.java:278)
    at edu.stanford.nlp.process.PTBTokenizer.getNext(PTBTokenizer.java:163)
    at edu.stanford.nlp.process.AbstractTokenizer.hasNext(AbstractTokenizer.java:55)
    at edu.stanford.nlp.process.PTBTokenizer.tokReader(PTBTokenizer.java:444)
    at edu.stanford.nlp.process.PTBTokenizer.tok(PTBTokenizer.java:416)
        at edu.stanford.nlp.process.PTBTokenizer.main(PTBTokenizer.java:760)
Caused by: java.io.IOException: Input/output error
    at java.base/java.io.FileInputStream.readBytes(Native Method)
    at java.base/java.io.FileInputStream.read(FileInputStream.java:279)
    at java.base/java.io.BufferedInputStream.read1(BufferedInputStream.java:290)
    at java.base/java.io.BufferedInputStream.read(BufferedInputStream.java:351)
    at java.base/sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284)
    at java.base/sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326)
    at java.base/sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
    at java.base/java.io.InputStreamReader.read(InputStreamReader.java:185)
    at java.base/java.io.BufferedReader.read1(BufferedReader.java:210)
    at java.base/java.io.BufferedReader.read(BufferedReader.java:287)
    at edu.stanford.nlp.process.PTBLexer.zzRefill(PTBLexer.java:24511)
    at edu.stanford.nlp.process.PTBLexer.next(PTBLexer.java:24718)
    at edu.stanford.nlp.process.PTBTokenizer.getNext(PTBTokenizer.java:276)
    ... 5 more

Again, I don't know why or what, but I just wanted to share that doing this fixed it:

cmd = ['/abs/path/to/java -cp /abs/path/to/stanford-corenlp-3.4.1.jar edu.stanford.nlp.process.PTBTokenizer -preserveLines -lowerCase /abs/path/to/temporary_file']

And changing cmd.append(os.path.join(path_to_jar_dirname, os.path.basename(tmp_file.name))) into cmd[0] += os.path.join(path_to_jar_dirname, os.path.basename(tmp_file.name)).

So making cmd into a list with only 1 element, containing the entire command with absolute paths at once. Thanks for your help!

In the same directory as the stanford-corenlp-3.4.1.jar :) so in coco-caption/pycocoevalcap/tokenizer/

jps · Accepted Answer · 2022-01-13 10:37:14Z

0

As @Lars mentioned above the issue I had was that I Java wasn't installed. Solved it with:

sudo apt update    
sudo apt install default-jdk
sudo apt install default-jre

Making this post since I had this issue twice (due to reinstallation problems) and forgot about it.

edited Jan 13, 2022 at 10:37

jps

22.9k18 gold badges92 silver badges121 bronze badges

answered Jan 13, 2022 at 10:28

watermelodic

411 silver badge4 bronze badges

Collectives™ on Stack Overflow

Why can't the Python command "subprocess.Popen" find the jar file to run?

3 Answers 3

5 Comments

2 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

5 Comments

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related