Running non-python scripts remotely using PyCharm

Question

I am using PyCharm to do remote deployment and execution of python on an SSH server. However, I would also like to be able to run other files directly in the same way. For example, I would like to "run" a "job.run" script through sbatch to submit it to a HPC server.

I can see PyCharm runs the following type of command for python

ssh://username@server:22/home/username/anaconda3/bin/python -u /home/username/project1/main.py

I would like to have another file called "job.run" that runs as follows

ssh://username@server:22/home/username/bin/sbatch /home/username/project1/job.run

This feels like it should be simple to do, however I cannot find any options that allow me to do so.

did you try to run ssh -p 22 username@server and see if you have ssh console? if you want, you can create custom Run/debug command and then you can run it directly from pycharm — matan h
– matan h, Commented Apr 17, 2022 at 15:37
Did you setup SLURM before trying to run the script like in this "submitting jobs" example? I think the IDE dialogues you need to follow are the ones in Use Built-in SSH terminal and remote SSH external tools but I don't have a SLURM server to test this. — bad_coder
– bad_coder, Commented Apr 19, 2022 at 10:26

damienfrancois · Accepted Answer · 2022-04-19 14:09:18Z

4

+50

One option is to configure a specific interpreter as explained here where you make sure to set Interpreter: /usr/bin/bash in the dialog window of step 6. Then you should set that interpreter for the project with the job.run file following this doc. You might need two distinct projects with different profiles/interpreter for that ; one with the Python code, another with the Slurm submission scripts.

Another option, to keep everything inside the same project, could be to wrap the submission of jobs in Python code, either using

os.system("sbatch job.run") or the more versatile subprocess package
the PySlurm package
the Slurm Rest API, following the steps explained in this answer

answered Apr 19, 2022 at 14:09

damienfrancois

60.5k9 gold badges116 silver badges128 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

sol25 · Accepted Answer · 2022-04-22 09:21:35Z

1

I think you could try using External (Remote) Tools option. It allows you to create context actions that you can then trigger from Project file view in PyCharm or assign to keymap.

Take a look at this documentation section: https://www.jetbrains.com/help/pycharm/configuring-third-party-tools.html#remote-ext-tools.

This should hook to your existing SSH server configuration that you are using for executing Python scripts.

answered Apr 22, 2022 at 9:21

sol25

1491 silver badge8 bronze badges

Collectives™ on Stack Overflow

Running non-python scripts remotely using PyCharm

2 Answers 2

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related