Set up virtualenv with Paramiko SSH

Question

I have some limited experience with Python and Django in Windows, and now I am trying to understand how to deploy my code to an Ubuntu 16.04 LTS VPS. Having read various tutorials and a lot of answers on SE, I managed to proceed pretty far (well, for me), but now I am stuck.

Manually (via Putty) I can do the following:

# check that Python 3.5 is installed
python3 --version  
# install pip
sudo -kS apt-get -y install python3-pip  
# upgrade pip to newest version
pip3 install --upgrade pip
# check result
pip3 --version  
# install venv
sudo -kS pip3 install virtualenv virtualenvwrapper 
# create venv
virtualenv ~/Env/firstsite  
# make sure venv is created 
ls -l ~/Env/firstsite/bin/python  # /home/droplet/Env/firstsite/bin/python3.5 -> python3
# switch on venv
source ~/Env/firstsite/bin/activate  # (firstsite) droplet@hostname:~$
# check that python3 is taken from venv
which python3  # /home/droplet/Env/firstsite/bin/python3

So the virtual environment is properly created and switched on. I could proceed installing Django.

However when I am trying to do exactly the same in the automated regime, using Paramiko (I execute commands using paramiko.SSHClient().exec_command(cmd, input_string, get_pty=False), everything goes exactly the same way, until the last command:

exec_command('which python3')

returns /usr/bin/python3. So I assume source activate doesn't work via Paramiko's SSH.

Why?
How can I cope with it?
Can I check that the venv is enabled in some more direct (and reliable) way?

Ranvijay sachan · Accepted Answer · 2019-11-04 07:38:38Z

2

We can easily activate the virtualenv and execute commands on same.

Example:

import paramiko

hostname = 'host'
port = 22
username = 'root'
password = 'root'
s = paramiko.SSHClient()
s.load_system_host_keys()
s.set_missing_host_key_policy(paramiko.AutoAddPolicy())
s.connect(hostname, port, username, password)
command = 'source /root/Envs/env/bin/activate;python3 --version;qark;echo hello'
(stdin, stdout, stderr) = s.exec_command(command)
for line in stdout.readlines():
    print(line)
for line in stderr.readlines():
    print(line)
s.close()

answered Nov 4, 2019 at 7:38

Ranvijay sachan

2,4424 gold badges32 silver badges53 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

WhyWhat Over a year ago

I disagree with it is possible to "easily activate", because the mere possibility of prefixing each inner command with source <venv_path>/bin/activate; <cmd> is not easy. It would be ideal for me if the source ... command could be executed once and then every command via paramiko would be typed within the virtual env. This does not seem to be possible. :(

Novice · Accepted Answer · 2022-03-08 13:38:42Z

2

If you are using anaconda and creating your virtual environments that way, I found a work around. Taken from [this github page][1] I use send the following command to my remote pc through paramiko

f'source ~/anaconda3/etc/profile.d/conda.sh && conda activate {my_env} && {command}'

I also wish you could just activate a venv and then all the following commands would be in the venv, but this work around is nice since the only thing I have to change is the venv name. Since everythnig is in one line, it executes perfectly and I don't need to reactivate anything. If you just have a wrapper function in python it makes it all very easy to use and read. Something like this:

def venv_wrapper(command, ssh, venv=None):
    if venv:
        conda_location = 'source ~/anaconda3/etc/profile.d/conda.sh'
        activate_env = f'conda activate {venv}'
        command = f'{conda_location} && {activate_env} && {command}'
    ssh.exec_command(command, get_pty=True)

I just send all of my commands through this code (which is a little more developed/complicated in my own toolkit) whether or not im using a venv. Works pretty nicely so far [1]: https://github.com/conda/conda/issues/7980

answered Mar 8, 2022 at 13:38

Novice

9151 gold badge8 silver badges18 bronze badges

1 Comment

Ray Hulha Over a year ago

This works well with miniconda too!!

aledj2 · Accepted Answer · 2017-06-07 12:39:46Z

1

Taken from @Pablo Navarro's answer here :How to source virtualenv activate in a Bash script helped me with this same issue (activating environments in a paramiko ssh session).

In the exec_command give the path to the python executable within the environment eg:

stdin, stdout, stderr = ssh.exec_command(/path/to/env/bin/python script.py)

In my case (using miniconda and a env called pyODBC):

stdin, stdout, stderr = ssh.exec_command(~/miniconda2/envs/pyODBC/bin/python run_script.py)

running the command ~/miniconda2/envs/pyODBC/bin/python -m pip list printed the list of modules in this env to confirm

answered Jun 7, 2017 at 12:39

aledj2

531 silver badge9 bronze badges

Collectives™ on Stack Overflow

Set up virtualenv with Paramiko SSH

3 Answers 3

1 Comment

1 Comment

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related