Parallel SSH with Custom Parameters to Each Host

Question

There are plenty of threads and documentation about parallel ssh, but I can't find anything on passing custom parameters to each host. Using pssh as an example, the hosts file is defined as:

111.111.111.111
222.222.222.222

However, I want to pass custom parameters to each host via a shell script, like this:

111.111.111.111 param1a param1b ...
222.222.222.222 param2a param2b ...

Or, better, the hosts and parameters would be split between 2 files.

Because this isn't common, is this misuse of parallel ssh? Should I just create many ssh processes from my script? How should I approach this?

My current solution is to create ssh processes and background them and wait for them to finish. This works nicely in my case because I can pass the parameters I want, but it makes me have to manage the logs and control them on my own. — Brandon Amos
– Brandon Amos, Commented Dec 3, 2012 at 8:10

Dima Chubarov · Accepted Answer · 2013-01-22 11:54:33Z

4

You could use GNU parallel.

Suppose you have a file argfile:

111.111.111.111 param1a param1b ...
222.222.222.222 param2a param2b ...

Then running

parallel --colsep ' ' ssh {1} prog {2} {3} ... :::: argfile

Would run prog on each host with the corresponding parameters. It is important that the number of parameters be the same for each host.

answered Jan 22, 2013 at 11:54

Dima Chubarov

17.3k7 gold badges45 silver badges85 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Brandon Amos Over a year ago

Great! It's been a while since I've looked at this stuff, but if this works, it looks like the most elegant solution. I didn't mention in my original question that I have an arbitrarily number of servers to pass these parameters to (could be hundreds), and this seems to easily scale. Thanks for sharing!

Ole Tange Over a year ago

If your arguments contain space, then you might want to use TAB (\t) as --colsep (Especially handy if the argfile comes from a spreadsheet).

Ole Tange Over a year ago

If you still want to split the hosts and params into 2 files, then use --xapply, which will read one line from each input source: parallel --xapply ssh {1} prog {2} :::: hosts argfile

Arribah · Accepted Answer · 2013-01-21 14:22:46Z

Here is a solution that you can use, after tailoring it to suit your needs:

#!/bin/bash
#filename: example.sh
#usage: ./example.sh <par1> <par2> <par3> ... <par6>

#set your ip addresses
$traf1=1.1.1.1
$traf2=2.2.2.2
$traf3=3.3.3.3

#set some custom parameters for your scripts and use them as you wish.
#In this example, I use the first 6 command line parameters passed when run the example.sh


ssh -T $traf1 -l username "/export/home/path/to/script.sh $1  $2" 1>traf1.txt 2>/dev/null &
echo "Fetching data from traffic server 2..."
ssh -T $traf2 -l username "/export/home/path/to/script.sh $3  $4" 1> traf2.txt 2>/dev/null &
echo "Fetching data from traffic server 3..."
ssh -T $traf3 -l username "/export/home/path/to/script.sh $5  $6" 1> traf3.txt 2>/dev/null &

#your application will block on this line, and will only continue if all 
#3 remotely executed scripts will complete
wait

Keep in mind that the above requires that you setup passwordless login between the machines, otherwise the solution will break to request for password input.

salva · Accepted Answer · 2013-01-22 10:25:59Z

0

If you can use Perl:

use Net::OpenSSH::Parallel;
use Data::Dumper;

my $pssh = Net::OpenSSH::Parallel->new;

$pssh->add_host('111.111.111.111');
$pssh->add_host('222.222.222.222');

$pssh->push('111.111.111.111', $cmd, $param11, $param12);
$pssh->push('222.222.222.222', $cmd, $param21, $param22);

$pssh->run;

if (my %errors = $ssh->get_errors) {
  print STDERR "ssh errors:\n", Dumper \%errors;
}

answered Jan 22, 2013 at 10:25

salva

10.3k4 gold badges31 silver badges61 bronze badges

Collectives™ on Stack Overflow

Parallel SSH with Custom Parameters to Each Host

3 Answers 3

3 Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

3 Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related