how to execute multiline shell command in perl?

Question

I want to run shell command as follow in perl:

tar --exclude="*/node_modules" \
--exclude="*/vendor" \
--exclude='.git' \
-zvcf /tmp/robot.tgz .

But it seems perl can not excute this:

`tar --exclude="cv/node_modules" \
--exclude="*/vendor" \
--exclude='.git' \
-zvcf /tmp/robot.tgz .`;

Here is the error:

tar: Must specify one of -c, -r, -t, -u, -x
sh: line 1: --exclude=*/vendor: No such file or directory
sh: line 2: --exclude=.git: command not found
sh: line 3: -zvcf: command not found

it seems perl treat each line as one command.

Why not run it as a one-line command instead of a multi-line command. Perl handles long strings fine. In any case, you need to double up the backslashes since Perl strips them. — Jonathan Leffler
– Jonathan Leffler, Commented Apr 26, 2016 at 3:08
@JonathanLeffler yes, you are right, I handle this in one line ok, but I still try to do this because one line are to long, sometimes, it is not readable. — roger
– roger, Commented Apr 26, 2016 at 3:10
You can always build the one long line by concatenating multiple shorter strings, which doesn't have the readability problems. But it is also valid to make the request. — Jonathan Leffler
– Jonathan Leffler, Commented Apr 26, 2016 at 3:16

Community · Accepted Answer · 2020-06-20 09:12:55Z

14

Update

I apologise. My original diagnosis was wrong

It is hard to clearly express in a post like this the contents of strings that contain Perl escape characters. If anything below is unclear to you then please write a comment to say so. I hope I haven't made things unnecessarily complicated

My original solution below is still valid, and will give you better control over the contents of the command, but my reasons for why the OP's code doesn't work for them were wrong, and the truth offers other resolutions

The problem is that the contents of backticks (or qx/.../) are evaluated as a double-quoted string, which means that Perl variables and escape sequences like \t and \x20 are expanded before the string is executed. One of the consequences of this is that a backslash is deleted if it is followed by a literal newline, leaving just the newline

That means that a statement like this

my $output = `ls \
-l`;

will be preprocessed to "ls \n-l" and will no longer contain the backslash that is needed to signal to the shell that the newline should be removed (or indeed to get the command passed to the shell in the first place)

Apart from manipulating the command string directly as I described in my original post below, there are two solutions to this. The first is to escape the backslash itself by doubling it up, like this

my $output = `ls \\
-l`;

which will prevent it from being removed by Perl. That will pass the backslash-newline sequence to the shell, which will remove it as normal

The other is to use qx'...' instead of backticks together with single-quote delimiters, which will prevent the contents from being processed as a double-quoted string

my $output = qx'ls \
-l';

This will work fine unless you have used Perl variables in the string that you want to be interpolated

Original post

The problem is that the shell removes newlines preceded by backslashes from the command string before executing it. Without that step the command is invalid

So you must do the same thing yourself in Perl, and to do that you must put the command in a temporary variable

use strict;
use warnings 'all';

my $cmd = <<'END';
tar --exclude="*/node_modules" \
--exclude="*/vendor" \
--exclude='.git' \
-zvcf /tmp/robot.tgz .
END

$cmd =~ s/\\\n//g;

my $output = `$cmd`;

There is no need for the backslashes of course; you can simply use newlines and remove those before executing the command

Or you may prefer to wrap the operations in a subroutine, like this

use strict;
use warnings 'all';

my $output = do_command(<<'END');
tar --exclude="*/node_modules" \
--exclude="*/vendor" \
--exclude='.git' \
-zvcf /tmp/robot.tgz .
END

sub do_command {
    my ($cmd) = @_;
    $cmd =~ s/\\\n//g;
    `$cmd`;
}

edited Jun 20, 2020 at 9:12

CommunityBot

11 silver badge

answered Apr 26, 2016 at 3:35

Borodin

127k9 gold badges72 silver badges146 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

sferencik Over a year ago

So Perl, when running qx//, behaves somewhat shell-like (according to the "perlop" perldoc), but not quite?

Borodin Over a year ago

@sferencik: It's not that straightforward, but qx// behaves the same way as system which, like much of Perl, is designed to do what you mean. Essentially, Perl will start a shell process to execute the command if it finds any shell metacharacters in the string, like wildcard file expansion, input and output redirection, pipes etc. There's a list here. Otherwise it will start the program directly as the shell would do itself

sferencik Over a year ago

Thanks, makes sense. So if Borodin had had a redirect in that multi-line qx// statement, it would have worked, correct?

Borodin Over a year ago

@sferencik: No. There are already wildcarded file names in there so it will be passed to the shell anyway. As I said in my update, the problem is that backslash followed by newline in a Perl double-quoted string is the same as just a newline: the backslash is removed so the shell never sees it. Backslash on its own is another shell metacharacter, so if you use single quotes or escape the backslash to stop it from being eaten then it would be passed to the shell and work fine. Getting metacharacters through multiple different languages with different rules is always awkward like this

Collectives™ on Stack Overflow

how to execute multiline shell command in perl?

1 Answer 1

Update

Original post

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Update

Original post

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related