How to force command to execute in bash script

Question

I want to use command substitution in a for loop in my bash script like this:

  for file in `ls`; do
    echo "$file"
    tar xzvf "$file"
  done

The problem is upon extracting each file and in the next iteration, ls would be executed again so the for loop iterate over new collection. I decided to capture ls output before starting the loop and use that in loop:

  files=`ls`
  for file in ${files}; do
    echo "$file"
    tar xzvf "$file"
  done

But it seems instead of running ls and store the result in $files, shell just replaces ${files} with ls and I'm at the same point as I was in first code example.

How can I force shell to run ls command in files=ls part of code?

Update

I'm on my Ubuntu 16.04 laptop:

$ uname -a
Linux laptop 4.4.0-131-generic #157-Ubuntu SMP Thu Jul 12 15:51:36 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

and GNU bash, version 4.3.48(1)-release (x86_64-pc-linux-gnu)

There is a folder in my home named test with two simple compressed files plus my script:

test
├── a.tar.gz
├── b.tar.gz
└── script.sh

script.sh content:

#!/usr/bin/env bash

  files=`ls`
  for file in ${files}; do
    echo "$file"
    tar xzvf "$file"
  done

And here is the output of bash -x script.sh from inside of test folder:

++ ls
+ files='a.tar.gz
b.tar.gz
script.sh'
+ for file in '${files}'
+ echo a.tar.gz
a.tar.gz
+ tar xzvf a.tar.gz
a
+ for file in '${files}'
+ echo b.tar.gz
b.tar.gz
+ tar xzvf b.tar.gz
b
+ for file in '${files}'
+ echo script.sh
script.sh
+ tar xzvf script.sh

gzip: stdin: not in gzip format
tar: Child returned status 1
tar: Error is not recoverable: exiting now

And finally bash script.sh (after deleting extracted files manually) output:

a.tar.gz
a
b.tar.gz
b
script.sh

gzip: stdin: not in gzip format
tar: Child returned status 1
tar: Error is not recoverable: exiting now

Thanks for your help and patience.

I am unable to reproduce this issue. for loops do not re-run the command on each iteration, and files=`ls` most definitely captures the output of ls once on assignment and not on each reference. Can you please run your script with bash -x yourscript to generate a debug log, and update your question with the complete and unabbreviated output? — that other guy
– that other guy, Commented Aug 26, 2018 at 19:15
Your loop shouldn't be re-processing any file. Why not write the loop as for zipfile in *.tar.gz; do tar xzvf "$zipfile" && mv "$zipfile" "$zipfile.done"; done so that it explicitly picks up only tar.gz files and is more robust, as well as immune to any issue if you rerun the script? — codeforester
– codeforester, Commented Aug 26, 2018 at 19:20
In passing, I'll point out that parsing the output of ls is a common anti-pattern. Consider instead using an array variable: files=(*); for file in "${files[@]}"; do - that's less fragile with filenames containing spaces or newlines, for example. Oh, and don't forget to run shellcheck on your scripts! — Toby Speight
– Toby Speight, Commented Aug 27, 2018 at 11:21

codeforester · Accepted Answer · 2018-08-26 19:31:31Z

4

Your script shouldn't be processing a file more than once and it wouldn't pick up the results of tar x in the same execution, unless it is rerun. However, there are a few issues there:

it would process any file, not just the tar.gz files
in case the script is rerun, it would pick up the result of previous extractions and it would also re-process all the tar.gz files previously expanded
it will fail in case any files have whitespaces in them (see ParsingLs)
there is no error checking to see if extraction was successful or not

So, it can be written in a better way to address the above issues:

for zipfile in *.tar.gz; do
  printf '%s\n' "Processing file '$zipfile'"
  tar xzvf "$zipfile" && mv "$zipfile" "$zipfile.done"
  if (($? == 0)); then
    printf '%s\n' "$zipfile: success"
  else
    printf '%s\n' "$zipfile: failure"
  fi
done

edited Aug 26, 2018 at 19:31

answered Aug 26, 2018 at 19:25

codeforester

43.8k21 gold badges123 silver badges159 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

ingroxd Over a year ago

if [ "${?}" -eq 0 ]; then will be better, for portability

user6435535 · Accepted Answer · 2018-08-27 11:07:23Z

0

It seems I was confused by tar command output because inside a.tar.gz there is a file named a and the same for b.tar.gz.

Changing the code as following solved the problem:

  files=`ls`
  for file in ${files}; do
    echo "$file"
    tar xzvf "$file" &> /dev/null
  done

answered Aug 27, 2018 at 11:07

user6435535

Collectives™ on Stack Overflow

How to force command to execute in bash script

2 Answers 2

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related