bash stdout redirection in a for loop

Question

Possible duplicate

Hello,

I'm struggling with output redirection within a bash for loop. I have several similar bash scripts, each scripts being launched on a different input file. The scripts called a tool with a few arguments.

The bash scripts are like this :

my_tool --input input_file --arg1 arg1_name --arg2 arg2_name > output_dir/result_X.dat

X being a string which is unique for each script.

I run all of them in a for loop :

for script in scripts_dir/*.sh
do
    bash $script
done

However, the stdout of each run still display on the terminal. The specified output files are created, but empty. How I can solve that ? I found other questions on stackoverflow where the answer is a redirection of the full loop in one big output file, but I'd like to avoid that.

If I replace > output_dir/result_X.dat by > /dev/null : the standard outputs display on the terminal

If I replace > output_dir/result_X.dat by ` 2> /dev/null : nothing is displayed.

Thanks in advance.

What is an_identifier_defined_in_the_script.dat? The shell parses that line before the script is even executed, so if you are trying to define the name of the output file in the script, this will not work. — William Pursell
– William Pursell, Commented Oct 1, 2019 at 14:34
@WilliamPursell it's simply a string for ID. The script contains a command line for another tool, and the string changes for every run (but there is a different bash file for every run, so the ID strings are already written) — Corsair
– Corsair, Commented Oct 1, 2019 at 14:38
@Amessihel I just tried with one file : nothing displays on the terminal, but the output file is still empty :/ — Corsair
– Corsair, Commented Oct 1, 2019 at 14:41
Yes, it is because your program is outputting to standard error! I think we said this before? 2>&1says redirect stderr to stdout. You could equally put 2>output/result_X.dat — Gem Taylor
– Gem Taylor, Commented Oct 1, 2019 at 16:19

Ljm Dullaart · Accepted Answer · 2019-10-01 17:41:37Z

2

When you start my_tool, there are normally 3 file-descriptors available in the tool:

STDIN
STDOUT
STDERR

STDIN is used for input, and therefore irrelevant for this question. STDOUT is used for standard output. This is file-descriptor 1. If you do

ls 1> /dev/null

the STDOUT of ls is written to /dev/null. If you do not add the 1, as in ls > /dev/null, it is assumed that you mean STDOUT.

STDERR is used as standard output for error messages, in the broadest sense of the word. The number of STDERR is 2.

Using ls instead of your my_command, ls > file will put the listing in the file. ls /non_existing_dir_ > file will put the STDOUT of ls in the file. But there is no output on STDOUT, and because STDERR is not redirected, it will be send to the terminal.

So, to conclude,

ls . /non_existing_dir 2>stderr >stdout

will put the directory listing of . in the file stdout and the error for the non-existing directory in stderr.

With 2>&1 you redirect the output of filedescriptor2 (STDERR) to file descriptor 1 (SDTOUT).

To complicate things a bit, you can add other file descriptor numbers. For example:

exec 3>file

will put the output of file descriptor 3 (which is newly created) in file. And

ls 1>&3

will then redirect the output of file descriptor 1 to file descriptor 3, effectively putting the output of ls in file.

answered Oct 1, 2019 at 17:41

Ljm Dullaart

5,0092 gold badges19 silver badges35 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Corsair Over a year ago

I know part of that already, but I had not clue this program was sending the output to stderr ! Silly me.

Collectives™ on Stack Overflow

bash stdout redirection in a for loop

1 Answer 1

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related