Create a new file each time through a bash for loop

Question

I'm trying to create multiple shell scripts from a file with variable name. I have a bash for loop that creates the necessary lines but I want a new file each time through the loop. The script I currently have puts everything into one file. The input file (test.txt) has each variable on a separate line:

a
b
c

Here is the code I currently have:

#!/bin/bash
num=0
echo $num
for x in $(<test.txt)
do
    echo \#\!/bin/bash
    echo \#SBATCH -N 1
    echo \#SBATCH -t 6:00:00
    echo \#SBATCH --job-name=${x}
    echo \. \~/\.profile
    echo time java -jar trimmomatic.jar PE -threads 20 ${x}_R1.fastq ${x}_R2.fastq
    num=$((num+1))

done  > trim_${num}.sh
echo $num
exit

This would write three loops with a,b,c variables to trim_0.sh. I want the a loop to be in trim_0.sh, the b loop to be in trim_1.sh, and the c loop to be in trim_2.sh.

See mywiki.wooledge.org/BashFAQ/001 for the correct way to read a file line by line. (Hint: it's not using a for loop.) — chepner
– chepner, Commented Apr 14, 2015 at 14:04

Tom Fenech · Accepted Answer · 2015-04-14 14:07:48Z

1

You can wrap the body of your loop in a block, like this:

while read -r x
do
    {
        echo '#!/bin/bash'
        echo '#SBATCH -N 1'
        echo '#SBATCH -t 6:00:00'
        echo "#SBATCH --job-name=$x"

        echo '. ~/.profile'
        echo "time java -jar /lustre/software/bioinfo/trimmomatic/0.32/trimmomatic.jar PE -threads 20 ${x}_R1.fastq ${x}_R2.fastq"
    } > "trim_$((num++)).sh"
done < test.txt

I've used quotes around each echo (single quotes where there aren't any shell variables involved, double quotes where there are) and removed your backslash escapes, which I don't think were necessary.

I've also used a while read loop, rather than a for loop to read the file. This makes no difference in your simple case but as pointed out in the comments, is the correct approach.

Alternatively, this may also be a good opportunity to use a heredoc:

while read -r x
do
    cat <<EOF >"trim_$((num++)).sh"
#!/bin/bash
#SBATCH -N 1
#SBATCH -t 6:00:00
#SBATCH --job-name=${x}
. ~/.profile
time java -jar /lustre/software/bioinfo/trimmomatic/0.32/trimmomatic.jar PE -threads 20 ${x}_R1.fastq ${x}_R2.fastq
EOF
done < test.txt

edited Apr 14, 2015 at 14:07

answered Apr 14, 2015 at 12:59

Tom Fenech

75.1k13 gold badges119 silver badges154 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

ian_informatics Over a year ago

This works great but if you use the quotes on lines where there are variables ${x} then those don't get replaced but printed as is.

Tom Fenech Over a year ago

@ian_informatics note that I've used double quotes on those lines, so shell parameters are expanded.

Tom Fenech Over a year ago

@ian_informatics I've also updated my answer to show the use of a while read loop. The motivation for doing so is explained in chepner's comment above.

Collectives™ on Stack Overflow

Create a new file each time through a bash for loop

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related