Looping over files in a folder for shell script with multiple inputs

Question

Specifying multiple inputs for command line tool?

I am new to bash and am wanting to loop a command line program over a folder containing numerous files.

The script takes two input files (in my case, these differ in one field of the file name ("...R1" vs "...R2"). Running a single instance of the tool looks like this:

tool_name infile1 infile2 -o outfile_suffix

Actual example:

casper sample_name_R1_001.out.fastq sample_name_R2_001.out.fastq -o sample_name_merged

File name format:

DCP-137-5102-T1A3_S33_L001_R1_001.fastq
DCP-137-5102-T1A3_S33_L001_R2_001.fastq

The field in bold will vary between different pairs (e.g., 2000, 2110, 5100 etc...) with each pair distinguished by either R1 or R2.

I would like to know how to loop the script over a folder containing numerous pairs of matched files, and also ensure that the output (-o) gets the 'sample_name' suffix.

I am familiar with the basic for file in ./*.*; do ... $file...; done but that obviously won't work for this example. Any suggestions would be appreciated!

Does every pair contain R1 and R2? Can you give some more examples of the filename pairs? — choroba
– choroba, Commented Mar 17, 2015 at 14:52
Yes, one member of the pair will have R1 and the other will have R2. I have updated the post with a more detailed description. — Colin Anthony
– Colin Anthony, Commented Mar 17, 2015 at 15:05

Paul Evans · Accepted Answer · 2015-03-17 15:18:11Z

2

You want to loop over the R1's and derive the R2 and merged-file names from that, something like:

for file1 in ./*R1*; do
    file2=${file1/R1/R2}
    merge=${file1#*R1}_merged
    casper ${file1} ${file2} -o ${merge}
done

Note: Markdown is showing the #*R1}_merged as a comment -- it's not

answered Mar 17, 2015 at 15:18

Paul Evans

27.7k3 gold badges41 silver badges56 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Colin Anthony Over a year ago

Thanks! This was a great help. Side note: "${file1#*R1}_merge" removes the characters preceding "R1" from the file path (including the sample name), rather than the tailing characters. But using "${file1/R1*/_}merged" solved that issue.

Paul Evans Over a year ago

@ColinAnthony No problem. Yes, that should have been ${file1%R1*}merged but glad you resolved it.

Collectives™ on Stack Overflow

Looping over files in a folder for shell script with multiple inputs

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related