Bash: reading same lines in two files in nested loop

Question

I'm trying to calculate confidence interval from several files: ones contains lines with means, and others contains lines with values (one per line). I'm trying to read one line from the file that contains the means, and all the lines from another file (because I have to do some computations). Here is what I've done (of course it's not working):

parameters="some value to move from a file to another one"

while read avg; do 
    for row in mypath/*_${parameters}*.dat; do
        for value in $( awk '{ print $2; }' ${row}); do

    read all the lines in first_file.dat (I need only the second column)
            read the first line in avg.dat
            combine data and calculate the confidence interval      

    done
    done
done < avg.dat

** file avg.dat (not necessarily 100 lines) **

.99
2.34
5.41
...
...
2.88

** firstfile.dat in mypath (100 lines) **

0 13.77
1 2
2 63.123
3 21.109
...
...
99 1.05

** secondfile.dat in mypath (100 lines) **

0 8.56
1 91.663
2 19
3 0
...
...
99 4.34

The first line of avg.dat refers to the firstfile.dat in mypath, the second line of avg.dat refers to the secondfile.dat in mypath, etc... So, in the example above, I have to do some computation using .99 (from avg.dat) with all the numbers in the second column of firstfile.dat. Same with 2.34 and secondfile.dat.

I can't reach my objective because I can't find a way to switch to the next line in the avg.dat when I've finished to read a file in mypath. Instead I read the first line in avg.dat and all the files in mypath, then the second line in avg.dat and, again, all the files in mypath, etc... Can you help me to find a solution? Thank you all!

This sounds like it could be much more efficient written in awk. Please show us some sample files and your desired output. — glenn jackman
– glenn jackman, Commented Jun 4, 2014 at 15:58

glenn jackman · Accepted Answer · 2014-06-04 16:52:12Z

1

In bash I would do this:

exec 3<avg.dat
shopt -s extglob
for file in !(avg).dat; do
    read -u 3 avg
    while read value; do
        # do stuff with $value and $avg
    done < <(cut -f 2 -d " " "$file")
done
exec 3<&-   # close the file descriptor

edited Jun 4, 2014 at 16:52

answered Jun 4, 2014 at 16:22

glenn jackman

249k42 gold badges233 silver badges362 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Apokalos Over a year ago

Because I'm new in bash language, can you explain me your code? There is a way to read the second column of all file (except avg.dat)? I was using awk with a for cycle. Thank you!

glenn jackman Over a year ago

I'll try to come back later to add some comments. I use some fairly advanced features including process substitution and extended pattern matching

Collectives™ on Stack Overflow

Bash: reading same lines in two files in nested loop

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related