Bash - Loop a command while iterating variables?

Question

I'm new to BASH, and I'm a bit lost. I'm also not sure if my particular issue has been addressed before or not (apologies if it has).

I have a csv file with a table that has 3 columns. I'd like to run a loop that will repeat a certain command and, after it's completed, re-run my command and insert the next values from the table my variables pull from. I'd like this process to repeat until each value from each column of the table has been pulled (my command pulls three at a time).

Here's what I have so far:

file=$(cat /path/my_ids.csv | awk -v FS=',' '{ print $1}'| head -1 | tail -n 1 )
id1=$(cat /path/my_ids.csv | awk -v FS=',' '{ print $2}'| head -1 | tail -n 2 )
id2=$(cat /path/my_ids.csv | awk -v FS=',' '{ print $3}'| head -1 | tail -n 3 )

These are the variables I am using to call values from their respective columns in my table (my_ids.csv). My command will need to pull three values from three different columns at once each time it runs through the loop. My command is supposed to look like this once the loop is ready - all I've done to the original command is include what is in between the asterisks:

command -base **$file** -tp **$id1** -tp **$id2** -all

(The command runs from a specific program that will create a new directory (file) while averaging the other two id directories. This is why I need a loop that runs through the next value in each variable column every time that it loops.)

How can I get bash to repeat that command and move down the three columns of values on each loop occurrence?

Thanks for your help and patience!

Note that cat | awk is generally wrong. And awk | head | tail is also an anti-pattern. You shouldn't be using awk at all to parse the line, but if you wanted to do so you could get the value from the Nth line with something like file=$(awk 'NR==n{print $1}' n=$N FS=, /path/my_ids.csv) rather than (ab-)using cat/head/tail. — William Pursell
– William Pursell, Commented Aug 20, 2021 at 20:48

KamilCuk · Accepted Answer · 2021-08-20 19:45:40Z

5

You read a file line by line field by field and execute the command.

{ 
   # ignore first line
   IFS= read -r _
   # read each line and split on ,
   while IFS=, read -r file id1 id2; do
     # execute the command
     command -base "$file" -tp "$id1" -tp "$id2" -all
   done
} < my_ids.csv

Read https://mywiki.wooledge.org/BashFAQ/001 . Check your scripts with https://shellcheck.net .

answered Aug 20, 2021 at 19:45

KamilCuk

146k8 gold badges86 silver badges155 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

William Pursell Over a year ago

This is absolutely the right thing to do. One potential caveat is that you need to make sure command does not attempt to read from stdin, since if it does it will consume data intended for the while read. It is probably safer to do command <&- -base ... to prevent that, or use a different file descriptor for read.

Vince Edwards Over a year ago

Thank you! This worked out! I've had some trouble with it over the last couple of days - it turns out Notepad++ needed to be set to UNIX. This is great, thanks a lot!

Collectives™ on Stack Overflow

Bash - Loop a command while iterating variables?

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related