Loop through all columns in Bash

Question

My goal is to read a .csv file, reverse each row, and then write out the result to another file and I successfully managed to do that.

The problem with my code is that I had to specify the number of columns in my csv file (f1, f2, f3, etc). How can I make it go through all columns without specifying the number of columns in a scenario where I don't know the content of the .csv file?

PS: I did try searching for an answer on the forum before asking but I really couldn't find anything that helps!

input="/path/to/file"
while IFS=',' read -r f1 f2 f3 f4 f5 f6 f7
do 
    echo "$f7 $f6 $f5 $f4 $f3 $f2 $f1" >> output.csv
done < "$input"

Small improvement: write >> output.csv outside the while loop. — Walter A
– Walter A, Commented Oct 23, 2016 at 19:56

mklement0 · Accepted Answer · 2016-10-23 20:17:16Z

Using awk not only makes dealing with a variable number of columns easier, but will also be much faster:

awk -F, '{ for(i=NF; i>=2; --i) printf "%s ", $i; print $1 }' "$input" > output.csv

-F, tells awk to split the input into fields by commas
for(i=NF; i>= 2; --i) loops over all fields indices, counting down from NF (the number of input fields) to the 2nd field.
$i references field (column) whose (1-based) index is i.
printf "%s " is the (implicit) loop body (you may enclose it in { ... } for clarity), and it prints the field at hand with a trailing space, as in your question's code.
Note that printf, unlike print does not automatically append a newline by default.
Finally, after the loop, print $1 prints the first field, followed by a newline (\n) to complete the output line (\n is the default value for ORS, the output record separator).

If you need a pure Bash solution (much slower):

Arrays combined with read -a are the solution:

while IFS= read -r line; do # read input line by line
  # Split line into fields and read into array.
  IFS=, read -ra fields <<<"$line"
  # Loop over fields from the back, down to the 2nd.
  for (( i = ${#fields[@]} - 1; i >= 1; --i )); do
    printf '%s ' "${fields[i]}"
  done
  # Print 1st field and complete line (append line).
  printf '%s\n' "${fields[0]}"
done < "$input" > output.csv

Note:

A single output redirection, > output.csv, is applied to the entire loop (just like the input redirection, < "$input"), which is both simpler and more efficient than using >> output.csv inside the loop (in the latter case, you'd also have to truncate output.csv beforehand in order to ensure that no preexisting content is appended to).
bash arrays are 0-based, whereas awk-based arrays start at index 1.

Collectives™ on Stack Overflow

Loop through all columns in Bash

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related