Transform file content to an specific output in bash

Question

I'm new to bash scripting, and I am trying to transform a file that:

Is a CSV
Has random line jumps that break the format (bad inputs from user in CSV)

The outcome needs to be a processed file that separates lines correctly (lines of a set # of elements/columns), removing the random jumps.

My first aproach was to remove all the line jumps by using

variable=$(cat $1)
tr -d "\n" $variable > $variable

and then though of reading char by char looking for ',' with a counter, and adding a line jump after every set number of them were found. When trying to do this, I found info on while IFS= read and tried the following:

while IFS=, read -r col1 col2 col3 col4 <<< $variable; do

    echo "$col1" "$col2" "$col3" "$col4" '\n'

done

I'm clearly missing something there (I even believe the while loop should already remove the line jumps, please correct me about it) and I am not sure how to keep going about it. I am not looking for a bunch of code that does the job, but trying to find someone here who could point me towards the right direction (maybe I'm not understanding IFS, or I need to process something prior to that step... whatever it could be).

Thanks in advance.

edit: I removed the line jump I was adding in the echo and it now prints col1 col2 col3 col4 and never ends, so I am clearly not being able to link each element from the file to the variables before printing.

I just did change the single ' ' in the \n and started printing "col1 col2 col3 col4 \n" so I removed the added line jump completely. Added an edit about it in the main post. Thanks! — Ukelele
– Ukelele, Commented Mar 1, 2020 at 13:34

Freeman · Accepted Answer · 2020-03-01 14:17:06Z

0

instead of echo "$col1" "$col2" "$col3" "$col4" '\n'

just write this

 echo -e "$col1 $col2 $col3 $col4 \n"

Actually the problem isn't with the shell, is echo command itself, and the lack of double quotes around the variable interpolation. You can try using echo -e but that isn't supported on all platforms, and one of the reasons printf is now recommended for portability.

You can also try and insert the newline directly into your shell script (if a script is what you're writing) so it looks like...

#!/bin/sh
echo "this is for
test"
#EOF

or equivalently

#!/bin/sh
string="this is for
test"
echo "$string"  
# note double quotes!

good luck

answered Mar 1, 2020 at 14:17

Freeman

12.9k8 gold badges39 silver badges63 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Transform file content to an specific output in bash

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related