How to group comma seperated columns into one single string in a unix for loop

Question

I am working with a pipe separated list(text file), which looks like

recipename|category|list_items
-------------------------------------------
veg_quesadilla|groupA|lettuce, spinach, beans
burrito_bowl|groupA|brown_rice, black_beans, lettuce, pepper
french_fries|groupB
beverage|groupC|pepsi

I have below statement

for recipename in `head -4 list_catalog.txt | awk -F "|" '{print $1}'`
do
if [ `cat list_catalog.txt | grep $recipename |  awk -F "|" '{print $3}'` != NULL ]
then
    for list_items in `cat list_catalog.txt | grep $recipename | awk -F "|" '{print $3}'`
do
echo -e ${list_items}
done
fi
done

In the first loop, i am iterating over each recipe name, and i am checking if the list_items exists for each recipe. If yes, then i want to print all the column seperated items as one string, instead of seperated items. I mean all the comma separated items (which starts after second pipe symbol) as one string

Like this:

lettuce, spinach, beans

Instead of:

lettuce,
spinach,
beans

And when i run the code i am getting error,

line 3: [: too many arguments

Not sure, if i am doing right with AWK or not. Please correct me.

Bill Karwin · Accepted Answer · 2017-04-02 19:04:30Z

1

You have several problems with your code.

if [ cat `list_catalog.txt | grep $recipename |  awk -F "|" '{print $3}'` != NULL ]

Your command inside the command-substitution quotes starts with list_catalog.txt as if it's the command. You probably meant to put cat inside the command-substitution quotes. Maybe that was just a mistake as you typed this question into Stack Overflow, because it gives a different error than the one you reported.

myscript.sh: line 3: list_catalog.txt: command not found

Even if you fix that, it generates an invalid test. You can spot it if you run sh with debugging output:

$ sh -x myscript.sh
...
+ '[' lettuce, spinach, beans '!=' NULL ']'
myscript.sh: line 3: [: too many arguments
...

The words returned by your command substitution are not quoted, so they show up as separate words. You can't have three words as the left hand side of the != operator.

To fix this, put the command-substitution inside quotes, so whatever it returns will be treated as one string.

if [ "`cat list_catalog.txt | grep $recipename |  awk -F '|' '{print $3}'`" != NULL ]

All that said, I can't tell why you're doing this at all. The input file already has your list_items on a single line, which appears to be your goal.

answered Apr 2, 2017 at 19:04

Bill Karwin

567k87 gold badges711 silver badges872 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

user1023627 Over a year ago

Thanks @Bill Karwin. That double quotes worked perfectly.I am actually new to unix and my list catalog contains 100 such lines and i am trying to generate an SQL query and hit the database. select distinct ${list_items} from database.$recipename order by ${list_items} desc. Though this solved my purpose, i am working to write better code. BTW, i corrected the if condition.

RavinderSingh13 · Accepted Answer · 2017-04-02 18:55:01Z

0

@try:

cat script.ksh
recipe_name="etc"
awk -vrec_name="$recipe_name" -F"|" '($0 ~ rec_name) && $3{print $3}'  Input_file

Simply making field separator as | and checking $3 is present or not if yes then print $3 which means 3rd field.

edited Apr 2, 2017 at 18:55

answered Apr 2, 2017 at 18:51

RavinderSingh13

135k14 gold badges61 silver badges100 bronze badges

3 Comments

Ed Morton Over a year ago

careful with that condition, it doesn't JUST test for $3 being present. Think about it...

RavinderSingh13 Over a year ago

I just edited my code, as it was not clear from user's question so read user's code and edited it above now. Thank you for Ed for your reply.

Ed Morton Over a year ago

I think you missed the point $3{print} will test for $3 being being presnet and not equal to a value that evaluates numerically to zero. If you just want to test for it being present that'd be $3!=""{print}. btw not putting a space between -v and rec_name= makes your script unnecessarily gawk-specific.

Collectives™ on Stack Overflow

How to group comma seperated columns into one single string in a unix for loop

2 Answers 2

1 Comment

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related