bash scripting - while loop not reading next line and storing value as intended

Question

I can't figure out why my while loop construct with these embedded if statements is not working. I have a file called source.txt

mmsGroupId=5ab5c0e04eff45274ce5e471
mmsApiKey=5ab5c22c4eff45274ce5e63f07536eddebedec576b0e02904deb001a

I want my bash script to compare the local values (i.e. mmsGroupId and mmsApiKey) stored in a configuration file (which i use a sed replace). Initially, the local config file will have no value for the keys, so it would look like this test.config.

mmsGroupId=
mmsApiKey=

If the keys are empty and don't match, then replace the local values with the ones source.txt.

There will also be the case if the mmsGroupId, mmsApiKey will also have old values and I want to replace that with what's in the source file

mmsGroupId=123456789
mmsApiKey=123456789abcdefghijklmnopqrstuvwxyz

I'm using a while loop to read the 2 lines in the source.txt file, however I'm only getting the first line's (mmsGroupId) and not the (mmsApiKey).

Result:

mmsGroupId=5cc5e22c4eff45274ce5e63f07536eddebedec576b0e02904deb001a
mmsApiKey=5cc5e22c4eff45274ce5e63f07536eddebedec576b0e02904deb001a

Expecting: (note: there's more to this configuration file - i'm just showing the expected key=value that i'm expecting )

mmsGroupId=5ab5c0e04eff45274ce5e471
mmsApiKey=5ab5c22c4eff45274ce5e63f07536eddebedec576b0e02904deb001a

So far my script looks like this:

#!/bin/bash

set -x

KEY=/tmp/source.txt

IFS='='
while read keyname value;
do
  echo "This is the mmsGroupId:$value"
  curr_group=$(grep mmsGroupId /tmp/test.config | cut -d "=" -f2);
  echo "This is the current mmsGroupId:$curr_group"
  if [[ "$keyname" = "mmsGroupId" && "$value" = "$curr_group" ]]; then
    echo "We have a match - $value:$curr_group - NOTHING TO DO"
  else
    echo "No match found - let us update it"
    sed -i 's/mmsGroupId='"$curr_group"'/mmsGroupId='"$value"'/g' /tmp/test.config
  fi
  echo "This is the mmsApiKey:$value"
  curr_apikey=$(grep mmsApiKey /tmp/test.config | cut -d "=" -f2);
  echo "This is the current mmsApiKey:$curr_apikey"
  if [[ "$keyname" = "mmsApiKey" && "$value" = "$curr_apikey" ]]; then
    echo "We have a match - $value:$curr_apikey - NOTHING TO DO"
  else
    echo "No match found - let us update it"
    sed -i 's/mmsApiKey='"$curr_apikey"'/mmsApiKey='"$value"'/g' /tmp/test.config
  fi
done < "$KEY"

set +x

Here's the debug log:

+ KEY=/tmp/OpsManagerKeys.txt
+ IFS==
+ read keyname value
+ echo 'This is the mmsGroupId:5bc5e0e04eff45274ce5e471'
This is the mmsGroupId:5bc5e0e04eff45274ce5e471
++ grep mmsGroupId /tmp/test.config
++ cut -d = -f2
+ curr_group=
+ echo 'This is the current mmsGroupId:'
This is the current mmsGroupId:
+ [[ mmsGroupId = \m\m\s\G\r\o\u\p\I\d ]]
+ [[ 5bc5e0e04eff45274ce5e471 = '' ]]
+ echo 'No match found - let us update it'
No match found - let us update it
+ sed -i s/mmsGroupId=/mmsGroupId=5bc5e0e04eff45274ce5e471/g /tmp/test.config
+ echo 'This is the mmsApiKey:5bc5e0e04eff45274ce5e471'
This is the mmsApiKey:5bc5e0e04eff45274ce5e471
++ grep mmsApiKey /tmp/test.config
++ cut -d = -f2
+ curr_apikey=
+ echo 'This is the current mmsApiKey:'
This is the current mmsApiKey:
+ [[ mmsGroupId = \m\m\s\A\p\i\K\e\y ]]
+ echo 'No match found - let us update it'
No match found - let us update it
+ sed -i s/mmsApiKey=/mmsApiKey=5bc5e0e04eff45274ce5e471/g /tmp/test.config
+ read keyname value
+ echo 'This is the mmsGroupId:5cc5e22c4eff45274ce5e63f07536eddebedec576b0e02904deb001a'
This is the mmsGroupId:5cc5e22c4eff45274ce5e63f07536eddebedec576b0e02904deb001a
++ grep mmsGroupId /tmp/test.config
++ cut -d = -f2
+ curr_group=5bc5e0e04eff45274ce5e471
+ echo 'This is the current mmsGroupId:5bc5e0e04eff45274ce5e471'
+ echo 'This is the current mmsGroupId:5bc5e0e04eff45274ce5e471'
This is the current mmsGroupId:5bc5e0e04eff45274ce5e471
+ [[ mmsApiKey = \m\m\s\G\r\o\u\p\I\d ]]
+ echo 'No match found - let us update it'
No match found - let us update it
+ sed -i s/mmsGroupId=5bc5e0e04eff45274ce5e471/mmsGroupId=5cc5e22c4eff45274ce5e63f07536eddebedec576b0e02904deb001a/g /tmp/test.config
+ echo 'This is the mmsApiKey:5cc5e22c4eff45274ce5e63f07536eddebedec576b0e02904deb001a'
This is the mmsApiKey:5cc5e22c4eff45274ce5e63f07536eddebedec576b0e02904deb001a
++ grep mmsApiKey /tmp/test.config
++ cut -d = -f2
+ curr_apikey=5bc5e0e04eff45274ce5e471
+ echo 'This is the current mmsApiKey:5bc5e0e04eff45274ce5e471'
This is the current mmsApiKey:5bc5e0e04eff45274ce5e471
+ [[ mmsApiKey = \m\m\s\A\p\i\K\e\y ]]
+ [[ 5cc5e22c4eff45274ce5e63f07536eddebedec576b0e02904deb001a = \5\b\c\5\e\0\e\0\4\e\f\f\4\5\2\7\4\c\e\5\e\4\7\1 ]]
+ echo 'No match found - let us update it'
No match found - let us update it
+ sed -i s/mmsApiKey=5bc5e0e04eff45274ce5e471/mmsApiKey=5cc5e22c4eff45274ce5e63f07536eddebedec576b0e02904deb001a/g /tmp/test.config
+ read keyname value
+ set +x

no. there's is much more to the config file. i was just trying to be simplistic with the example. yes, a simple cp would suffice if it was just 2 lines. i should clarify that more. thanks. — noober
– noober, Commented Apr 29, 2019 at 4:07
Still though, you could do the replacements unconditionally since it's doesn't hurt to overwrite a value with itself. What's the output (debug log) you get when you run it? — that other guy
– that other guy, Commented Apr 29, 2019 at 5:33

Colin Moreno Burgess · Accepted Answer · 2019-04-29 07:05:49Z

2

I think you should distinguish operations depending which key:value you are using in your loop, example:

#!/bin/bash

function subst
{
  echo "This is the $keyname:$value"
  curr_value=$(grep $keyname test.config | cut -d "=" -f2);
  echo "This is the current $keyname:$curr_value"
}


#set -x

KEY=source.txt

IFS='='
while read -r keyname value;
do
  if [[ $keyname = 'mmsGroupId' ]]
    then
      echo "do group things function"
      subst
    else
      echo "do apikey things function"
      subst
  fi
done < "$KEY"

Having added function for substitution simplifies code reusability.

# bash test.sh 
do group things function
This is the mmsGroupId:5ab5c0e04eff45274ce5e471
This is the current mmsGroupId:testconfiggroupvalueXXX
do apikey things function
This is the mmsApiKey:5ab5c22c4eff45274ce5e63f07536eddebedec576b0e02904deb001a
This is the current mmsApiKey:testconfigaplikeyvalueXXX

edited Apr 29, 2019 at 7:05

answered Apr 29, 2019 at 6:56

Colin Moreno Burgess

1,6121 gold badge14 silver badges18 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

noober Over a year ago

Thanks for your reply. I agree with the readability. Your example is much better. How did you replace your values? i.e. "mmsGroupId:testconfiggroupvalueXXX"? thanks

Colin Moreno Burgess Over a year ago

I have not, the current values are the ones I put on the test.config file, you should add your code block for substitution, the 'sed' part, on the subst function

noober Over a year ago

So, i see how you read the key. If I add a 'sed' statement in the function it would only handle one key - i.e. sed -i 's/mmsGroupId='"$curr_value"'/mmsGroupId='"$value"'/g' /tmp/test.config - i'm thinking we'd need a second function to handle - sed -i 's/mmsApiKey='"$curr_value"'/mmsApiKey='"$value"'/g' /tmp/test.config

Colin Moreno Burgess Over a year ago

I think you should be able to use variables on the sed statement and avoid using multiple functions. like ""$keyname""=""$curr_value"" kind of

that other guy · Accepted Answer · 2019-04-29 16:00:08Z

1

The issue is that you do the replacements on the wrong key. In this if statement

if [[ "$keyname" = "mmsGroupId" && "$value" = "$curr_group" ]]; then
    echo "We have a match - $value:$curr_group - NOTHING TO DO"
  else
    echo "No match found - let us update it"
    sed -i 's/mmsGroupId='"$curr_group"'/mmsGroupId='"$value"'/g' /tmp/test.config
  fi

The else block was supposed to run when the value in the file is incorrect, but it also runs when the $keyname is wrong.

You could use nested if statements or elif [[ "$keyname" = "mmsGroupId ]] to ensure that the replacement doesn't run on the wrong key.

A better solution is to get rid of the while loop and if statements:

for key in mmsGroupId mmsApiKey
do
  value=$(grep "$key" /tmp/source.txt | cut -d "=" -f2);
  sed -i "s/$key=.*/$key=$value/" /tmp/test.config
done

answered Apr 29, 2019 at 16:00

that other guy

125k12 gold badges188 silver badges214 bronze badges

1 Comment

noober Over a year ago

thank you for your analysis on this - that definitely helps

Collectives™ on Stack Overflow

bash scripting - while loop not reading next line and storing value as intended

2 Answers 2

4 Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

4 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related