Bash - replace in file strings from first array with strings from second array

Question

I have two arrays. The first one is filled with values greped from the file that I want to replace with the new ones downloaded.

Please note that I don't know exactly how the first array will look like, meaning some values will have _, other - and some won't have any of that and immediately after the name will be placed : (colon).

Example arrays:

array1:

[account:123 shoppingcart-1:123 notification-core_1:123 notification-dispatcher_core_1:123 notification-dispatcher-smschannel_core_1:123]

array2:

[account_custom_2:124 shoppingcart_custom_2:124 notification_custom_2:124 notification-dispatcher_custom_2:124 notification-dispatcher-smschannel_custom_2:124]

Those arrays are the only example, there are more than 50 values to replace.

I am doing a comparison of every item in the second array with every item in the first array as shown below:

file_name="<path_to_file>/file.txt"
for i in "${!array1[@]}"
do
        for j in "${!array2[@]}"
        do
                array_2_base="`echo ${array2[$j]} | awk -F_ '{print $1}'`"
                if [[ "${array1[$i]}" == *"$array_2_base"* ]]
                then
                        sed -i "s=${array1[$i]}=${array2[$j]}=g" $file_name
                 fi
        done
done

Here I substring only the first part for every item in the second array so I can compare it with the item in the first array.

e.g. account_custom_2:124 -> account or notification-dispatcher_custom_2:124 -> notification-dispatcher.

This works nice but I encounter problem when notification is in notification-core_1:123 and notification-dispatcher_core_1:123 and notification-dispatcher-smschannel_core_1:123.

Can you please give advice on how to fix this or if you can suggest another approach to this?

the complexity of your script might be reaching a tipping point. Once you start juggling arrays in Bash, it's worth considering if your program is outgrowing a simple shell script (a 'glue' language running a sequence of commands with minimal data manipulation or complex branching logic). You might want to consider refactoring into something like Python/Perl/Ruby/anything-not-shell-script. — Z4-tier
– Z4-tier, Commented Feb 17, 2020 at 15:07

tshiono · Accepted Answer · 2020-02-18 05:09:20Z

The point is the base of array2 element may include other element as a substring and will cause an improper replacement depending on the order of matching. To avoid this, you can sort the array in descending order so the longer pattern comes first.

Assuming the strings in the arrays do not contain tab characters, would you please try:

file_name="<path_to_file>/file.txt"
array1=(account:123 shoppingcart-1:123 notification-core_1:123 notification-dispatcher_core_1:123 notification-dispatcher-smschannel_core_1:123)
array2=(account_custom_2:124 shoppingcart_custom_2:124 notification_custom_2:124 notification-dispatcher_custom_2:124 notification-dispatcher-smschannel_custom_2:124)

# insert the following block to sort array2 in descending order
array2=( $(for j in "${array2[@]}"; do
    array_2_base=${j%%_*}
    printf "%s\t%s\n" "$array_2_base" "$j"
done | sort -r | cut -f2-) )

# the following code will work "as is"
for i in "${!array1[@]}"
do
    for j in "${!array2[@]}"
    do
        array_2_base="`echo ${array2[$j]} | awk -F_ '{print $1}'`"
        if [[ "${array1[$i]}" == *"$array_2_base"* ]]
        then
            sed -i "s=${array1[$i]}=${array2[$j]}=g" "$file_name"
            delete="${array1[$i]}"
            array1=( "${array1[@]/$delete}" )
        fi
    done
done

The script above will be inefficent in execution time due to the repetitive invocation of sed -i command.
The script below will run faster by pre-generating the sed script and executing it just once.

file_name="<path_to_file>/file.txt"
array1=(
    account:123
    shoppingcart-1:123
    notification-core_1:123
    notification-dispatcher_core_1:123
    notification-dispatcher-smschannel_core_1:123
)
array2=(
    account_custom_2:124
    shoppingcart_custom_2:124
    notification_custom_2:124
    notification-dispatcher_custom_2:124
    notification-dispatcher-smschannel_custom_2:124
)

while IFS=$'\t' read -r base a2; do     # read the sorted list line by line
    for a1 in "${array1[@]}"; do
        if [[ $a1 == *$base* ]]; then
            scr+="s=$a1=$a2=g;"         # generate sed script by appending the "s" command
            continue 2
        fi
    done
done < <(for j in "${array2[@]}"; do
    array_2_base=${j%%_*}               # substring before the 1st "_"
    printf "%s\t%s\n" "$array_2_base" "$j"
                                        # print base and original element side by side
done | sort -r)

sed -i "$scr" "$file_name"              # execute the replacement at once

Thank you for your answer, I tested it and it is working (second suggestion).

Ivan · Accepted Answer · 2020-02-17 14:41:38Z

1

If number of items in your arrays are equal then you can process them in one loop

for i in "${!array1[@]}"; {
    value=${array1[$i]}
    new_value=${array2[$i]}
    sed -i "s/$value/$new_value/" file
}

answered Feb 17, 2020 at 14:41

Ivan

7,5311 gold badge22 silver badges27 bronze badges

1 Comment

Vladimir Todorov Over a year ago

Thank you for the suggestion, but I don't know how many items are going to be downloaded so I don't know if arrays are going to be the same.

Vladimir Todorov · Accepted Answer · 2020-02-17 14:26:47Z

0

I found a way to fix this.

I am deleting string from the first array once replaced.

file_name="<path_to_file>/file.txt"
for i in "${!array1[@]}"
do
        for j in "${!array2[@]}"
        do
                array_2_base="`echo ${array2[$j]} | awk -F_ '{print $1}'`"
                if [[ "${array1[$i]}" == *"$array_2_base"* ]]
                then
                        sed -i "s=${array1[$i]}=${array2[$j]}=g" $file_name
                        delete="${array1[$i]}"
                        array1=( "${array1[@]/$delete}" )
                 fi
        done
done

answered Feb 17, 2020 at 14:26

Vladimir Todorov

791 gold badge2 silver badges13 bronze badges

1 Comment

tshiono Over a year ago

It seems to replace notification-core_1:123 with notification_custom_2:124 and notification-dispatcher_core_1:123 also with notification_custom_2:124. Are they your desired substitutions?

Collectives™ on Stack Overflow

Bash - replace in file strings from first array with strings from second array

3 Answers 3

1 Comment

1 Comment

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

1 Comment

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related