2

I am a newbie in bash script.

Here is my environment:

Mac OS X Catalina

/bin/bash

I found here a mix of several commands to remove the duplicate string in a string.

I needed for my program which updates the .zhrc profile file.

Here is my code:

#!/bin/bash
a='export PATH="/Library/Frameworks/Python.framework/Versions/3.8/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/local/bin:"'

myvariable=$(echo "$a" | tr ':' '\n' | sort | uniq | xargs)

echo "myvariable : $myvariable"

Here is the output:

xargs: unterminated quote
myvariable :

After some test, I know that the source of the issue is due to some quotes "" inside my variable '$a'.

Why am I so sure?

Because when I execute this code for example:

#!/bin/bash
a="/Library/Java/JavaVirtualMachines/jdk1.8.0_271.jdk/Contents/Home:/Library/Java/JavaVirtualMachines/jdk1.8.0_271.jdk/Contents/Home"

myvariable=$(echo "$a" | tr ':' '\n' | sort | uniq | xargs)

echo "myvariable : $myvariable"

where $a doesn't contain any quotes, I get the correct output:

myvariable : /Library/Java/JavaVirtualMachines/jdk1.8.0_271.jdk/Contents/Home

I tried to search for a solution for "xargs: unterminated quote" but each answer found on the web is for a particular case which doesn't correspond to my problem.

As I am a newbie and this line command is using several complex commands, I was wondering if anyone know the magic trick to make it work.

Improve this question
5
  • Thank you so much. It almost works. I get 99% correct output. It misses the 2nd quote " at the end => export PATH="/Library/Frameworks/Python.framework/Versions/3.8/bin Commented Dec 14, 2020 at 15:42
  • The code as a whole is pretty dangerous: You split the string in a at the colons into several lines. Therefore one of the strings starts with export and ends with Home, and has a quote somewhere in the middle. Another string ends with a quote. Then you sort them, and then glue this pieces together. It is not clear that the parts will end up in a way that the result is a well-formed command again, where the quotes are in the right place. Commented Dec 14, 2020 at 15:48
  • A simplified version of the problem with your approach can be shown with (echo '"a'; echo 'b"')|xargs, which results in an unmatched double quote error message. Commented Dec 14, 2020 at 15:52
  • you might consider reading stackoverflow.com/questions/18135451/… Commented Dec 14, 2020 at 16:28
  • 1
    one issue with sorting is that you'll likely change the order of the entries which in turn could have undesired consequences (eg, $PATH was originally built with a specific precedence/directory-ordering in mind, but that precedence/ordering has been mangled due to the sorting); net result is that for variables like $PATH you'll probably want to look at solutions that maintain the order of the individual directories Commented Dec 15, 2020 at 14:40

2 Answers 2

Reset to default
0

Basically, you want to remove duplicates from a colon-separated list.

I don't know if this is considered cheating, but I would do this in another language and invoke it from bash. First I would write a script for this purpose in zsh: It accepts as parameter a string with colon separtors and outputs a colon-separated list with duplicates removed:

#!/bin/zsh

original=${1?Parameter missing}  # Original string

# Auxiliary array, which is set up to act like a Set, i.e. without 
# duplicates
typeset -aU nodups_array

# Split the original strings on the colons and store the pieces
# into the array, thereby removing duplicates. The core idea for
# this is stolen from:
# https://stackoverflow.com/questions/2930238/split-string-with-zsh-as-in-python
nodups_array=("${(@s/:/)original}")

# Join the array back with colons and write the resulting string
# to stdout.
echo ${(j':')nodups_array}

If we call this script nodups_string, you can invoke it in your bash-setting as:

#!/bin/bash
a_path="/Library/Frameworks/Python.framework/Versions/3.8/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/local/bin:"
nodups_a_path=$(nodups_string "$a_path")
my_variable="export PATH=$nodups_a_path"
echo "myvariable : $myvariable"

The overall effect would be literally what you asked for. However, there is still an open problem I should point out: If one of the PATH components happens to contain a space, the resulting export statement can not validly be executed. This problem is also inherent into your original problem; you just didn't mention it. You could do something like

my_variable=export\ PATH='"'$nodups_a_path"'"'

to avoid this. Of course, I wonder why you take such an effort to generat a syntactically valid export command, instead of simply building the PATH by directly where it is needed.

Side note: If you would use zsh as your shell instead of bash, and only want to keep your PATH free of duplicates, a simple

typeset -iU path

would suffice, and zsh takes care of the rest.

Improve this answer
Sign up to request clarification or add additional context in comments.

Comments

0

With awk:

awk -v RS=[:\"] 'NR > 1 { pth[$0]="" } END { for (i in pth) { if (i !~ /[[:space:]]+/ && i != "" ) { printf "%s:",i } } }' <<< "$a"

Set the record separator to : and double quotes. Then when the number record is greater than one, set up an array called pth with the path as the index. At the end, loop through the array, re printing the paths separated with :

Improve this answer

1 Comment

Thanks @Raman but I get this error :"no matches found: RS=["]".

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.