Bash/Shell How to remove extra spaces in a variable?

Question

How to remove extra spaces in a variable?

VAR="    Eget ac vel volutpat      dic  tumst   est dui      adipiscing  "

Correct result: VAR="Eget ac vel volutpat dic tumst est dui adipiscing"

There are over 16,000,000 to be edited; sed is too slow!

!!! E D I T !!!

String OK! Thank you all

BUT

Text in array problem, gaps remain :-(

ITM=("  Eget ac  vel volutpat   |  Vel volutpat     dic tumst "
     "    Vestibulum  laoreet     a semper  |orttitor      eu laoreet justo congue ")

IFS=$'|'

for (( i=0 ; i<16000000 ; i++ ))
do
    AAA=( ITM[$i] )
    B=${AAA[0]}     # "  Eget ac  vel volutpat   "
    C=${AAA[1]}     # "  Vel volutpat     dic tumst "

done

I very much doubt that pushing all 16 million lines at once through sed will be that slow compared to anything you script in bash. — bitmask
– bitmask, Commented Nov 7, 2012 at 21:50
@bitmask: it depends on whether he's processing all 16 million lines through a single instance of sed (fast) or running sed 16 million times (once per array element) (very slow). It all depends on where those 16 million elements are coming from; if they're coming a file, I'd run it through sed first, then process that output a line at a time (and for God's sake and don't put it in an array unless you really need to). — Gordon Davisson
– Gordon Davisson, Commented Nov 7, 2012 at 23:35
@GordonDavisson: Yes, that was what I meant. Use one sed instance and have it process all lines, each holding one of the strings. — bitmask
– bitmask, Commented Nov 7, 2012 at 23:40
@bitmask: Now that I reread it, that was actually explicit in your original comment. But I suspect that's not what Charlie was doing... — Gordon Davisson
– Gordon Davisson, Commented Nov 7, 2012 at 23:46

ghoti · Accepted Answer · 2012-11-07 22:01:54Z

2

You don't need to spawn subshells for this. Just use bash's built-in pattern substitution.

[ghoti@pc ~]$ shopt -s extglob
[ghoti@pc ~]$ VAR="    Eget ac vel volutpat      dic  tumst   est dui      adipiscing  "
[ghoti@pc ~]$ VAR=${VAR//+( )/ }
[ghoti@pc ~]$ echo "$VAR"
 Eget ac vel volutpat dic tumst est dui adipiscing 
[ghoti@pc ~]$ VAR=${VAR#+( )}
[ghoti@pc ~]$ echo "$VAR"
Eget ac vel volutpat dic tumst est dui adipiscing
[ghoti@pc ~]$

edited Nov 7, 2012 at 22:01

answered Nov 7, 2012 at 21:53

ghoti

47.2k8 gold badges71 silver badges108 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Dan Bliss Over a year ago

That reduces the spaces but does not collapse them.

ghoti Over a year ago

It replaces +( ) (any number of spaces) with a single space. What do you mean by "collapse"?

Dan Bliss Over a year ago

Am I just seeing things? I meant this for a post that suggested ${VAR// / }. That post was apparently deleted. Objection retracted.

Dan Bliss · Accepted Answer · 2012-11-07 21:54:07Z

0

Shell is not really set up to do massive data processing, even processing as simple as collapsing spaces. But here is a solution without using sed

VAR="    Eget ac vel volutpat      dic  tumst   est dui      adipiscing  "
VAR=$(echo $VAR)

This code will set VAR to the collapsed version of the original var. You can see this by executing:

echo "$VAR"

result:

Eget ac vel volutpat dic tumst est dui adipiscing

answered Nov 7, 2012 at 21:54

Dan Bliss

1,75413 silver badges10 bronze badges

2 Comments

Dan Bliss Over a year ago

You edited your post to include that. Your minus 1 on my answer is unnecessary.

Gilles Quénot Over a year ago

=) Don't be paranoid, it's not me. Moreover, "edited 22 mins ago" while you have answered 13 mins ago...

Gilles Quénot · Accepted Answer · 2012-11-07 22:05:39Z

0

Just remove the quotes, ex :

$ VAR="    Eget ac vel volutpat      dic  tumst   est dui      adipiscing  "     
$ echo $VAR
Eget ac vel volutpat dic tumst est dui adipiscing
$ echo "$VAR"
    Eget ac vel volutpat      dic  tumst   est dui      adipiscing  
$

If you prefer :

$ VAR="    Eget ac vel volutpat      dic  tumst   est dui      adipiscing  "
$ VAR="$(echo $VAR)"
$ echo "$VAR"
Eget ac vel volutpat dic tumst est dui adipiscing

This behaviour is called word splitting.

NOTE

as mentioned in the comments, you should use this only if you know that your variables contains no special characters like *.

edited Nov 7, 2012 at 22:05

answered Nov 7, 2012 at 21:34

Gilles Quénot

188k43 gold badges232 silver badges229 bronze badges

5 Comments

Charlie Over a year ago

sputnick: thank, you but I do not want to write, I need to have it in a variable ...

Gilles Quénot Over a year ago

@Lev Levitsky : x=$(</usr/share/dict/british); echo $x > /dev/null : no errors. Try it, you will see.

chepner Over a year ago

An alternative to command subsitition: arr=( $VAR ); VAR="${arr[*]}"

bitmask Over a year ago

Removing the quotes is dangerous. There might be any other sort of funny character in the string that will be expanded.

ghoti Over a year ago

And what happens when VAR="echo *" ? If the OP provides a dangerous example, it's better to show him a better way.

Collectives™ on Stack Overflow

Bash/Shell How to remove extra spaces in a variable?

3 Answers 3

3 Comments

2 Comments

5 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

3 Comments

2 Comments

5 Comments

Your Answer

Sign up or log in

Post as a guest

Related