1

I've got two variables VAR1 and VAR2 that contain strings. What I want to do is go through a list of files that have a .txt extension and change all occurences of VAR1 to VAR2. So far, it looks like this:

for i in `find . -name "*.txt"`
do
 echo $i
 sed -i -E "s|\$VAR1|\$VAR2|g" $i
done

I think everything except the sed line is working well. I think it's a syntax issue, but I haven't been able to figure out what it is. Any help would be appreciated Thanks

Improve this question
3
  • Note that on Mac OS X (and BSD), sed will backup your files with the -E suffix (so xyz.txt will be backed up to xyz.txt-E). GNU sed will handle things differently (the backup suffix, if it exists, must be attached to the -i option). When you're not running into that issue, the -E option in Mac OS X (BSD) sed means 'use extended regular expressions' instead of 'sed-style basic regular expressions'. The normal equivalent option for GNU sed is -r. Commented Jan 25, 2016 at 2:57
  • 1
    You seem to want to replace $VAR1 with $VAR2 only when there is a dollar sign in front. Is this correct? Please edit your question to clarify. Several of the answers here assume you want to interpolate the values of the shell variables VAR1 and VAR2 into the sed script instead. Commented Jan 25, 2016 at 8:19
  • 1
    Please edit your question to include a minimal example that demonstrates what you're getting at. For example, a list of files, both before and after running the script. Including examples will provide clarity that may not be available from the description or code you've provided. Commented Jan 25, 2016 at 8:24

4 Answers 4

Reset to default
2

You shouldn't need to escape your $ variable. Also make sure to use the lower case -e and quote the filename in case it has spaces:

sed -ri -e "s|$VAR1|$VAR2|g" "$i"
Improve this answer
Sign up to request clarification or add additional context in comments.

4 Comments

Whoops. That -E is actually supposed to be a -r. It's to be used for extended regular expressions
I see. I take it you are using OS X because you had the -E option originally?
@MartinKonecny: You should use -E, it's supported by GNU sed, so your solution will work in BSD sed, too.
I didn't know that -E was for OS X. I got the sed syntax from somewhere else online. Thanks
1

Since sed's "find-and-replace" functionality is oriented to regular expressions rather than literal strings, you might wish to consider an alternative to sed, e.g. using awk as follows:

 awk -v from="$VAR1" -v to="$VAR2" '
   function replace(a,b,s,  n) {
     n=index(s,a);
     if (n==0) {return s}
     return substr(s,1,n-1) b replace(a,b, substr(s,n+length(a)));
   }
   {print replace(from, to, $0)} '

The above can easily be combined with the find ... | while read f ; do .... done pattern mentioned elsewhere on this page.

GNU awk supports the equivalent of sed's '-i' option, but it's probably better simply to direct the output of awk to a temporary file, and then mv it into place.

Improve this answer

Comments

1

You managed to quote the dollar sign from the shell (which would not have been necessary if you had used single quotes instead of double) but this does not change the fact that dollar signs also have a meaning in regular expressions. Double the backslashes to escape from both the shell and sed, or use single quotes so the backslashes get through to sed. Alternatively, use a notation which does not require backslashes.

sed -i -E 's|[$]VAR1|$VAR2|g' "$i"

Incidentally, your loop has a number of problems. Your for loop will not work correctly if there are file names with whitespace in them, and you need to quote the arguments inside the loop. To completely cope with file names with special characters in them, you want to use find -exec instead.

find . -name "*.txt" -exec sed -i -E 's|[$]VAR1|$VAR2|g' {} \;

If your find supports \+ instead of \;, by all means use that.

Improve this answer

1 Comment

Great answer. I would like to also suggest ( as a best practice ) to write VAR1 and VAR2 in lower case, since by convention, environment variables (PATH, EDITOR, SHELL, ...) and internal shell variables (BASH_VERSION, RANDOM, ...) are fully capitalized. All other variable names should be lowercase. Since variable names are case-sensitive, this convention avoids accidentally overriding environmental and internal variables.
0

(1) Using the idiom for i in $(find ....) ; do ...; done will often work as intended, but it is not robust. Significantly better is the pattern:

find ... | while read i ; do ... ; done

(2) If $VAR1 and/or $VAR2 contain characters that have special significance in regular expressions, then some care will be required. For example, parentheses ("(" and ")") have special significance, and so if VAR1 contains these, using the -r option (or on a Mac, the -E option) is probably asking for trouble.

(3) Chances are that sed -i -e "s|$VAR1|$VAR2|g" will do the trick if VAR1 does not contain any of the eight characters: ^$*[]\|. and if VAR2 does not contain "|", "\" or "&".

(4) If you want to prepare your strings ($VAR1 and $VAR2) programatically for use with sed, then see this SO page; it shows how to munge the strings -- using sed of course!

Improve this answer

3 Comments

Never do this: for x in $(command) or command or $var. for-in is used for iterating arguments, not (output) strings. Instead, use a glob (eg. *.txt), arrays (eg. "${names[@]}") or a while-read loop (eg. while read -r line). See mywiki.wooledge.org/BashPitfalls#pf1 and mywiki.wooledge.org/DontReadLinesWithFor
@Rany Albeg Wein - Your comment would be better placed with one of the posts that suggest using "for", wouldn't it? Here, your comment mainly reiterates (1).
Just giving a side note so that people won't run dangerous stuff as you suggested "will often work as intended, but it is not robust." The provided links show some great examples of the issue.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.