1

I am new to shell scripting. I am using ksh.

I have this particular line in my script which I use to append text in a variable q to the end of a particular line given by the variable a containing the line number .

sed -i ''$a's@$@'"$q"'@' test.txt

Now the variable q can contain a large amount of text, with all sorts of special characters, such as !@#$%^&*()_+:"<>.,/;'[]= etc etc, no exceptions.
For now, I use a couple of sed commands in my script to remove any ' and " in this text (sed "s/'/ /g" | sed 's/"/ /g'), but still when I execute the above command I get the following error 

sed: -e expression #1, char 168: unterminated `s' command

Any sed, awk, perl, suggestions are very much appreciated

Improve this question
7
  • 2
    This sounds an awful lot like an XY Problem. Can you backtrack a bit, and give some examples of what you're trying to do (input and outputs)? Commented Mar 23, 2017 at 15:26
  • So $a is here a line number in test.txt? Commented Mar 23, 2017 at 15:39
  • perl -i -pe 's/$/ $./' file Commented Mar 23, 2017 at 15:50
  • 1
    You could try: perl -pi -E 'BEGIN {$q = shift; $a = shift} s/$/$q/ if $. == $a' "$q" "$a" test.txt Commented Mar 23, 2017 at 15:55
  • 1
    @Håkon Hægland, great great suggestion!, it worked brilliantly, perl is king no doubt. Would you be so kind to post this as an answer with explanation for me to also learn :) Commented Mar 23, 2017 at 20:47

3 Answers 3

Reset to default
1

The difficulty here is to quote (escape) the substitution separator characters @ in the sed command:

sed -i ''$a's@$@'"$q"'@' test.txt

For example, if q contains @ it will not work. The @ will terminate the replacement pattern prematurely. Example: q='a@b', a=2, and the command expands to 

sed -i 2s@$@a@b@ test.txt

which will not append a@b to the end of line 2, but rather a@.

This can be solved by escaping the @ characters in q:

sed -i 2s@$@a\@b@ test.txt

However, this escaping could be cumbersome to do in shell. Another approach is to use another level of indirection. Here is an example of using a Perl one-liner. First q is passed to the script in quoted form. Then, within the script the variable assigned to a new internal variable $q. Using this approach there is no need to escape the substitution separator characters:

perl -pi -E 'BEGIN {$q = shift; $a = shift} s/$/$q/ if $. == $a' "$q" "$a" test.txt
Improve this answer
Sign up to request clarification or add additional context in comments.

Comments

0

Do not bother trying to sanitize the string. Just put it in a file, and use sed's r command to read it in:

echo "$q" > tmpfile
sed -i -e ${a}rtmpfile test.txt

Ah, but that creates an extra newline that you don't want. You can remove it with:

sed -e ${a}rtmpfile test.txt | awk 'NR=='$a'{printf $0; next}1' > output
Improve this answer

3 Comments

Hi William, thanks for the suggestion, but I really want to handle this operation in memory. This line of code is inside a For loop that will be executed more than 10,000 times
@DineshKumar Just IMHO - calling sed 10k times (and even more, as you said in the question) doesn't sound for me as an efficient solution. but - ok - ymmv...
Write the tempfile outside of the loop. Chances are very good that it will never hit disk, as the OS will keep it in memory.
0

Another approach is to use the patch utility if present in your system.

patch test.txt <<-EOF
${a}c
$(sed "${a}q;d" test.txt)$q
.
EOF

${a}c will be replaced with the line number followed by c which means the operation is a change in line ${a}.

The second line is the replacement of the change. This is the concatenated value of the original text and the added text.

The sole . means execute the commands.

Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.