I am trying to write a bash script to use sed to delete some lines of a file. The line numbers are stored in another file in reverse order. The command I am trying to do is the following:
sed -e '{lineNumber}d' ./file.txt
This is what I have so far but it's not working
while read -r line
do
sed -e "/${line}d" ./file.txt
done < ./lineNum.txt
I am getting the following error:
sed: -e expression #1, char 4: unterminated address regex
Actually what you did wrong is this
sed -e "/${line}d" ./file.txt
You see, sed has this syntax
sed -e "/REGEX/d" ./file.txt
which deletes all lines that contains match(es) to REGEX pattern. Since you have the first /, sed thinks you are trying to use regex matching, hence it's saying unterminated address regex.
The minimal fix required is simply removing the offending backslash, i.e.
sed -e "${line}d" ./file.txt
Aside: Not a sed solution like OP requested, but does what OP wants more efficiently.
awk 'NR==FNR {arr[$0]++; next} {if (!arr[FNR]) print }' linenum.txt file.txt
{if (!arr[FNR]) print }, you can write: !(FNR in arr). It's a little neater. HTH.
!arr[FNR], but I think !(FNR in arr) is better since it doesn't create the element if it didn't exist before.As long as there aren't outrageously many lines to be deleted and you aren't working on a system with a woefully limited version of sed (at one time, sed on HP-UX was limited to about 100 commands), then you can use:
sed 's/$/d/' linenum.txt | sed -f - file.txt
This uses the first sed to convert the line numbers into delete commands (note that part of your trouble was a stray unwanted slash) and then tells the second sed to read its script from standard input (-f -) and apply it to file.txt.
The above works with GNU sed; it did not work with BSD sed on Mac OS X 10.7.5 (sed: -: No such file or directory). Test it before using it on your system.
Of course, if you've got a sufficiently recent version of bash (works with bash 4.2 but not with 3.2), then you can use 'process substitution' to work around the limitation of sed:
sed -f <(sed 's/$/d/' linenum.txt) file.txt
If that doesn't work either, you can write the output of the first sed command to a file and then use that (temporary) file as the name for the sed script. So, there are lots of ways to do it. However, anything over 3 processes (two runs of sed and one of rm) is extravagant. It's probably not a problem if you only need to do it once, but it could be an issue if you've got to do it many times a minute.
sed: -: No such file or directory problem could also be solved by using /dev/stdin instead. Can not try since I am not on a Mac.while read -r line; do sed -i "${line} d" ./file.txt; done < ./linenum.txt
This works (I think your problem was to use -e); but it's not efficient. It may be better to pass multiple lines at a time to sed, to avoid reading and writing the file once per line. E.g., you could transform linenum.txt into something like "6 d;2 d;1 d;" and then pass it to sed for one scoop processing.
You can make the changes directly using sed without using a loop:
sed 's/.*/&d/' lineNum.txt | sed -i -f - file.txt
sed -n '/^[0-9][0-9]*$/{s/.*/&d/;p}' lineNum.txt | sed -i -f - file.txt to avoid problems with bad formatted lines in lineNum (particularly empty lines which would generate a stripped 'd' and delete all file content)sed -n '/^[0-9][0-9]*$/{s/$/d/;p}' lineNum.txt | sed -i -f - file.txt to be more efficient and (guessing) sed -n '/^[0-9][0-9]*$/s/$/d/p' lineNum.txt | sed -i -f - file.txt to do the same with less code. Also just out of curiousity: What would happen if a number starts with leading zeros or if a line contained just zero? Would [1-9][0-9]* be more robust?