29,013 questions
3
votes
4
answers
159
views
Array elements in sed operation
I am trying to replace elements on an array using a sed command to compare values between two arrays and replace them accordingly, with the following terminal input:
#!/bin/bash
icons=("" &...
- 75
1
vote
4
answers
99
views
sed works, but just for some files
I'm migrating a blog from Jekyll (source) to Hugo (source).
Both use Markdown, but their syntaxes are slightly different.
Let's say that in the old Jekyll blog I have this:
{% include image.html src=&...
- 3,320
0
votes
0
answers
60
views
How to escape the backward \ in sed -i to insert to top of file [duplicate]
I have a file that I'm trying to put a spool command and the dos path as the first line:
To get the dos path from bash I use:
CWDDOS=$(pwd | sed 's/\//\\/g' | sed 's/\\c/C:/g')
and it gives me this:
...
-4
votes
2
answers
117
views
append to file and remove first lines if more than N
I want to edit a file in a single command to append a line and remove first lines if more than N.
The furthest I got was to use sed:
sed -i -e "\$a\[$(date '+%F %T')] $1" -e :a -e "\$q;...
1
vote
2
answers
98
views
How do I place the output of a command in the search pattern section of sed?
I'm trying to place the output of awk 'NR==4 {split($0, a, "$"); print a[3]}' filename in sed -i 's/1/2/' hyprpaper.conf such that 1 is replaced by the output of the command awk 'NR==4 {...
- 13
1
vote
2
answers
111
views
How to append each line of a file to the corresponding line of another file?
I have 2 files of data and I'd like to add data from one file to the other, but rather than appending to the file as a whole I'd like to append line-by-line.
E.g. I have:
ip.txt
192.168.0.10
192.168.0....
- 13
2
votes
5
answers
114
views
sed: skip match if has already been replaced
I am playing with text automatically converting to markup.
I can make sed enclose some inline code:
echo "this is inline_code test 1_2 (_)" | sed -re "s/([a-z0-9]+_[a-z0-9]+)/\`\1\`/g;s/...
- 256
1
vote
5
answers
140
views
sed failing to detect combination of literal and escape characters
I have a long file, where lines are sometimes split up with an equal sign, followed by \r\n. Example:
Hello wor=\r\nld.
I want to delete all instances of =\r\n, so that the offending lines are joined ...
- 45
0
votes
1
answer
88
views
bash aws cloudshell for loop only returns the first item, then hangs [closed]
Pulling my hair out, looping over 100K S3 files in AWS Cloudshell; to copy in place to kick off a file processing lambda. when I run my for loop with an aws s3 cp on the dataset I want to, the for ...
- 35
3
votes
2
answers
80
views
sed - combine two commands into one command
The following two sed commands works.
sed -E "s#Example(.*)#\nconst ex='Example\1';\n#; s#Input:(.*)#const\1;#" in.txt | sed '/\S/!d'
However, how to rewrite them so that only one sed ...
- 606
1
vote
2
answers
70
views
BSD-Sed regex filtering to strip IP addresses on tcsh shell
I've been struggling with this for quite a while. Little differences in escaping and the inner workings of sed between BSD and linux throwing me off.
I want to do something deceptively simple: I have ...
- 91
1
vote
0
answers
73
views
Usage of sed in Windows OS via Make
I am working on Windows OS. And here is my makefile snippet.
COMPILER_ROOT := C:/Users/kpt
DATE := $(COMPILER_ROOT)/build/busybox_glob.exe date
SED := $(COMPILER_ROOT)/build/busybox_glob.exe sed
...
- 45
-1
votes
1
answer
167
views
Using sed to extract second apperance of a block [duplicate]
I have the following source file named test.txt:
text before
---BEGIN MARKER---
content 1
---END MARKER---
text between
---BEGIN MARKER---
content 2
---END MARKER---
text between
---BEGIN MARKER--...
- 3,753
1
vote
2
answers
83
views
Using sed -i within a loop
I'm reformatting a big file with sample metadata. I have a file (let's call it File2) with the group each sample belong to, with one id and pop per line. My idea was to while read over that file and ...
- 65
2
votes
4
answers
136
views
Can't get `sed` to work with text containing escape sequences
I want to be able to use sed (POSIX) to replace some text in the output from a command with a different message that includes escape sequences to allow me to set the colour.
To illustrate the problem.
...
- 417
3
votes
7
answers
333
views
Remove only odd lines' line breaks, with Vim, sed or awk
I have the following file:
line 1
abc
line 2
def
line 3
ghi
.....
.....
I need it to become:
line 1 abc
line 2 def
line 3 ghi
......
......
I know how to remove newlines, but not odd or even line ...
- 57
2
votes
8
answers
189
views
Remove a string character between 2 special characters in the headers of a fastq file
I have a fastq file containing several sequences with headers such as :
tail SRR11149706_1.fastq
@SRR11149706.16630586 16630586/1
CCCAACAACAACAACAGCAACCTCCTCACGCCAACGCCGATCCCGCCGCTGTTTTCCAA
@...
- 149
6
votes
5
answers
151
views
Unpivot a line in sed or awk retaining parent fields
Source lines formatting, for example:
Value11 | Value12 | ValueA,ValueB,ValueC
Value21 | Value22 | ValueA2
Desired output:
Value11 | Value12 | ValueA
Value11 | Value12 | ValueB
Value11 | Value12 | ...
- 2,023
-1
votes
1
answer
66
views
sed -e expression char 42 unterminated command `s' command [closed]
I have this sed command for adding a password to a file:
sed -i "s/password:/password: $escaped_password/" "$file"
If I replace the $escaped_password with the password, the script ...
- 551
7
votes
7
answers
140
views
How to filter word between two patterns - Linux
I would like to filter word between two patterns. In this case, between CN= and a comma (,)
Input:
Subject : "O=This is a test,CN=abc.def.com,L=North,ST=Arizona,C=CA"
Expected Output:
abc....
- 145
0
votes
4
answers
120
views
Is it possible to replace a digit with the next digit using regex?
What I want to do is say I have the multi-line string:
1.10...
1.11...
1.12...
1.13...
...
1.18...
I would like to match 1.1([0-9]) and replace with 1.1(\1+1) if \1 is the captured digit from the ...
-1
votes
4
answers
142
views
How to get substring between html tags in awk
I have a curl response with some plain strings like below -
"version":"1.1.8".
I would like to extract 1.1.8 from the raw text using sed or awk.
I tried the below ...
- 10.6k
0
votes
0
answers
35
views
Why `sed` behaves differently/errors with `-i` switch? (freebsd, bash) [duplicate]
TL;DR: Same sed commands (regexp, captures/edits) work in same environment (bash in freebsd 14), running from script, but fail when using -i switch. Why?
Background: Debugging a script on bsd (no x-...
- 182
0
votes
1
answer
33
views
How to use insert function in BSD/macOS sed
I'm trying to figure out how the i command in BSD/macOS sed works. From the man page, it looks like sed -e '/^/i\' -e $'\t' <<< a should print: <tab>a (where <tab> is a tab char, ...
- 2,075
5
votes
4
answers
84
views
sed delete and replace on top of file is not working as expected
I want to replace top 3 lines of my file. For this task, these two commands work fine, but then I get two .bak (saved) files...
These work:
sed -i.orig1 '1,3d' /my/path/file;
sed -i.orig2 '1s/^/line1 ...
- 1,489
3
votes
4
answers
113
views
How to insert a new line after a certain row number in a file in environment z/OSS USS (OMVS)
I am working in a USS environment, under z/OS. I need to insert a new line in a file, the line has to be inserted after a certain row number.
I've tried to use the command
sed '4i\newtext' file.txt ...
-1
votes
1
answer
189
views
Suggestions for making a file from a bigger file with grep or?
Looking for a suggestion that would be much faster. I have a large (232GB) file mongo backup. I want to take out only the April 24th lines and make a new file containing only this date or any date of ...
- 856
2
votes
4
answers
135
views
Surrounding whitespace-separated URLs with quotes using sed
Problem
I was trying to get sed command to do the same thing I could do with Python regex flavour, but I encountered some problems
Python regex example: (tested it on regex101 and it was working fine)
...
- 31
-2
votes
1
answer
92
views
sed parsing substring with a percentage sign
I can't seem to get the needed string with sed. Can someone help me see my error? I have tried so many combinations, but no luck. I know it has to be simple, but I'm at my wits' end.
echo '/documents/...
- 856
2
votes
5
answers
197
views
sed does not what I expect it to do
I want to do simple string replacements with sed to rename some files. Unfortunately it does not work the way I expect.
My Input (these files):
'Djrum - Under Tangled Silence - 01 A Tune For Us.flac'
'...
- 563
0
votes
4
answers
90
views
Update maven dependency with sed in Jenkinsfile
I have a Jenkinsfile which analyses some local artifactory server for new versions of the parent and dependencies. When I confirm that there is a new version, I want to replace the old version with a ...
- 3,442
-2
votes
1
answer
116
views
Extract Jira ticket number in GitHub Actions with sed [closed]
I have a requirement where I have to extract the Jira ticket number using sed regex. I am using a self-hosted Windows runner. I have used solutions from other similar posts on Stack Overflow but ...
- 349
11
votes
7
answers
1k
views
Reformat numbers, inserting separators at fixed positions
There are many lines in a file, like the following:
000100667 ===> 000102833
005843000 ===> 005844000
011248375 ===> 011251958
I would like to insert specific separators into the numbers ...
- 155
0
votes
3
answers
200
views
How to replace a question mark in sed
I want to replace a? with b? where ? is a character, not a wildcard. Here is what I tried:
echo "a?c" | sed 's/a\?/b?/'
I expect b?c, but it returns b??c. I also tried two backslashes ...
- 95
-4
votes
1
answer
98
views
Find and replace XML tag and not it's value [closed]
I have an XML file that contains the following data:
<Extrinsic name="CommodityVendor">1234567</Extrinsic>
<Extrinsic name="buyerVatID">1122334455</Extrinsic&...
- 101
1
vote
1
answer
111
views
Easiest way to collapse mkfs output (backspaces and spaces)
I am dumping the output of the mkfs into the log file, but it displays its process interactively, printing backspaces to get cursor back, then prints spaces to erase, then doing backspaces again and ...
- 751
6
votes
6
answers
187
views
Extracting strings from a file with sed and regular expressions
I would like to ask about extracting specific strings from a file using sed and regular expressions.
Below is the example of the input text file (testfile.txt):
# This file contains a short ...
- 89
-2
votes
5
answers
127
views
Bash getting a specific section from a file that spans multiple lines [closed]
I am using bash on Ubuntu server. My goal is to get the lines from specific section. I can't think of what tool to use to accomplish this (awk, grep -P, sed, ???). It is the specific section that I ...
- 856
0
votes
4
answers
187
views
How can I do a large number recursive word substitutions in a large file using bash in fast time?
This question is based on How do I fix sed commands becoming extremely slow when load is high? with the advice of @markp-fuso, @jhnc, and @Jetchisel to avoid a chameleon question as many of the ...
- 363
4
votes
7
answers
463
views
How do I fix sed commands becoming extremely slow when load is high?
I have a bash script that takes a simple properties file and substitutes the values into another file. (Property file is just lines of 'foo=bar' type properties)
INPUT=`cat $INPUT_FILE`
while read ...
- 363
2
votes
2
answers
89
views
insert line with command from one bash file to another
I would like to inject to in docker bash file 2nd line (after #!/bin/bash -x) such a command: set -a && . .env && set +a. I've tried this inside source.sh:
TEXT_INSERT='set -a &&...
- 51
-1
votes
5
answers
102
views
Adding an additional line of code to a file after the logical end of a statement with a match string
I have a file that is full of lines like:
snprintf(log_buffer,...
...
...
...
);
If I use sed, I can find these lines with "snprint.*...
0
votes
3
answers
100
views
Issue with Regex and SED matching Date in Apache date format
I was trying to match date in apache log in many format as you can see below
Regex101
https://regex101.com/r/Cug8lX/1
The regex is working but with sed, I can't succeed to make it match. It don't say ...
- 47
0
votes
9
answers
213
views
linux sed expression to select text between markers
Here is a challenge for regex gurus. Need a very simple sed expression to select text between markers.
Here is an example text. Please mind it can contain any special chars, TABS and white spaces even ...
- 523
4
votes
5
answers
184
views
How to extract links from an html page
I have an html page that has data like so:
<td><a href="test-2025-03-24_17-05.log">test-2025-03-24_17-05.log</a></td>
<td><a href="PASS_report_test_2025-...
- 389
3
votes
4
answers
98
views
Delete string that spans multiple lines
I have some extraneous html table rows I'd like to remove using sed. I want to match and delete these two lines.
[tr]
[/tr]
I've tried sed -i '/\[tr\](\r|\n|\r\n|\n\r)\[\/tr\]/d' ./file which matches ...
2
votes
5
answers
200
views
Picking out file names from grep results where file name contains numbers and hyphens
I have a script that runs a grep command and formats the results nicely for me, asking if I want to open any of the resulting files in an editor etc.
The core of my script is a command like this:
grep ...
- 10.1k
2
votes
2
answers
129
views
Remove a string containing a substring that will require a wildcard
First time I have had to post here to solve a problem. I am guessing I am missing something easy. Spent a bout four hours yesterday beating my head up against something I though was going to be simple....
- 31
0
votes
3
answers
86
views
duplicate line print when using sed to append data to the end of a block [closed]
I'm trying top use sed to find a block of text starting with "listener socks5 {" and ending with "}" and append a line to the end of the last "map" entry. here's the file ...
- 137
0
votes
2
answers
95
views
Cutting a string with 'read', but the variables returned don't seem to work
So, I'm trying to be clever and slice out the output of virsh via read so I can make something cool. (That's open to interpretation.)
However, my little script is not working like I expect. The ...
