Assigning output from awk to variables

Question

I'm trying to create a bash script that ingests the output of another script cpu_latency.bt

The output of cpu_latency.bt is generated every second and looks similar to:

@usecs: 
[0]                    3 |@@@@@@@@@@                                          |
[1]                    5 |@@@@@@@@@@@@@@@@@                                   |
[2, 4)                 5 |@@@@@@@@@@@@@@@@@                                   |
[4, 8)                 0 |                                                    |
[8, 16)                5 |@@@@@@@@@@@@@@@@@                                   |
[16, 32)              15 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@|
[32, 64)               1 |@@@                                                 |
[64, 128)              0 |                                                    |
[128, 256)             1 |@@@                                                 |

@usecs: 
[0]                    1 |@@@                                                 |
[1]                    1 |@@@                                                 |
[2, 4)                 6 |@@@@@@@@@@@@@@@@@@@@@@                              |
[4, 8)                 2 |@@@@@@@                                             |
[8, 16)                4 |@@@@@@@@@@@@@@                                      |

I am trying to capture only the first number after the [ and then the number before the first | (so in the last line above that would be 8 and 4

The script below is fairly close (with the exception of handling [0] and [1] lines):

duration=10
while read line
do
        echo $line | cut -d "|" -f1 | sed 's/\[//g; s/\,//g; s/)//g' | awk '{print $1,$3}' |
        while read key value; do
                print "The Key is "$key "and the Value is "$value
        done

done < <(timeout $duration cpu_latency.bt | grep "\[")

However the output it returns is not quite right:

Error: no such file "The Key is 16"
Error: no such file "and the Value is 10"
Error: no such file "The Key is 16"
Error: no such file "and the Value is 9"

Can recommend a better way of assigning the output of $1 and $3 to variables so I can write them out to a file?

Thanks CiCa

@RavinderSingh13 I'm not sure if I I'm misunderstanding how I can use an array for this, but a little more work with a while loop has gotten me considerably closer to what I'm aiming for:

while read key value
do  
    echo `hostname`".cpu-lat."$key"\\"$value"\\`date +"%s"`"  >> /tmp/stats.out
done < <(
timeout $duration /root/bpftrace/cpu_latency.bt | awk '
match($0,/^\[[0-9]+/){
  val=substr($0,RSTART+1,RLENGTH-1)
  match($0,/[0-9]+ +\|/)
  val2=substr($0,RSTART,RLENGTH)
  sub(/ +\|/,"",val2)
  print val,val2
  val=val2=""
}' )

What is the point of the shell loop in the final script in your question? You should read unix.stackexchange.com/q/169716/133219, mywiki.wooledge.org/Quotes, and mywiki.wooledge.org/BashFAQ/082 to learn how to deal with some of the general shell programming issues in your scripts and also run every shell script you write through shellcheck.net for help spotting some of the problems until you get some familiarity with shell programming. — Ed Morton
– Ed Morton, Commented Dec 15, 2019 at 16:26

Ed Morton · Accepted Answer · 2019-12-15 16:49:21Z

1

Is this what you're trying to do?

$ awk -F'[][,[:space:]]+' 'sub(/ \|.*/,""){print $2, $NF}' file
0 3
1 5
2 5
4 0
8 5
16 15
32 1
64 0
128 1
0 1
1 1
2 6
4 2
8 4

.

$ awk -v host="$(hostname)" -v date="$(date +%s)" -F'[][,[:space:]]+' '
    sub(/ \|.*/,"") { printf "%s.cpu-lat.%s\\%s\\%s\n", host, $2, $NF, date }
' file
mypc.cpu-lat.0\3\1576428453
mypc.cpu-lat.1\5\1576428453
mypc.cpu-lat.2\5\1576428453
mypc.cpu-lat.4\0\1576428453
mypc.cpu-lat.8\5\1576428453
mypc.cpu-lat.16\15\1576428453
mypc.cpu-lat.32\1\1576428453
mypc.cpu-lat.64\0\1576428453
mypc.cpu-lat.128\1\1576428453
mypc.cpu-lat.0\1\1576428453
mypc.cpu-lat.1\1\1576428453
mypc.cpu-lat.2\6\1576428453
mypc.cpu-lat.4\2\1576428453
mypc.cpu-lat.8\4\1576428453

edited Dec 15, 2019 at 16:49

answered Dec 15, 2019 at 16:43

Ed Morton

209k18 gold badges90 silver badges212 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

CiCa Over a year ago

That's exactly what I was trying to do! Thank you Sir. And Thank you to all other contributors of this thread, a lot learnt, and even more to learn :)

Luuk · Accepted Answer · 2019-12-15 16:36:07Z

1

sed -e 's/[^0-9]/ /g' lala  |gawk '{ print $1, substr($0,20,5) }'

The 'sed' remove all non-numbers The 'awk' will print first argument, and from position 20, 5 characters of text. (Which can be changed to numeric by addin 0 to it):

sed -e 's/[^0-9]/ /g' lala  |gawk '{ print $1, 0+substr($0,20,5) }'

Of course, Ed is right!:

gawk '{ gsub(/[^0-9]/," "); print $1, 0+substr($0,20,5) }'

edited Dec 15, 2019 at 16:36

answered Dec 15, 2019 at 14:15

Luuk

15.4k5 gold badges28 silver badges44 bronze badges

1 Comment

Ed Morton Over a year ago

You never need sed when you're using awk. sed -e 's/[^0-9]/ /g' lala |gawk '{ print $1, substr($0,20,5) }' = awk '{gsub(/[^0-9]/," "); print $1, substr($0,20,5) }' lala.

RavinderSingh13 · Accepted Answer · 2019-12-15 17:36:26Z

1

EDIT: After seeing OP's attempt from my code, adding following now.

awk -v host="$(hostname)" -v date="$(date +%s)" '
match($0,/^\[[0-9]+/){
  val=substr($0,RSTART+1,RLENGTH-1)
  match($0,/[0-9]+ +\|/)
  val2=substr($0,RSTART,RLENGTH)
  sub(/ +\|/,"",val2)
  printf("%s.cpu-lat.%s\\%s\\%s\n", host, val, val2, date)
  val=val2=""
}' Input_file

Could you please try following. I have mentioned Input_file to pass to awk here in case you want to pass s command's output into awk then try like: your_script | following awk code without Input_file

awk '
match($0,/^\[[0-9]+/){
  val=substr($0,RSTART+1,RLENGTH-1)
  match($0,/[0-9]+ +\|/)
  val2=substr($0,RSTART,RLENGTH)
  sub(/ +\|/,"",val2)
  print val,val2
  val=val2=""
}' Input_file

edited Dec 15, 2019 at 17:36

answered Dec 15, 2019 at 14:14

RavinderSingh13

135k14 gold badges61 silver badges100 bronze badges

Collectives™ on Stack Overflow

Assigning output from awk to variables

3 Answers 3

1 Comment

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related