For loop runs twice,

Question

So I am fairly new to bash scripting, but I want a bash job that will:

Create a post-dated timestamp. (either as a var, or to a file)

Read lines from a file named feeds.txt.

Separate the number string from the whole string into two variables $feedNum and $wholeString.

Then execute a cat command to create or append a new file named $wholeString append the contents of timestamp.txt to that file.

Lastly, execute a wget command to insert $feedNum into the url string, and output the response of the wget to the previously created $wholeString file.

Here's what I have.

feeds.txt looks like (it will eventually be longer):

8147_feed_name.xml
19176_nextfeed_name.xml

And the script I've cobbled together looks like this.

#Changes the directory to the correct location
cd /home/scripts
# Inserts the proper timestamp in the file , postdating for one hour.
date -d '1 hour ago' "+%m/%d/%Y %H:%M:%S" >/home/scripts/timestamp.txt

#Loop Logic
for feedNum in `cat feeds.txt |cut -d _ -f 1`;do
   for  wholeString in `cat feeds.txt`; do
    cat /home/scripts/timestamp.txt >>/home/scripts/tmp/$wholeString ;
    wget https\://some.url.com/reports/uptimes/${feedNum}.xml?\&api_key=KEYKEYKEYKEY\&location=all\&start_date=recent_hour -O ->>/home/scripts/tmp/$wholeString;
done
done

My issue is, that it runs 4 times, and so if I ditch cat and wget and replace them with a simple echo, like this,

#Changes the directory to the correct location
cd /home/scripts
# Inserts the proper timestamp in the file , postdating for one hour.
date -d '1 hour ago' "+%m/%d/%Y %H:%M:%S" >/home/scripts/timestamp.txt

#Loop Logic
for feedNum in `cat feeds.txt |cut -d _ -f 1`;do
   for  wholeString in `cat feeds.txt`; do
    echo $feedNum $wholeString
done
done

I get an output like this, where the top and bottom lines are correct. The middle two are a mix-mash.

8147 8147_feed_name.xml
8147 19176_nextfeed_name.xml
19176 8147_feed_name.xml
19176 19176_nextfeed_name.xml

I understand why it is doing this, but not how to fix it.

if $feedNum is unique, you could grep it from the file in the inner loop. Or, reorganize your loop to read feeds.txt line-by-line and cut out the two pieces you want inside one for loop. — Jeff Schaller
– Jeff Schaller ♦, Commented May 12, 2016 at 17:14

DopeGhoti · Accepted Answer · 2016-05-12 18:15:45Z

1

This should do the trick for splitting the string (though as always, be wary of spaces in your filenames):

#Changes the directory to the correct location
cd /home/scripts
# Inserts the proper timestamp in the file , postdating for one hour.
date -d '1 hour ago' "+%m/%d/%Y %H:%M:%S" >/home/scripts/timestamp.txt

#Loop Logic
while IFS=_ read feedNum xmlFile; do
  echo "$feedNum $xmlFile"
done < feeds.txt

It should be trivial to assemble your wget call from here.

edited May 12, 2016 at 18:15

answered May 12, 2016 at 17:37

DopeGhoti

79.3k10 gold badges107 silver badges141 bronze badges

Why not skip the cuts and use while IFS=_ read -r feedNum xmlFile; do ... ?

steeldriver
– steeldriver

2016-05-12 18:12:43 +00:00
Commented May 12, 2016 at 18:12
Because I didn't think of it at the time? I'll roll that in; thanks.

DopeGhoti
– DopeGhoti

2016-05-12 18:14:38 +00:00
Commented May 12, 2016 at 18:14
... also might be worth suggesting using a variable (i.e. command substitution rather than redirection to a file) for the date string?

steeldriver
– steeldriver

2016-05-12 18:17:42 +00:00
Commented May 12, 2016 at 18:17
I thought of it, but wanted to keep it at least superficially similar to the OP's script to allow it to be more clear where I was going. timestamp=$(date -d '1 hour ago' "+%m/%d/%Y %H:%M:%S") isn't a terrible idea though- why write to disk when you don't need to?

DopeGhoti
– DopeGhoti

2016-05-12 18:35:45 +00:00
Commented May 12, 2016 at 18:35

Add a comment |

user170121 · Accepted Answer · 2016-05-12 19:13:56Z

OP Here, as a different user name. I ended up using the first code you posted DopeGhoti. With some slight modification. Thanks for showing me how to grab a line, and then execute from it rather than trying to re-read the source file and pull for each.

 #Create Timestamp
    date -d '1 hour ago' "+%m/%d/%Y %H:%M:%S" >/home/scripts/sitename/timestamp.txt

    #Loop Logic
    while read line; do
       feedNum="$(echo "$line" | cut -d_ -f1)"
       wholeString="$(echo "$line")"
       cat /home/scripts/sitename/timestamp.txt >>/home/scripts/sitename/$wholeString
       wget https\://my.sitename.com/reports/uptimes/${feedNum}.xml?\&api_key=KEY\&location=all\&start_date=recent_hour -O ->>/home/scripts/sitename/$wholeS$
    done < feeds.txt

Which resulted in the correct output for all 18 feeds we had.

Thank you!

Stack Exchange Network

For loop runs twice,

2 Answers 2

You must log in to answer this question.

Hot Network Questions

For loop runs twice,

2 Answers 2

You must log in to answer this question.

Related

Hot Network Questions