Bash scripting curl request with twist

Question

I'm attempting to pull in data from an API using curl, and have the majority of the script done, but need some assistance. The API has results in multiple pages I need to retrieve, and I've been able to script that function, but I can't figure out how to make the loop stop. Also, I need some assistance in figuring out how to break each pages data up into individual events. Each event starts with the same string, so I know the text to use in order to break up the events, but I don't know the syntax for doing so. Can someone assist with that as well? My code is below:

#!/bin/sh
string='next\":null'
  for ((i=1;until $string;i++)); do
    curl -H "Authorization: Token insertlongtokenhere" -H "Content-Type: application/json" https://api.app.secunia.com/api/tickets/?page=$i
done

Using this string method gives me a

./secunia.sh: line 3: ((: until next\":null: syntax error: invalid arithmetic operator (error token is "\":null")

I've tried other ways of commenting out the double quotes to no avail, so I'm kind of stuck.

You can test for the end inside the for loop and use the break statement to stop the loop. — Barmar
– Barmar, Commented Mar 17, 2017 at 20:43
And I thought I was done, but I'm not. I was hoping to get Splunk to break up the content of the pages into events, but it's unable to. I'm trying to sed the comma that's in between the events out, but it's not going well. This is a portion of the json coming in "last_updated":"2017-02-28T17:56:19Z"},{"id":588699,"name":null, and this is the sed line I'm trying sed -e "s/},{/}+{/" -e "s/}[^}]*$/}/" secunia.txt | tr "+" "\n", and I've put it outside the for loop, but it's not pulling out the ,. What am I missing? — manderson
– manderson, Commented Mar 20, 2017 at 20:00
You should post a new question, since this is a different problem. — Barmar
– Barmar, Commented Mar 20, 2017 at 20:02

Barmar · Accepted Answer · 2017-03-20 15:28:06Z

1

Take the end test out of the for header, and test for it inside the loop. You can then use break to end the loop.

for ((i=1; ; i++)); do
    contents=$(curl -H "Authorization: Token insertlongtokenhere" -H "Content-Type: application/json" https://api.app.secunia.com/api/tickets/?page=$i)
    echo "$contents"
    if [[ $contents =~ 'next":null' ]]
    then break
    fi
done

edited Mar 20, 2017 at 15:28

answered Mar 17, 2017 at 20:47

Barmar

789k57 gold badges555 silver badges669 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

manderson Over a year ago

The script still tries downloading the next page after the last page. This is the first applicable characters that show the last page is the last page: {"count":66,"next":null, while {"count":66 is on every page. Also, thank you for attempting to assist. Edit: Same thing occurs when I make the break line '"next":null'.

Barmar Over a year ago

Need a $ before contents. I thought [[ didn't require this, but it does for this.

Collectives™ on Stack Overflow

Bash scripting curl request with twist

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related