Make curl not overwrite files but change the name automatically?

Question

I have a file generated by discord with about 8000 lines of curl commands and most of them are similar to one another. The deal is I can't edit every line and add a different output statement to each of them. is there any way of making curl automatically assign a different name to the file so that it doesn't overwrite the previous one?

curl 'https://cdn.discordapp.com/emojis/585807750991183872.png?v=1' \
    -H 'sec-fetch-mode: no-cors' -H 'accept-encoding: gzip, deflate, br' \
    -H 'accept-language: en-US' \
    -H 'user-agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) discord/0.0.90 Chrome/76.0.3809.94 Electron/6.0.0 Safari/537.36' \
    -H 'accept: image/webp,image/apng,image/*,*/*;q=0.8' \
    -H 'referer: https://canary.discordapp.com/channels/584291323893383169/607165146476969984' \
    -H 'authority: cdn.discordapp.com' \
    -H 'cookie: __cfduid=d36533327b001d6036b90a8c278c91d491562662533' \
    -H 'sec-fetch-site: same-site' \
    --compressed \
    --output emoji.png ;

The --output statement is added by me since I can use the sublime split into lines function to edit the end of each lines separately. all of the endings did not contain --output and stopped at --compressed (tho I don't know if it's necesary)

TL;DR is there any way of making curl download each file with a different name? like file1 then file 2 and so on?

Are all the commands of the same form, and one-per-line or something? It might be possible to script the editing of the file — muru
– muru, Commented Aug 3, 2019 at 15:26
Your note that you added the --output statement is a bit perplexing... curl normally sends output to stdout, which seems unuseful when downloading files in graphic formats. How do you specify the output file in your real-life use case? — Jim L.
– Jim L., Commented Aug 3, 2019 at 20:13
@muru yes they are mostly the same with a only a few expections regarding canary links which are longer — Iverol
– Iverol, Commented Aug 5, 2019 at 22:25

Jim L. · Accepted Answer · 2019-08-03 20:53:23Z

You could write a shell function to put a wrapper around curl and parameterize your calls, something like:

fetch_via_curl() {
# $1 is URL to fetch
# $2 is referrer string
# $3 is cookie string
# $4 is output file name

  curl_fmt="$(cat << 'EOF'
curl '%s' \
    -H 'sec-fetch-mode: no-cors' -H 'accept-encoding: gzip, deflate, br' \
    -H 'accept-language: en-US' \
    -H 'user-agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) discord/0.0.90 Chrome/76.0.3809.94 Electron/6.0.0 Safari/537.36' \
    -H 'accept: image/webp,image/apng,image/*,*/*;q=0.8' \
    -H 'referer: %s' \
    -H 'authority: cdn.discordapp.com' \
    -H 'cookie: %s' \
    -H 'sec-fetch-site: same-site' \
    --compressed \
    --output '%s' ;
EOF
)"

  file_num=
  while [ -f "$4${file_num:+.${file_num}}" ]
  do
    file_num=$(($file_num+1))
  done
  curl_cmd="$(printf $curl_fmt "$1" "$2" "$3" "$4${file_num:+.${file_num}}")"

  sh -c "$curl_cmd"
}

Then fetch your files with syntax like:

fetch_via_curl \
    "https://cdn.discordapp.com/emojis/585807750991183872.png?v=1" \
    "https://canary.discordapp.com/channels/584291323893383169/607165146476969984" \
    "__cfduid=d36533327b001d6036b90a8c278c91d491562662533" \
    "emoji.png"

If emoji.png does not exist, the file will be saved to emoji.png. If it does exist, the file will be saved to emoji.png.N where N is the smallest positive integer that results in a filename that does not already exist.

Stack Exchange Network

Make curl not overwrite files but change the name automatically?

1 Answer 1

You must log in to answer this question.

Hot Network Questions

Make curl not overwrite files but change the name automatically?

1 Answer 1

You must log in to answer this question.

Related

Hot Network Questions