Possible to urlencode a variable in a shell script?

Question

Is it possible to url encode a variable within a shell script?

#!/bin/bash 

now=$(date +"%T") 

DATA=$(wget -q -O -   "http://someurl.com/x.htm?callback=webRequest&exthrs=1&extMode=&fund=1&entitlement=0&skipcache=&extendedMask=1&partnerId=2&output=json&noform=1")

wget -q -O - "http://somewhere.com?abc=$1&responseData=$DATA"

echo "-----COMPLETE----- $now   $1 $RANDOM  
"

I want to url encode the DATA variable, since its results have & in it, it messes up the params in the second wget, is there a way to url encode that DATA variable without using PHP to url encode?

John1024 · Accepted Answer · 2015-04-20 18:52:46Z

13

Here is one method for URL encoding the shell string DATA:

DATA=$(python -c "import urllib, sys; print urllib.quote(sys.argv[1])"  "$DATA")

Here is another:

DATA=$(perl -MURI::Escape -e 'print uri_escape($ARGV[0]);' "$DATA")

answered Apr 20, 2015 at 18:52

John1024

115k15 gold badges152 silver badges183 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

Sven Kahn Over a year ago

Yes perfect, they both work (y) thanks. I thought maybe there was a builtin way in shell to do it, but it's okay. Out of perl/python which one has less spend on CPU/Mem you think? Many instances of this script are to be ran concurrently, so every bit helps.

John1024 Over a year ago

@SvenKahn On my system (Debian stable), the perl version is faster.

Sven Kahn Over a year ago

Thank you, I appreciate that you looked into it.

smartblonde Over a year ago

I tried this but it converts = signs which messes up some receiving servers.

user532477 Over a year ago

@smartblonde - It should include equal signs because that's part of the specification (w3schools.com/tags/ref_urlencode.asp?_sm_au_=iVVDMg0TSmrMV6Dm). You are not supposed to encode the entire query string (the "?a=b&c=d" string), you only encode the names and values (the "a", "b", "c" and "d") – not the question mark, equal signs, or the ampersands.

Hari S. Khalsa · Accepted Answer · 2021-08-26 18:23:57Z

4

By the way, the way to do this in Python 3 is this:

DATA=$(python3 -c "import urllib.parse, sys; print(urllib.parse.quote(sys.argv[1]))" "$DATA")

answered Aug 26, 2021 at 18:23

Hari S. Khalsa

1311 silver badge4 bronze badges

Collectives™ on Stack Overflow

Possible to urlencode a variable in a shell script?

2 Answers 2

5 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

5 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related