Why is cURL decoding my URL encoded values when POSTing?

Question

When using cURL to send data via POST, if that data string is URL encoded or if parts of it are URL encoded, cURL automatically decodes the data when sending it.

This happens when using cURL in PHP or directly in the command line.

I've tested with 2 different version of cURL: 7.19 and 7.49. Both exhibit the same behavior

I've sent the cURL request from two different servers thinking that the the way the servers was configured somehow influenced this, but the result was the same.

Here is a simple PHP cURL request that I've used for my test:

$ua = 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/525.13 (KHTML, like Gecko) Chrome/0.A.B.C Safari/525.13';

$data = "https%3A%2F%2Fexample.com%3A8081%2Ftemoignez%3FQid%3D%26"       

$ch = curl_init( "https://example.com/test/webhook.php" );
curl_setopt($ch, CURLOPT_USERAGENT, $ua);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, "payload=".$data);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);

$response = curl_exec( $ch );

The data will be sent decoded even though the initial string is URL encoded.

I'm retrieving the data by dumping the POST data into a file on disk using PHP.

Is this normal? Any idea what may cause this?

That is irrelevant to my question, it is project specific that the URLs are to be encoded — Remus Negrota
– Remus Negrota, Commented Sep 26, 2016 at 11:51
$response = curl_exec( $ch ); is probably why curl is decoding the url I'm not sure how you would disable it (if its possible) short of searching php.net under Curl and see if there is an option to turn that part off. I'm interested to find that out as well — ben jay hutton
– ben jay hutton, Commented Sep 26, 2016 at 12:16
Wouldn't reading from php://input instead of $_POST actually be a solution to this problem? — simon
– simon, Commented Sep 26, 2016 at 13:24

Daniel W. · Accepted Answer · 2016-09-26 12:34:13Z

3

You have two different assertions here:

cURL automatically decodes the data when sending it.

...

I've simply dumped the POST data into a file after retrieving it.

It is PHP that automatically DECODES the data when receiving it. It is NOT getting decoded upon sending it!

This integrates with the behaviour of other values, like cookie data, post and get variables, header information like referrer, ... everything get's decoded automatically when it is received, because it is expected to be sent encoded.

When you want to see the exact data that is getting send over the wires, use a tool like ngrep on port 80 to sniff the TCP HTTP traffic.

answered Sep 26, 2016 at 12:34

Daniel W.

32.6k15 gold badges99 silver badges155 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

Remus Negrota Over a year ago

Then how come URL encoded data sent with Postman and Hurl.it remains encoded when received?

deceze Over a year ago

@Remus Triple check what Postman actually sends. It's probably encoding your data for you, so you're sending it double-encoded.

Remus Negrota Over a year ago

@deceze I've checked with Postman documentation and it specifically mentions that parameters entered in the URL bar or in the key/value editor will not automatically be URL-encoded

Remus Negrota Over a year ago

@deceze I've also double checked with Charles Proxy. Anyway to triple check? :)

deceze Over a year ago

@Remus You'll have to provide a compelling demonstration. Fact is that data in $_POST is decoded by PHP. Your example with curl behaves exactly as one would expect.

|

Collectives™ on Stack Overflow

Why is cURL decoding my URL encoded values when POSTing?

1 Answer 1

6 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

6 Comments

Your Answer

Sign up or log in

Post as a guest

Related