Get The Redirect URL that has json Data Using CURL

Question

I have URL like this www.example.com/abc/xyz and when it is loaded it turns into the different url like:

www.example.com/abc/xyz#facet=c_State+s_FL+p14_2+r14_5g6tVndJ+n10_2+x10_2BLnMX+b5_0+h5_0+g3_0+f3_0+v_No Preference+t8_0+a30_0+u_0+k_0+q_0+w_false+j_Q+e_1+i_2

I just want the second url ..so that I can use that parameters to send the json data using curl.

The second url is requesting the json data so that I am not able to track it.I want that data using first URL.

Here is my code:

$url = "http://www.lennar.com/New-Homes/Florida/Tampa";
$ch = curl_init();
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION , false);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, array(
    'Content-Type: application/json',
    )
);
curl_setopt($ch, CURLOPT_URL, $url);
$result = curl_exec($ch);
curl_close($ch);
var_dump($result);

and My output is :

string '



    <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">



    <html xmlns="http://www.w3.org/1999/xhtml" lang="en-us" xml:lang="en-us" xmlns:fb="http://ogp.me/ns/fb#" xmlns:og="http://opengraphprotocol.org/schema/">

    <head id="Head1"><meta name="keywords" content="quality, new homes, home builder" />

    <meta name="description" content="Lennar Homes: If a home builder provided everything you want and everything you need, and by doing so, was a'... (length=165671)

Do you want curl to prevent the redirection to the second url ? — Himal
– Himal, Commented Sep 26, 2014 at 10:11
NO..I just want the second URL. But it is not getting in curl_getinfo($ch, CURLINFO_EFFECTIVE_URL); — chirag
– chirag, Commented Sep 26, 2014 at 10:19
Yes, that's what i meant. to get the second url you have to prevent the redirection and get the second url from the Location header.is that what you want ? — Himal
– Himal, Commented Sep 26, 2014 at 10:21
Have you tried setting CURLOPT_FOLLOWLOCATION to 'false' ? — Himal
– Himal, Commented Sep 26, 2014 at 10:22
I am using curl and I cant prevent the redirection ..Because on the base of the second url's data I will make another request.I just want second url anyhow my friend. — chirag
– chirag, Commented Sep 26, 2014 at 10:22

Community · Accepted Answer · 2017-05-23 11:57:05Z

1

The so called "second URL" that you desire to get is generated by JavaScript.

Notice that you are not redirected by the first URL to the second one. The JS generated under the first URL after page load just adds some data to the first URL.

Neither curl, nor wget or file_get_contents will get you what you want, since none of those parses/executes JavaScript code.

If you want to simulate creating the second URL, you could try inspecting JS on that page (I'd say bad idea, needs loads of work and is vulerable to external logic changes) or try looking at SO: Get sourcecode after javascript execution with curl, and then using JS engine to get the second URL.

If you want to play with simulating the URL, I'd look at var facetContextJSON present in loaded HTML - it's some kind of config JSON, that you can get by cURL, regexp+parse it and figure out how to build the URL you actually need.

edited May 23, 2017 at 11:57

CommunityBot

11 silver badge

answered Sep 26, 2014 at 11:07

Kleskowy

2,6681 gold badge18 silver badges19 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

chirag · Accepted Answer · 2014-10-01 07:48:12Z

0

I have found the way to get the json parameters which I was not able to get through curl. Here is the solution :

$.get(
                                        'http://www.example.com',
                                        function(response) {
                                           //your response as html of that page

                                        });

From the above script I got whole html of that page and also get the json data.Then by using the string operations I found exact variables which I wanted for my next url.

answered Oct 1, 2014 at 7:48

chirag

61310 silver badges14 bronze badges

Collectives™ on Stack Overflow

Get The Redirect URL that has json Data Using CURL

2 Answers 2

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related