0

I am trying to scrape a website in order to extract XHTML data to save as variables in PHP.

The website uses AJAX which means the data I am seeking is returned following an XMLHttpRequest using a search string such as http://website.com/ajax.php?mid=2&pid=4

I have tried setting the request headers obtained from both Chrome and Firefox (which differ slightly) and included them trying both curl_setop() and stream_context_create() options with file_get_contents(), but the data is still not being returned. When I check the request headers of my script in Chrome it is displaying the method as GET even though this has been set to POST. It does this for both cURL and stream contexts.

The website does not require a login, but I have noticed that it sends session cookies as part of the request, which I have also included in the HTTP request to no avail.

One other point to note is that removing the Content-Length value from the request header returns a blank page immediately, whereas leaving it in stays loading for around one minute before timeout, but this may be irrelevant.

My questions are has anyone else had experience with PHP and AJAX requests using HTTP headers and why am I seeing a GET request when I have specified this as POST?

Thankyou in advance for any insights.

9
  • 1
    We can't really give insights without seeing the actual code you've tried. Commented Apr 15, 2013 at 12:28
  • please share your code Commented Apr 15, 2013 at 12:33
  • Which site is it? Maybe can help you out by knowing that. Commented Apr 15, 2013 at 12:40
  • uk.msi.com/product/vga/ Commented Apr 15, 2013 at 12:43
  • 1
    remember to use curl_setopt($handle, CURLOPT_POST,1). Also server may require the data to come as POST data and not as parameters in the URL, use CURLOPT_POSTFIELDS. Commented Apr 15, 2013 at 12:44

1 Answer 1

2

This is a way to do POST in CURL:

<?php
session_start();
if( empty( $_SESSION ) ) {
  header('Location: /directar/index.php/site/login/return/Test');
}
echo "Info de sesion:";
var_dump( $_SESSION );

$c = curl_init('http://ws048-nueva/directar/index.php/AccessTest/CheckAcess');
$parametros_post = 'action=verChau';
curl_setopt($c, CURLOPT_POST, true); 
curl_setopt($c, CURLOPT_POSTFIELDS, $parametros_post); 
curl_setopt($c, CURLOPT_VERBOSE, TRUE);
curl_setopt($c, CURLOPT_COOKIE, 'PHPSESSID=' . $_COOKIE['PHPSESSID']);
curl_setopt($c, CURLOPT_RETURNTRANSFER, true);
session_write_close();
$page = curl_exec ($c);
echo "<br>";
echo "<br>";
echo  $page;
curl_close ($c);

if( $page === 'true' ) {
  echo "<br>si";
}
else {
  echo "<br>no";
}

?>

This was done by me to use CheckAccess method in Yii to return true or false if the current logged in user has permissions to do the action I send as POST parameter, and it's working. (I am passing session info cookie).

Sign up to request clarification or add additional context in comments.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.