Scraping data with php

Question

I am a beginner in scraping and I am using PHP simple_html_dom to scrape data from a website. My current code is not displaying any results. Maybe I don't target proper html tag. Second thing is I need that if there are no results for searched query that code displays message: "Results not found" or something like that. Any help is appreciated.

Here are sample queries:

3lnhl2gc9br764854 1J4FF28SXXL550156

  <?php 



require "simple_html_dom.php";

$trazi=$_POST['trazi'];

  $url="http://lookupvin.com/check/";



$ch = curl_init();

curl_setopt($ch, CURLOPT_URL,$url);

curl_setopt($ch, CURLOPT_POST, 1);

curl_setopt($ch, CURLOPT_POSTFIELDS,

            "VIN=$trazi");

curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);

$server_output = curl_exec ($ch);

curl_close($ch);

    $html = str_get_html($server_output);

    foreach($html->find('p.nmar') as $element)

  echo $element->innerText();


?>

index.php
<form action="vin.php" method="POST">
    <input type="text" name="trazi">
    <input type="submit">
</form>

what kind of value you input in trazi. post an example of it. — ameenulla0007
– ameenulla0007, Commented Jan 27, 2016 at 17:43

ameenulla0007 · Accepted Answer · 2016-01-28 17:42:22Z

1

include "simple_html_dom.php";
    $trazi="1J4FF28SXXL550157";
    $url="http://lookupvin.com/check/";
    $ch = curl_init();
    curl_setopt($ch, CURLOPT_URL,$url);
    curl_setopt($ch, CURLOPT_POST, 1);
    curl_setopt($ch, CURLOPT_POSTFIELDS, "VIN=$trazi");
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
    $server_output = curl_exec ($ch);
    curl_close($ch);

$html = new simple_html_dom();
$html->load($server_output);
$items = $html->find('.nmar');

if(count($items)!=0) {
    foreach($items as $post) {
        echo $post->children(0);
        echo "<br>";
    }
}
else {
    echo "Wrong Input";
}

found a better class online for better html parsing. http://code.tutsplus.com/tutorials/html-parsing-and-screen-scraping-with-the-simple-html-dom-library--net-11856 you can download it from here.

the result what i received.

edited Jan 28, 2016 at 17:42

answered Jan 27, 2016 at 17:51

ameenulla0007

2,6831 gold badge15 silver badges15 bronze badges

Sign up to request clarification or add additional context in comments.

13 Comments

fredcampbell Over a year ago

i've removed it and it still doesn't show anything. can you please check if html tag is properly set?

ameenulla0007 Over a year ago

actually server returns some error.. if user triggers the same url from same ip.. something like this.. ERROR: Too many attempts. Please contact us in order to get access to the VIN decoder API.

fredcampbell Over a year ago

Ok let's try another site where i have been actually able to pull results but i haven't managed to figure out how to print not found message if there is no reord for that query. I've updated the script.

fredcampbell Over a year ago

Can you figure it out?

ameenulla0007 Over a year ago

yup trying it.. but the new site excepts post requests. trying with those aspects now..

|

Oleg Shemetov · Accepted Answer · 2016-01-27 17:48:28Z

0

Try removing CURLOPT_POST and CURLOPT_POSTFIELDS parameters. The server responses to GET request, no need to do POST. you pass the parameter properly as part of the query string. Also check out http://php.net/manual/en/function.curl-error.php, this may help you check if your request was successful.

answered Jan 27, 2016 at 17:48

Oleg Shemetov

5003 silver badges16 bronze badges

10 Comments

Oleg Shemetov Over a year ago

are you getting any response whatsoever? print the $server_output, check the error codes like this if(curl_exec($ch) === false) { echo 'Curl error: ' . curl_error($ch); }

fredcampbell Over a year ago

It seems that html tag is not properl set

Oleg Shemetov Over a year ago

let's ensure your selector works. instead of echo $element->innerText(); do smth like echo 'found'. if it works it should display 'found' several times. maybe 'table' element itself has no innertext, so you need to loop through its children.

fredcampbell Over a year ago

it still prints out entire page. Can you run script on your machine to check it out? You need index.php to accpet $trazi and vin.php to preform search and print results.

Oleg Shemetov Over a year ago

` foreach($html->find('table.table-striped') as $element) { foreach ($element->find('td') as $row) { echo $row->innerText(); } break; } ` sorry, not sure how to format. as I suspected, the table itself has no contents, only its children do. the break is because there are 4 tables with same class, you only need first.

|

Collectives™ on Stack Overflow

Scraping data with php

2 Answers 2

13 Comments

10 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

13 Comments

10 Comments

Your Answer

Sign up or log in

Post as a guest

Related