Unable to scrape particular data from a URL

Question

I want to scrape this URL.

I am using phpQuery library for this purpose.

I want to scrape #weedmenuPane from that link which contains some items and its prices.

Problem is, when I get scraped HTML, the required DIV is not in the scraped HTMl.

I have checked with Firebug but it is not coming from any AJAX call.

I thought it was coming by some kind of JS operation after HTML is loaded but I have also checked by disabling JS in my browser, and that DIV is there, means not coming by any JS operation.

I want to know how it is coming and how it is being generated.

Here is code I have tried.

 $html = file_get_contents($link, false, $context);
 $html = phpQuery::newDocumentHTML($html);

 echo pq("",$html);

 echo pq("#weedmenuPane",$html)->html() . " here";

Please provide some of the code that already does part of what you described. — dakab
– dakab, Commented Aug 23, 2015 at 8:08
ok. i posted only relevent code ..... and echo pq("",$html); has whole page HTML in it but not the desired DIV in it ... and the second echo is empty obviusly — Arslan Akbar
– Arslan Akbar, Commented Aug 23, 2015 at 8:11
Just dump the html and look at it or connect through a Fiddler-style debug proxy. — pguardiario
– pguardiario, Commented Aug 23, 2015 at 23:43

chalasr · Accepted Answer · 2015-08-23 11:13:02Z

0

So, I have downloaded phpQuery and written a little test with your url.

This :

<?php

require 'phpQuery-onefile.php';

$url= "https://weedmaps.com/deliveries/green-door-west";
$HTML = file_get_contents($url);
$pq = phpQuery::newDocumentHTML($HTML);
$div = pq('#weedmenuPane', $pq);
echo $div; // Content of #weedmenuPane
// or with your code
echo pq("#weedmenuPane", $pq)->html() . " here";
?>

Return the content of the #weedmenuPane DIV.

So try to use this code or show me your code and we will find your problem.

edited Aug 23, 2015 at 11:13

answered Aug 23, 2015 at 10:53

chalasr

13.2k5 gold badges44 silver badges83 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Unable to scrape particular data from a URL

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related