I am trying to collect reviews on a google map webpage, however i can't find a way to scroll down the page to get all the reviews.
I'm using python 3 and Selenium package,
I've found different ways to scroll down social media pages such as facebook or IG, but the code doesn't work on google map. Also, I tried to locate the end key of the body tag, didn't work neither.
I'll be happy if someone can help,
thank you in advance,
You can use the execute_script from selenium to scroll to the bottom of the page.
browser = webdriver.Chrome('chromedriver.exe', options=self.browserProfile)
brownser.get('your url here')
browser.execute_script("window.scrollTo(0, document.body.scrollHeight);")
If this does not scroll on the correct area you can perform a click on the div containing the reviews and then perform the window.scrollTo
review_box= lambda: self.browser.find_element_by_xpath("xpath to div")
review_box().click()
Hope this helps! :)
One way to scroll down with selenium can be to click on a particular element on the page. If you do the following...
reviews_divs = driver.find_elements_by_class_name('section-review')
reviews_divs[-1].click()
... it will click on the last review and therefore scroll to it. Good but not yet what you want since the page is not loading new results...
However if you click on the first div after the section-review div, new reviews are now properly loaded:
driver.find_element_by_class_name('section-loading').click()
EDIT
The code above will be effective once you have clicked on the "Show all reviews" when you land on the store page you want to test:
driver.find_element_by_css_selector("button[class*='__button-text'").click()
