Python - selenium - google map not visible in page_source

Question

I am trying to scrape GPS coordinates data off a property listing

https://www.primelocation.com/new-homes/details/55357965?search_identifier=d10d089a335cc707245fb6c924bafbd2

The coordinates are located on the "Map & Nearby" tab in the following HTML block of code:

<a target="_blank" rel="noopener" href="https://maps.google.com/maps?ll=51.517204,-0.126447&amp;z=15&amp;t=m&amp;hl=en-GB&amp;gl=US&amp;mapclient=apiv3" title="Open this area in Google Maps (opens a new window)" style="position: static; overflow: visible; float: none; display: inline;"><div style="width: 66px; height: 26px; cursor: pointer;"><img alt="" src="https://maps.gstatic.com/mapfiles/api-3/images/google_white5_hdpi.png" draggable="false" style="position: absolute; left: 0px; top: 0px; width: 66px; height: 26px; user-select: none; border: 0px; padding: 0px; margin: 0px;"></div></a>

Specfically: https://maps.google.com/maps?ll=51.517204,-0.126447&z=15&t=m&hl=en-GB&gl=US&mapclient=apiv3

The intended output is to extract the coordinates, and have them printed out, i.e.

Latitude: 51.517204
Longitude: -0.126447

With the help of fellow SO users, I have managed to get to "Map & Nearby" tab, where I was hoping I could then just go to appropriate HTML block, and regex the coordinates.

The code so far is as follows:

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.action_chains import ActionChains
import time

chromeOptions = webdriver.ChromeOptions()
chromeOptions.add_experimental_option('useAutomationExtension', False)

PATH = r"C:\Users\...\chromedriver.exe"
driver = webdriver.Chrome(executable_path=PATH, options=chromeOptions, desired_capabilities=chromeOptions.to_capabilities())

url = 'https://www.primelocation.com/new-homes/details/55357965?search_identifier=d10d089a335cc707245fb6c924bafbd2'
driver.get(url)
element = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, "//*[contains(text(), 'Map & nearby')]")))
element.click()

The chrome window that pops up displays the website correctly, however when I run print(driver.page_source) to check how Python sees it, google map is missing, hence I am not able to extract the coordinates. Output from page_source is here: https://codeshare.io/5wgRvB

Any help greatly appreciated!

Swaroop Humane · Accepted Answer · 2020-07-28 11:06:28Z

Try this code -

import re

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
from selenium.webdriver import ActionChains
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import time

driver = webdriver.Chrome()
driver.get(
    "https://www.primelocation.com/new-homes/details/55357965?search_identifier=d10d089a335cc707245fb6c924bafbd2")

element = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.XPATH, "//span[text()='Map & nearby']")))
element.click()
time.sleep(5)
Map_Url = driver.find_element_by_xpath("//a[contains(@href,'https://maps.google.com/maps?ll=')]")
myurl = str(Map_Url.get_attribute('href'))


def find_between(s, first, last):
    try:
        start = s.index(first) + len(first)
        end = s.index(last, start)
        return s[start:end]
    except ValueError:
        return ""


Info = find_between(myurl, "https://maps.google.com/maps?ll=", "&z=15&t=m&hl=en-US&gl=US&mapclient=apiv3")
lat_long = Info.split(',')
print("Latitiue :-", lat_long[0])
print("Longitude :-", lat_long[1])

Note - If this is what you are looking for then please mark it as answer.

KunduK · Accepted Answer · 2020-07-28 11:47:41Z

To get the Lat & Long value you can use the regular expression as well. You need to import re and use the following regex expression.

import re
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By

driver = webdriver.Chrome()
driver.get("https://www.primelocation.com/new-homes/details/55357965?search_identifier=d10d089a335cc707245fb6c924bafbd2")
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, "//span[text()='Map & nearby']"))).click()
nearesturl=WebDriverWait(driver,10).until(EC.visibility_of_element_located((By.CSS_SELECTOR,"a[href^='https://maps.google.com/maps?ll']"))).get_attribute('href')
match=re.findall("\d+\.\d+",nearesturl)
print(match[0])
print("-" + match[1])

Collectives™ on Stack Overflow

Python - selenium - google map not visible in page_source

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related