0

Using Selenium & Python I need to click on targets.simple.csv download a .csv file from this page: https://www.opensanctions.org/datasets/default/

Here is the code I have written:

import pandas as pd
import numpy as np
from datetime import date
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium import webdriver
import time
from selenium.webdriver.support import expected_conditions
from selenium.webdriver.support import expected_conditions
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait

chromedriver_path = r"./driver/chromedriver"
browser = webdriver.Chrome(executable_path=chromedriver_path)
url = "https://www.opensanctions.org/datasets/default/"
topics_xpath = '/html/body/div[1]/div/div/div[1]/section[2]/table/tbody/tr[6]/td[2]/a/code'
browser.get(url)
browser.maximize_window()
time.sleep(10)  #Wait a little for page to load.
WebDriverWait(browser, 10).until(EC.element_to_be_clickable((By.XPATH, "/html/body/div[1]/div[2]/div/div/div[2]/div/button[1]"))).click()
WebDriverWait(browser, 10).until(EC.element_to_be_clickable((By.XPATH, "/html/body/div[1]/div/div/div[1]/section[2]/table/tbody/tr[6]/td[2]/a/code"))).click()

But I get this error:

ElementClickInterceptedException: element click intercepted: Element is not clickable at point (360, 1282)
  (Session info: chrome=114.0.5735.198)

However, the error does not occur if:

  • I run the script
  • As soon as the page opens, I manually scroll (with the mouse) where the downloadable csv file is.

If I do so, then the code downloads the .csv file.

Is there a way in Python to scroll to the file I want to download without me having to do it manually?

2 Answers 2

1

you can refer this codes, the document is https://www.selenium.dev/documentation/webdriver/actions_api/wheel/#scroll-to-element

perform() can move down one by one, once it find the href, it will break the while loop. so you can use a little step one by one.

from_viewport() used to locate the first coordinate in the screen.

kkk_y = int(kkk.rect['y']) can get the element's y coordinate

below code can scroll to the element.
ActionChains(driver).scroll_from_origin(scroll_origin, 0, kkk_y).perform()

I don't install the chrome and driver, so I use the edge.

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium import webdriver
import time
from selenium.webdriver.edge.service import Service
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.actions.wheel_input import ScrollOrigin
from selenium.webdriver import ActionChains

driverpath = r"C:\Users\10696\Desktop\access\6\msedgedriver.exe"

service=Service(executable_path=driverpath)
driver = webdriver.Edge(service = service)
url = "https://www.opensanctions.org/datasets/default/"

topics_xpath = '/html/body/div[1]/div/div/div[1]/section[2]/table/tbody/tr[6]/td[2]/a/code'
driver.get(url)
kk = driver.find_element(By.XPATH, "/html/body/div[1]/div[2]/div/div/div[2]/div/button[1]")
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, "/html/body/div[1]/div[2]/div/div/div[2]/div/button[1]")))
kk.click()

kkk = driver.find_element(By.XPATH, "/html/body/div[1]/div/div/div[1]/section[2]/table/tbody/tr[6]/td[2]/a")
scroll_origin = ScrollOrigin.from_viewport(0, 0)
kkk_y = int(kkk.rect['y'])
ActionChains(driver).scroll_from_origin(scroll_origin, 0, kkk_y).perform()
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, "/html/body/div[1]/div/div/div[1]/section[2]/table/tbody/tr[6]/td[2]/a")))
time.sleep(2)
kkk.click()

another thing

you should get the website and use requests to download something, like below not using click, I can download this file normally targets.simple.csv.


from selenium.webdriver.common.by import By
from selenium import webdriver
import os
from selenium.webdriver.edge.service import Service
from selenium.webdriver.common.by import By
import requests

driverpath = r"C:\Users\10696\Desktop\access\6\msedgedriver.exe"

service=Service(executable_path=driverpath)
driver = webdriver.Edge(service = service)
url = "https://www.opensanctions.org/datasets/default/"

driver.get(url)
kkk = driver.find_element(By.XPATH, "/html/body/div[1]/div/div/div[1]/section[2]/table/tbody/tr[6]/td[2]/a")
hrefkkk = kkk.get_attribute("href")
if hrefkkk:
    con = requests.get(hrefkkk)
    if con.status_code==200:
        with open(os.path.join(r'C:\Users\10696\Desktop\access\6', kkk.text), 'wb') as obj:
            obj.write(con.content)
Sign up to request clarification or add additional context in comments.

Comments

1

To click on the clickable element you need to scroll down a few pixels and inducing WebDriverWait for the element_to_be_clickable() and you can use either of the following locator strategies:

  • Using PARTIAL_LINK_TEXT:

    browser.get("https://www.opensanctions.org/datasets/default/")
    WebDriverWait(browser, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "div[class^='Analytics_consentButtons'] button.btn.btn-secondary"))).click()
    browser.execute_script("scroll(0, 1000);") # Scroll Down
    WebDriverWait(browser, 20).until(EC.element_to_be_clickable((By.PARTIAL_LINK_TEXT, "targets.simple.csv"))).click()
    
  • Using CSS_SELECTOR:

    browser.get("https://www.opensanctions.org/datasets/default/")
    WebDriverWait(browser, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "div[class^='Analytics_consentButtons'] button.btn.btn-secondary"))).click()
    browser.execute_script("scroll(0, 1000);") # Scroll Down
    WebDriverWait(browser, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "a[href*='targets.simple.csv'] > code"))).click()
    
  • Using XPATH:

    browser.get("https://www.opensanctions.org/datasets/default/")
    WebDriverWait(browser, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "div[class^='Analytics_consentButtons'] button.btn.btn-secondary"))).click()
    browser.execute_script("scroll(0, 1000);") # Scroll Down
    WebDriverWait(browser, 20).until(EC.element_to_be_clickable((By.XPATH, "//a/code[text()='targets.simple.csv']"))).click()
    
  • Note: You have to add the following imports :

    from selenium.webdriver.support.ui import WebDriverWait
    from selenium.webdriver.common.by import By
    from selenium.webdriver.support import expected_conditions as EC
    
  • Browser Snapshot:

opensanctions

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.