0

I searched a lot how to solve this problem but I didn't find anything I could make it work. Basically I have this webpage:

http://databank.worldbank.org/data/embed-int/Table-1-SDDS-new/id/4f2f0c86

and what I'm trying to do is change the country with Python in order to extract the data from the HTML (I already know how to extract the data). Point is, I don't know how to change the country. Could you help me with that?

I saw many solutions that were similar but, probably due to my lack of experience with HTML, I didn't understand them well.

Thank you in advance.

Improve this question
6
  • 1
    Hi Lorenzo, welcome to SO. Please add more details - what code do you have so far? Commented Dec 16, 2018 at 16:04
  • Hello @Josh, Thanks for the answer. Right now I don't have much code as I got stuck since the beginning. Checking online, I feel I should use POST from request but then I don't know the way to pass the parameter taht refers to the country. This is what I tried: r = requests.post("databank.worldbank.org/data/embed-int/Table-1-SDDS-new/id/…) Commented Dec 16, 2018 at 16:17
  • Please share what you have tried. Commented Dec 16, 2018 at 16:18
  • you want to target the options inside the select element i.e. after id ddp-SDDS_Country_Vw There are options with 3 letter country abbreviations which you can use select syntax on e.g. value="[SDDS_Country_Vw].[List].&[ARG]" Commented Dec 16, 2018 at 16:21
  • You definitely don't want to use POST, since you can't change the contents on the World Bank's website! Also, I'm no expert but I suspect that the change is done through Javascript. Commented Dec 16, 2018 at 16:23

2 Answers 2

Reset to default
1

You are better off using a method like selenium. In that case you can click the dropdown and use the inputbox to send your country of interest and then enter.

from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys

d = webdriver.Chrome()
d.get("http://databank.worldbank.org/data/embed-int/Table-1-SDDS-new/id/4f2f0c86")
dropdown = WebDriverWait(d,10).until(EC.visibility_of_element_located((By.CSS_SELECTOR, ".chosen-single")))
dropdown.click()
input = d.find_element_by_css_selector('.chosen-search input')
input.send_keys('Brazil')
input.send_keys(Keys.RETURN)
Improve this answer
Sign up to request clarification or add additional context in comments.

5 Comments

Thank you very much! I get this error: - WebDriverException: Message: Service C:\Program Files (x86)\Google\Chrome\Application\chrome.exe unexpectedly exited. Status code was: 0 - I saw checking online that chromedriver.exe should be in the PATH. I put it there but is still giving me the same error.
you can try passing the path as an argument
d= webdriver.Chrome(executable_path="C:/Utility/chromedriver.exe")
Are you using latest chrome and chromedriver?
QHarr ok It worked. I had to put the path of Chrome Drive even if it was already on the PATH. Thank you very much for the help.
0

I don't know how to select option from drop-down menu too, but here is worldbank api

Improve this answer

2 Comments

Thanks for the answer, unfortunately that type of data is not available in the API. If you try to download data as explained in the WB website, you can get the dates (example: Q2 2018) but the values are all "None".
Oh.. I’m sorry for not helping for you.. ㅠㅠㅠ

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.