Python request get href url from button

Question

I´m trying to create a small application using Python and requests library that can retrieve results from a Spanish website that contains several advertisements. The website application does not directly allows the user to enter coordinates to look for results but it has a "map interface" that can help.

If you look at this URL you can see that a map appears with some results. If I want to review them, I have to click on the button that says "Listado". Once there, it is easy to get the information.

The problem is that the HTML code behind the "map interface" page, does not contain the URL to the results list page. It is generated by a complex javascript. Is there any way to get this URL without using any physical simulator like Selenium nor replicating the javascript?

Thank you in advance.

QHarr · Accepted Answer · 2019-04-24 17:13:37Z

1

You should really include your attempt but here is a way using a url found in network tab

import requests  
headers = {
    'Referer' :'https://www.idealista.com/point/alquiler-viviendas/40.35324/-3.51313/14/con-metros-cuadrados-mas-de_62,metros-cuadrados-menos-de_116,pisos/mapa-google',
    'User-Agent' : 'Mozilla/5.0'
}
with requests.Session() as s:
    r = s.get('https://www.idealista.com/ajax/listingcontroller/livesearchmap.ajax?locationUri=&typology=1&operation=2&freeText=&liveSearch=true&zoom=14&northEast=40.372249460035604%2C+-3.485406687622117&southWest=40.3342251797178%2C+-3.540853312377976&uid=djaedoeimntj86uui175glt4cxh25wr3rvuqb4abrfib&adfilter_pricemin=default&adfilter_price=default&adfilter_area=62&adfilter_areamax=116&adfilter_amenity=default&adfilter_homes=1&adfilter_chalets=&adfilter_countryhouses=&adfilter_duplex=&adfilter_penthouse=&adfilter_rooms_0=&adfilter_rooms_1=&adfilter_rooms_2=&adfilter_rooms_3=&adfilter_rooms_4_more=&adfilter_baths_1=&adfilter_baths_2=&adfilter_baths_3=&adfilter_newconstruction=&adfilter_goodcondition=&adfilter_toberestored=&adfilter_housingpetsallowed=&adfilter_hasairconditioning=&adfilter_wardrobes=&adfilter_lift=&adfilter_flatlocation=&adfilter_parkingspace=&adfilter_garden=&adfilter_swimmingpool=&adfilter_hasterrace=&adfilter_boxroom=&adfilter_top_floor=&adfilter_intermediate_floor=&adfilter_ground_floor=&adfilter_agencyisabank=&adfilter_published=default&onlySavedAds=false', headers = headers).json()
    print(r['jsonResponse']['listingSearchUrl'])

answered Apr 24, 2019 at 17:13

QHarr

84.5k14 gold badges58 silver badges105 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

juancar Over a year ago

Thank you very much. That works great, I didn't realized this way to find the URL. Even tough is not my intention to perform a big scrap but only collect a few results, this helps a lot. Best regards.

Collectives™ on Stack Overflow

Python request get href url from button

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related