python web scraping Max retries exceeded with url

Question

I wish to download a buildings database from BREEAM(Building Research Establishment Environmental Assessment Methodology). It has 65 pages so I would like to write a code to download more efficiently.

I had the following code

import library

from bs4 import BeautifulSoup import requests

Request to website and download HTML contents

url='https://tools.breeam.com/projects/explore/buildings.jsp?sectionid=0&projectType=Offices&rating=&countryID=56&client=&description=&certBody=&certNo=&developer=&location=London&buildingName=&assessor=&subschemeid=0&Submit=Search' req=requests.get(url) content=req.text

But I kept receiving error message: ConnectionError: HTTPSConnectionPool(host='tools.breeam.com', port=443): Max retries exceeded with url: /projects/explore/buildings.jsp?sectionid=0&projectType=Offices&rating=&countryID=56&client=&description=&certBody=&certNo=&developer=&location=London&buildingName=&assessor=&subschemeid=0&Submit=Search (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x0000020523F02048>: Failed to establish a new connection: [Errno 11001] getaddrinfo failed',))

Any ideas?

Thanks.

DevConcepts · Accepted Answer · 2022-10-27 20:29:34Z

0

For me this already works:

import requests

url = "https://tools.breeam.com/projects/explore/buildings.jsp?sectionid=0&projectType=Offices&rating=&countryID=56&client=&description=&certBody=&certNo=&developer=&location=London&buildingName=&assessor=&subschemeid=0&Submit=Search"

response = requests.get(url)

print(response)

No headers and cookies needed. Have you already looked at this or this? And please format your question the right way next time :)

answered Oct 27, 2022 at 20:29

DevConcepts

1748 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

python web scraping Max retries exceeded with url

import library

Request to website and download HTML contents

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

import library

Request to website and download HTML contents

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related