None type object has no attribute error in webcrawler Python

Question

I am new to Python so any help would be appreciated. I have a web crawler using beautifulsoup. It works but for the below it returns the error 'None type object has no attribute'. I know it means it has come across a page where there is no entry. How do I stop this error and make it return all the other pages that have entries. Some of the pages in the web crawler have the entry and some are blank.

bbb = re.compile('First listed')
    next_s = soup.find(text=bbb).parent.parent.get_text(strip=True)

Thanks

Linch · Accepted Answer · 2017-06-17 17:57:16Z

1

bbb = re.compile('First listed')
next_s = soup.find(text=bbb)
if next_s is not None:
    # node exists
else:
    # node does not exists

answered Jun 17, 2017 at 17:57

Linch

5414 silver badges11 bronze badges

Sign up to request clarification or add additional context in comments.

7 Comments

hello11 Over a year ago

thanks what do i put under the 'if next_s is not None:'?

Linch Over a year ago

depends on the logic of your script. if you have a list of links, and iterate over it in for loop then use continue, for next iteration.

hello11 Over a year ago

I'm pretty new to coding so i'm not too sure what that means. It doesnt have many links in the crawler, most of my code is below: where would I put the for and continue statements? Def get_single_item_data(item_url): source_code = requests.get(item_url) plain_text = source_code.text soup = BeautifulSoup(plain_text) bbb = re.compile('First listed') next_s = soup.find(text=bbb) writer.writerow([address, price, Bedroom, Description, Sold, next_s, next_ss])

Linch Over a year ago

can you post the example of HTML for parsing?

hello11 Over a year ago

The below is the beginning of the code: I posted the rest of the code above. Thanks for your help. url = 'zoopla.co.uk/for-sale/property/manchester/…' source_code = requests.get(url) plain_text = source_code.text soup = BeautifulSoup(plain_text) for link in soup.findAll('a', {'class': 'listing-results-price text-price'}): href = "zoopla.co.uk" + link.get('href') get_single_item_data(href)

|

Collectives™ on Stack Overflow

None type object has no attribute error in webcrawler Python

1 Answer 1

7 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

7 Comments

Your Answer

Sign up or log in

Post as a guest

Related