1

I am trying to get the value of a data-attribute with scrapy:

response.css('.product-header-top div::attr("data-background-image")').get()

But I do not get the value of data-background-image and Python throws an error:

raise SelectorSyntaxError(cssselect.parser.SelectorSyntaxError: Got pseudo-element ::FunctionalPseudoElement[::attr(['data-background-image'])] not at the end of a selector

Here is the relevant HTML Code of the webpage:

<div data-background-image="/images/image.jpg" style="background-image: url(&quot;/images/image.jpg&quot;);"></div>

Thanks

UPDATE F.Hoque is right and it works fine. The website is dynamic and renders the data-background-image with JS. So the ::attr("data-...") is working. Thanks for your help @F.Hoque!

Improve this question

1 Answer 1

Reset to default
2

Your CSS selection is working fine. There is a typo ); just remove it.

response.css('.product-header-top div::attr("data-background-image")').get()

Proven by Scrapy shell:

In [26]:  sel.css('div::attr("data-background-image")').get()
Out[26]: '/images/image.jpg'
Improve this answer
Sign up to request clarification or add additional context in comments.

8 Comments

Sadly not: raise SelectorSyntaxError( cssselect.parser.SelectorSyntaxError: Expected ident or '*', got <STRING 'data-background-image' at 24>
@Waldgeist, I've updated. Now it's working fine. Thanks
Now it is not throwing an error anymore but returns None :-/ ?
Oh, thanks. I corrected the typo .. I am adding the domain in front and had to put a str() around the response.css(). Interesting that you get the image url. I still get a none when trying to get the url: print(response.css('.product-header-top div::attr("data-background-image")').get())
Because the website is most likely dynamic and scrapy can't render js
|

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.