css locator returns empty list

Question

i am making a web scraping course on datacamp :D

in the session it self we are scraping datacamp websites to get courses names and urls however, i am trying to reach the same result on pycharm on my pc ,

unfortunately nothing happens

i have tried many css locators and the same result always empty list

can you just guide me what is the wrong ?

from scrapy import Selector import requests

url = 'https://www.datacamp.com/courses-all' html = requests.get(url).content sel = Selector ( text = html)

courses_names = sel.css('section.css-6b6e9e-Search>article.css-1t5b87c-HitCard>h2.css-172ju3k-Box ::text').extract()

print(courses_names)

If you are receiving an error or exception you should post it along with the full traceback. I am not certain that this is the cause of your problem, but I am pretty sure you are trying to feed a bytes object into the text parameter of the scrapy selector. — Alexander
– Alexander, Commented Oct 9, 2023 at 13:30
i got no error just empty cell ... as like the selector did not find anything even if i used this code boxes = sel.css('div ::text').extract() — mohamed sultan
– mohamed sultan, Commented Oct 9, 2023 at 16:46

Ahmed Ellaban · Accepted Answer · 2023-10-09 19:19:46Z

0

there isn't enough info so I will just explain what you are doing from that CSS selector I can see, you are using a general selector

sel.css('section.css-6b6e9e-Search>article.css-1t5b87c-HitCard>h2.css-172ju3k-Box ::text').extract()

you shouldn't do that maybe because the website probably changes the class name every time

just, check if that selector is working on that website from the dev tools and try to print the result of

sel.css('section > article').get()

it should print HTML tags if that selector is correct

hope you a good luck with web scraping

answered Oct 9, 2023 at 19:19

Ahmed Ellaban

1567 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

mohamed sultan Over a year ago

unfortunately i used this code courses_names = sel.css('section > article').get() the result is NONE ... can i ask you weather this has any relation that the site is using java script ? i found some information says that but i even do not know how to handle it... some advised me to use selenium .. so what is your opinion ؟ and thank you so much

Ahmed Ellaban Over a year ago

Assalamu alaikum first I thought that too but , you said you are following tutorial so I thought your teacher could get this data so you should get it too but if the website has been updated yes that may be the problem

Collectives™ on Stack Overflow

css locator returns empty list

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related