How to find an element on a webpage using Selenium WebDriver?

Question

I trying to automate a task using Selenium WebDriver for FireFox on YouTube.

I have a playlist created on YouTube and I want to get the hyperlink of all the videos in that playlist.

The html looks like this:

<a href="/watch?v=StJLvbPIvTw&amp;list=PLt5xbw4ekDQssXxfaIfh_XbKe-iuOTZo_&amp;index=1" title="ADELE - Skyfall (Official video HD)" class="yt-uix-tile-link yt-uix-sessionlink" data-sessionlink="feature=plpp_video&amp;ei=RodgUazfOKWlhAHOioGoDA">
    <span class="title video-title" dir="ltr">ADELE - Skyfall (Official video HD)</span>
  </a>

I tried finding the element using the find_by_partial_link_text but failed.

My ultimate motive is to get the hyperlink for all the videos in the playlist so that i can pass it to next script to access them individually. Any help will be highly appreciated.

Note: I found an answer in SO similar to this SO but the answer provided here is iterating over a WebElement object which throws an exception in my case saying object is not iterable.

EDIT:

def init():

    d = webdriver.Firefox()
    d.implicitly_wait(15)
    print "in init"
    return d

def youtube(d, uname, pwd):

    link_list = []
    d.get("http://www.youtube.com")
    print "in you"
    signin = d.find_element_by_partial_link_text("Sign in")
    signin.click()

    email = d.find_element_by_id("Email")
    passwo = d.find_element_by_id("Passwd")
    submit = d.find_element_by_id("signIn")

    email.send_keys(uname)
    passwo.send_keys(pwd)
    submit.click()

    list = d.find_element_by_partial_link_text("Playlists")
    list.click()

    play = d.find_element_by_partial_link_text("Fav songs")
    play.click()
    print play

    link_list = d.find_element_by_xpath('//*[@id="playlist-pane-container"]/div[1]')
    print "done"
    print link_list # prints None here
    hr = link_list.get_attribute("css=a@href")
    print hr

@RossPatterson added the code I have tried. Any inputs will be immensely appreciated. — abhi
– abhi, Commented Apr 8, 2013 at 19:55
You have 3 uses of find_element_by_partial_link_text(). None of them come anywhere near to matching the HTML fragment you showed. — Ross Patterson
– Ross Patterson, Commented Apr 8, 2013 at 23:06
@RossPatterson I was first trying to do that with 'find_element_by_partial_text()' but it was not helping, so instead i tried using the xpath to find all the elements like that on the page as suggested on this SO question stackoverflow.com/questions/8121886/… but instead i getting an empty list. — abhi
– abhi, Commented Apr 9, 2013 at 6:34

e1che · Accepted Answer · 2013-04-08 08:54:14Z

Try to get link by : String hrefSpecs = driver.findelement(By.cssSelector("a.yt-uix-tile-link.yt-uix-sessionlink")).getAttribute("href")

normally you get

/watch?v=StJLvbPIvTw&list=PLt5xbw4ekDQssXxfaIfh_XbKe-iuOTZo_&index=1

so you substring that like :

String sublink = hrefSpecs.substring(0,20); // check the 20 i'm not sure

and you'll get

/watch?v=StJLvbPIvTw

since you get this you can write something like that

String youtube = "www.youtube.com";
String link = youtube + sublink;

and you finally get : www.youtube.com/watch?v=StJLvbPIvTw

But if you have a page with all links, you'll build something like :

List<String> listLink = driver.findElements(By.cssSelector("a.yt-uix-tile-link.yt-uix-sessionlink")).getAttribute("href");

and you 'll iterate that with a foreach with previous substring etc. i let you test and tell me if you have problems.

EDIT : WebElement is not iterable, List<WebElement> or [] in your case are iterable. I wrote it in Java, i hope you can "translate" it =)

scottydelta · Accepted Answer · 2013-04-07 03:19:16Z

-1

you can get the link by simple string manipulation.

load the string in a variable, split the string by the spaces, the second element will give you the href + link, remove the href part and you will get the link.

try this

html = '<a href="/watch?v=StJLvbPIvTw&list=PLt5xbw4ekDQssXxfaIfh_XbKe-iuOTZo_&index=1" title="ADELE - Skyfall (Official video HD)" class="yt-uix-tile-link yt-uix-sessionlink" data-sessionlink="feature=plpp_video&ei=RodgUazfOKWlhAHOioGoDA"> <span class="title video-title" dir="ltr">ADELE - Skyfall (Official video HD)</span> </a>'

parts = html.split(" ")

link = parts[1][6:-1]

answered Apr 7, 2013 at 3:19

scottydelta

1,8164 gold badges20 silver badges39 bronze badges

3 Comments

abhi Over a year ago

Thanks for the answer.But I have 100+ videos in the playlist and i think it will not be efficient to manually copy the html. I am trying to automate the task

scottydelta Over a year ago

I thought you had separate file for each video, anyways you can always use string manipulation to extract all the videos' html from the html source of the page.

Ross Patterson Over a year ago

Oh, the horror! This is Selenium code, the API has plenty of methods to make this sort of thing much easier. Given an element <a href="...">blah</a>, something as simple as for element in driver.find_elements_by_tagname("a"): print element.get_attribute("href"); would do the job.

Collectives™ on Stack Overflow

How to find an element on a webpage using Selenium WebDriver?

2 Answers 2

Comments

3 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

3 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related