Ruby web scraping with selenium

Question

I am trying to scrape a website and strip two different things and print them together. The title and description of PlayStation Trophies.

require 'selenium-webdriver'

  driver = Selenium::WebDriver.for(:chrome, options: options)

  driver.get('https://www.playstationtrophies.org/game/far-cry-4/trophies/')

  puts driver.title

  trophies = driver.find_elements(:class, 'link_ach')
  description = driver.find_elements(:class, 'ac3') 

  trophies.each do |trophy|
    description.each do |desc|
      puts trophy,desc.text.strip
     end
  driver.quit
end

I can strip them both individually but when trying to put them together it goes horribly wrong.

Can you describe what you mean by horribly wrong and what the expected output is? — Mike K.
– Mike K., Commented Mar 2, 2019 at 15:04

Mike K. · Accepted Answer · 2019-03-02 22:28:58Z

1

I'm not entirely sure what you mean by 'horribly wrong' but my best guess based on running what you have is that for trophy you're actually printing the inspection of the variable trophy to the screen instead of the string value. So you're getting lots of:

#<Selenium::WebDriver::Element:0x00007ff4b60e5eb8>

I think you'll be happier with the output if you print the text values of both Element's:

puts "#{trophy.text.strip}\n #{desc.text.strip}"

If this is indeed the issue you're seeing you may want to review to_s vs inspect in ruby.

edited Mar 2, 2019 at 22:28

answered Mar 2, 2019 at 15:11

Mike K.

3,81731 silver badges42 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Rusty Over a year ago

Thanks. That makes sense. Although I am seeing a disconnect between the two. How do I print the associated trophy with the description underneath?

tadman Over a year ago

puts "#{...}\n #{...}" where \n is just a newline.

Mike K. Over a year ago

added the newline for completeness

Maruf Hasan Bulbul · Accepted Answer · 2019-03-03 10:12:10Z

0

Iteration over trophies and description is very confusing here. You have two separate collections/arrays, trophies and description. Then you have used a nested loop and quit driver inside the loop. So it will display all the descriptions under 1st trophy.

If you want to display each description under associated trophy then you can do this:

puts driver.title

trophies = driver.find_elements(:class, 'link_ach')
descriptions = driver.find_elements(:class, 'ac3')
data_sets = trophies.zip(descriptions)

datasets.each do |trophy, description|
    puts trophy.text.strip, description.text.strip
end

driver.quit

Then you can customize your display text in puts as your needs.

answered Mar 3, 2019 at 10:12

Maruf Hasan Bulbul

4624 silver badges14 bronze badges

Collectives™ on Stack Overflow

Ruby web scraping with selenium

2 Answers 2

3 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

3 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related