Web Scraping using Ruby - If statment

Question

I have built a web scraper. I need it to scrape the prices and bedrooms of a given neighborhood. Sometimes the span.first_detail_cell will return Furnished and the rest of the time it will return the price. I need to write something that can overlook the span.first_detail_cell if it is furnished and look in the next cell for the price. I think I need to write an if statement, but not sure of the parameters. Any help would be great!

require 'open-uri'
require 'nokogiri'
require 'csv'

url = "https://streeteasy.com/for-rent/bushwick"
page = Nokogiri::HTML(open(url))

page_numbers = []
page.css("nav.pagination span.page a").each do |line|
  page_numbers << line.text
end

max_page = page_numbers.max

beds = []
price = []

max_page.to_i.times do |i|

  url = "https://streeteasy.com/for-rent/bushwick?page=#{i+1}"
  page = Nokogiri::HTML(open(url))

  page.css('span.first_detail_cell').each do |line|
    beds << line.text
  end

  page.css('span.price').each do |line|
    price << line.text
  end

end

CSV.open("bushwick_rentals.csv", "w") do |file|
  file << ["Beds", "Price"]

  beds.length.times do |i|
    file << [beds[i], price[i]]
  end
end

You can always next if line.text.to_f != 0.0 Ruby .to_f will return 0.0 if the price is not convertible to a number — Cyril Duchon-Doris
– Cyril Duchon-Doris, Commented Feb 4, 2017 at 19:49

max pleaner · Accepted Answer · 2017-02-04 22:56:44Z

1

  page.css('span.first_detail_cell').each do |line|
    if line.text.include?("Furnished")
      # do something hre
    else
      beds << line.text
    end
  end

answered Feb 4, 2017 at 22:56

max pleaner

26.9k10 gold badges73 silver badges132 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Mike Wiesenhart Over a year ago

Thanks for the help!

Collectives™ on Stack Overflow

Web Scraping using Ruby - If statment

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related