A bit of a strange question. Is there a way to ask a webserver to return only the headers and not the HTML itself ?
I want to ask a server for a URL and see if its valid (not 404/500/etc) and follow the redirections (if present) but not get the actual HTML content.
Thanks
use HEAD instead of GET or POST
http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html Section 9.4
As suggested, check the Net::HTTP library..
require 'net/http'
Net::HTTP.new('www.twitter.com').request_head('/').class
This is exactly what HEAD HTTP method does.
For Ruby, there is a beautiful gem, much simpler than the low-level net/http that allows you to perform HEAD requests.
gem install rest-open-uri
then
irb> require 'rubygems'
=> true
irb> require 'rest-open-uri'
=> true
irb> sio = open("http://stackoverflow.com", :method => :head)
=> #
irb> sio.meta
=> {"expires"=>"Tue, 30 Nov 2010 18:08:47 GMT", "last-modified"=>"Tue, 30 Nov 2010 18:07:47 GMT", "content-type"=>"text/html; charset=utf-8", "date"=>"Tue, 30 Nov 2010 18:08:27 GMT", "content-length"=>"193779", "cache-control"=>"public, max-age=18", "vary"=>"*"}
irb> sio.status
=> ["200", "OK"]
It follows redirections. You have to rescue for SocketError when host doesn't exists or OpenURI::HTTPError if file doesn't exists.
If you want something more powerfull have a look at Mechanize or HTTParty.
Use Ruby's net/http and the HEAD method that Mak mentioned. Check ri Net::HTTP#head from the command line for info.
actually i had to fold pantulis' answer into my own. it seems like there are two kinds of urls neither fns worked alone so i did
module URI
def self.online?(uri)
URI.exists?(uri)
end
def self.exists?(uri)
URI.exists_ver1?(uri)
end
def self.exists_ver1?(url)
@url = url
["http://", "https://"].each do |prefix|
url = url.gsub(prefix, "")
end
begin
code = Net::HTTP.new(url).request_head('/').code
[2,3].include?(code.to_i/100)
rescue
URI.exists_ver2?(@url)
end
end
def self.exists_ver2?(url)
url = "http://#{url}" if URI.parse(url).scheme.nil?
return false unless URI.is_a?(url)
uri = URI(url)
begin
request = Net::HTTP.new uri.host
response= request.request_head uri.path
#http status code 200s and 300s are ok, everything else is an error
[2,3].include? response.code.to_i/100
rescue
false
end
end
end
