Is there a way to find out the fully qualified url in Nodejs?
Basically I want to crawl a website using my custom node.js based crawler. I fetch all the anchor tags and make http requests on them. The problem is that I am not able to figure out the exact url that I need to make the request to. Please help
Example URLS
http://aaa.com/bbb
//aaa.com/bbb or //aaa/bbb
/aaa.com/bbb or /aaa/bbb
aaa.com/bbb or aaa/bbb
The url.resolve feature worked for me (suggested by @Karman and @micnic). The first argument was the base url, which represented the url of the page that I was scraping for anchor tags. The second argument is the url which is saved in the href tag of the anchors.
Look at the url module http://nodejs.org/api/url.html you can use url resolve property or have detail look at the above mentioned documentation of url module
also e.g.
var obj= require('url').parse('http://google.com?q=tobi');
will result in
obj=
{ protocol: 'http:',
slashes: true,
host: 'google.com',
hostname: 'google.com',
href: 'http://google.com/?q=tobi',
search: '?q=tobi',
query: 'q=tobi',
pathname: '/' }
url.join()?url.join()+url.resolve()can do the magic, just provide some examples where you can not use them