NodeJS - TCP - Sending an HTTP request

Question

I'm trying to send an HTTP request via TCP sockets.

But I'm not getting any response from www.google.com at all. No idea what I'm doing wrong.

Here is the code:

var client, net, raw_request;

net = require('net');

raw_request = "GET http://www.google.com/ HTTP/1.1\nUser-Agent: Mozilla 5.0\nhost: www.google.com\nCookie: \ncontent-length: 0\nConnection: keep-alive";

client = new net.Socket();

client.connect(80, "www.google.com", function() {
  console.log("Sending request");
  return client.write(raw_request);
});

client.on("data", function(data) {
  console.log("Data");
  return console.log(data);
});

Hope someone can help me.

Just to clarify... the requst was missing two ending newlines and all newlines had to be in the format of /r/n.

Thanks everyone! :)

Why do you send \n and not \r\n as the specs require? You also need to finish the request by sending two \r\n's. Do you want to manually build a request or can you use http.request() instead? — CodeCaster
– CodeCaster, Commented Jan 19, 2013 at 18:54
Just wanted to write some HTTP requests using TCP sockets to learn something new. — RadiantHex
– RadiantHex, Commented Jan 19, 2013 at 18:56
@bryanmac thanks for the link. But really want to learn how to do this at a lower level. — RadiantHex
– RadiantHex, Commented Jan 19, 2013 at 18:57
@RadiantHex then you should start looking at RFC 2616, especially section 5, Request, how to construct valid HTTP messages. — CodeCaster
– CodeCaster, Commented Jan 19, 2013 at 18:57

Florin Petriuc · Accepted Answer · 2013-01-19 19:15:49Z

If you have google chrome installed you can see the exact get request that is sent to google. This is how mine looks like:

GET https://www.google.com/ HTTP/1.1
:host: www.google.com
accept-charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
accept-encoding: gzip,deflate,sdch
accept-language: en-US,en;q=0.8
user-agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1312.52 Safari/537.17
:path: /
accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
:version: HTTP/1.1
cache-control: max-age=0
cookie: <lots of chars here>
:scheme: https
x-chrome-variations: CMq1yQEIjLbJAQiYtskBCKW2yQEIp7bJAQiptskBCLa2yQEI14PKAQ==
:method: GET

At a first view I can see that chrome is sending the request to https://www.google.com and you are sending to http://www.google.com

Another thing is that you are using "\n" and you need to use "\r\n", and the request has to end with "\r\n\r\n".

If you still can't get any response try using http://77.214.52.152/ instead of http://google.com.

Michael Lorton · Accepted Answer · 2013-01-19 19:04:40Z

1

 GET /

The way you wrote it, you are trying get something like http://www.google.com/http://www.google.com/

And you need two new-lines at the end, so the web-server knows you're done. That's why you're getting nothing back -- it's still waiting for you.

Always try these things with telnet first:

$ telnet www.google.com 80
GET http://www.google.com/ HTTP

HTTP/1.0 400 Bad Request
Content-Type: text/html; charset=UTF-8
Content-Length: 925
Date: Sat, 19 Jan 2013 19:02:06 GMT
Server: GFE/2.0

<!DOCTYPE html>
<html lang=en>
  <meta charset=utf-8>
  <meta name=viewport content="initial-scale=1, minimum-scale=1, width=device-width">
  <title>Error 400 (Bad Request)!!1</title>
  <style>
    *{margin:0;padding:0}html,code{font:15px/22px arial,sans-serif}html{background:#fff;color:#222;padding:15px}body{margin:7% auto 0;max-width:390px;min-height:180px;padding:30px 0 15px}* > body{background:url(//www.google.com/images/errors/robot.png) 100% 5px no-repeat;padding-right:205px}p{margin:11px 0 22px;overflow:hidden}ins{color:#777;text-decoration:none}a img{border:0}@media screen and (max-width:772px){body{background:none;margin-top:0;max-width:none;padding-right:0}}
  </style>
  <a href=//www.google.com/><img src=//www.google.com/images/errors/logo_sm.gif alt=Google></a>
  <p><b>400.</b> <ins>That’s an error.</ins>
  <p>Your client has issued a malformed or illegal request.  <ins>That’s all we know.</ins>
Connection closed by foreign host.

answered Jan 19, 2013 at 19:04

Michael Lorton

44.7k26 gold badges111 silver badges159 bronze badges

6 Comments

RadiantHex Over a year ago

Thanks a lot for the information. But I'm not getting any reply at all from Google. At least via client.on('data', ...) I'm not.

Florin Petriuc Over a year ago

"The way you wrote it, you are trying get something like google.com/http://www.google.com" - that is wrong. The http get request supports absolute URI. Moreover, using absolute URI is the only way to get a http get request to work with proxy.

Michael Lorton Over a year ago

@RadiantHex -- did you give it the two newlines?

Michael Lorton Over a year ago

@FlorinPetriuc -- I don't know what to tell you. I used the absolute URI with Google and, as you see, I got a 400 error, Malformed Request. Call Sergei Brin and take it up with him. In the meanwhile, I would use the HOST: header to get proxies to work.

RadiantHex Over a year ago

@Malvolio it was the 2 newlines! Thanks!

|

Collectives™ on Stack Overflow

NodeJS - TCP - Sending an HTTP request

2 Answers 2

Comments

6 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

6 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related