1

I have a problem when receiving large packets in Node.js through a TCP connection. It seems that there is a cap in the buffer set to about 55kB. When I receive large amounts of data (about 70-80k) it splits this data up.

Now, I've worked with UDP sockets, I know how to expect on('data', function(){}) events and how to expect an on('end', function(){}) when the message is finally received, the problem with the TCP is that I never receive an on('end', function(){}) event, the server (in C#) never sends a FIN packet. This is due to me requiring to keep the socket alive and ongoing (we don't want to shut it down).

Is there some sort of way to send a FIN packet, from the C# to the Node.js one, without having to shut down/close the socket?

Or, alternativelly, is some sort of way to trigger the on('end', function(){}) event inside node itself?

Regards.

EDIT: To add up to the question, say for instance I have 2 clients attempting to acquire information through the TCP socket. One connects first, and gets a huge request (no problem on the size here).

But whilst this request is being handled, another one does the same request, since the TCP connection is through node, they're using the same socket. Is there any way to differentiate the requests? or will I have to set up a socket for each request being done to the TCP server?

Improve this question
5
  • Do you know how large the message will be, i.e. does the message contain the size of itself or its payload? Is there any kind of end-of-message marker? Commented Jan 3, 2012 at 12:13
  • well to solve it for now we set up an end of message marker, but we're hoping for a better solution than that for this. Commented Jan 3, 2012 at 12:33
  • 3
    The problem is that TCP is a streaming protocol, data flows like a stream and there is no built-in boundaries between messages. It is reliable and easier to use, but generally need another protocol layered above it if not used as a pure stream of data. Commented Jan 3, 2012 at 12:39
  • I see, well to add up to the question, say for instance I have 2 clients attempting to acquire information through the TCP socket. One connects first, and gets a huge request (no problem on the size here). But whilst this request is being handled, another one does the same request, since the TCP connection is through node, they're using the same socket. Is there any way to differentiate the requests? or will I have to set up a socket for each request being done? I'll add this up as an edit to the question. Commented Jan 3, 2012 at 12:59
  • The server socket for the two clients will be the same, but the socket-pair that defines the connection for each will be different. The socket object returned by accept() on the server will be unique for each connected client & so it is easy to differentiate the requests. Commented Jan 3, 2012 at 15:47

3 Answers 3

Reset to default
6

UDP is packet-oriented. You can send as much data as will fit in a packet, up to an absolute maximum of 65,507 bytes. It's fire and forget, with no intrinsic mechanism for handling packet loss or reordering.

TCP is stream-oriented. Your data will be split up into packets based on a few factors including volume of data and transmission rate. This is mostly beyond your control and managed by the TCP/IP stack. What you get in return is an ordered sequence of packets and a degree of fault-tolerance (correct packet ordering is guaranteed and lost/duplicate packets are detected and dealt with by the TCP/IP protocol stack transparently to application code).

If you're working at the socket level in node you will almost certainly get multiple callbacks, each containing a chunk of data. It's up to you to stitch it back together. This is a good thing - it means you can define your own mechanism for identifying message boundaries, data compression etc etc.

Improve this answer
Sign up to request clarification or add additional context in comments.

1 Comment

PS: accepting this since it explains well the problem, also, to those whom might be wondering, there's also a problem in the OS's handling of socket packet data (this same issue happened with Flash), it was fixed when updating Windows 7 to SP1 tho.
1

I had a similar issue, it looks like you'll need to employ two techniques:

  1. A way to parse apart the message and chuck it together---I used the following answer [question]: nodejs - parsing chunked twitter json
  2. You can either start a new tcp connection for the second client or create an array holding a clientID for each request made. When a message comes back from the server you would grab the first clientID in the array and pass along the message to them.
Improve this answer

Comments

-1

I would say to try to buffer and append

Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.