Python socket recv() doesn't get every message if send too fast

Question

I send mouse coordinates from python server to python client via socket. Mouse coordinates are send every time when mouse movement event is catch on the server which means quite often (dozen or so per second).

Problem is when I use python server and python client on different hosts. Then only part of messages are delivered to the client. e.g. 3 first messages are delivered, 4 messages aren't delivered, 4 messages are delivered etc...

Everything is fine when server and client are on the same host (localhost).

Everything is fine when server and client are on different hosts but instead of python client I use standard windows Telnet client to read messages from the server.

I noticed that when I use time.sleep(0.4) break between each message that is send then all messages are delivered. Problem is I need that information in real time not with such delay. Is it possible to achieve that in Python using sockets?

Below python client code that I use:

import pickle
import socket
import sys


host = '192.168.1.222'
port = 8888
try:
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
except socket.error, msg:
    print "Faile. Error:" + str(msg[0]), "Error message : " + msg[1]
    sys.exit()

mySocket = socket.socket()
mySocket.connect((host,port))

while 1:
    data = mySocket.recv(1024)
    if not data: break
    load_data = pickle.loads(data)

    print 'parametr x: ' + str(load_data[0])
    print 'parametr y : ' + str(load_data[1])

mySocket.close()

Steffen Ullrich · Accepted Answer · 2016-10-08 11:33:19Z

5

You are using TCP (SOCK_STREAM) which is a reliable protocol which (contrary to UDP) does not loose any messages, even if the recipient is not reading the data fast enough. Instead TCP will reduce the sending speed.
This means that the problem must be somewhere in your application code.

One possibility is that the problem is in your sender, i.e. that you use socket.send and do not check if all the bytes you've intended to send are really got send. But this check needs to be done since socket.send might only send part of the data if the socket buffer of the OS is full which can happen if the client does not read the data fast enough.

Another possibility is that your socket.recv call receives more data than your pickle.loads needs and that the rest of the data gets discarded (not sure if pickle.loads will throw an exception if too much data are provided). Note that TCP is not a message but a stream protocol so it might be that you have more that socket.recv will return a buffer which contains more than one pickled object but you only read the first. The chance that this will happen on a network is higher than on localhost because by default the TCP layer will try to concatenate multiple send buffers into a single TCP packet for better use of the connection (i.e. less overhead). And the chance is high that these will then be received within the same recv call. By putting a sleep(0.4) on the sender side you've effectively switch off this optimization of TCP, see NAGLE algorithm for details.

Thus the correct way to implement what you want would be:

Make sure that all data are delivered at the server, i.e. check the return of socket.send.
Make sure that you unpack all messages you receive. To do this you probable need to add some message layer on top of the TCP stream to find out where the message boundary is.

edited Oct 8, 2016 at 11:33

answered Oct 8, 2016 at 11:26

Steffen Ullrich

125k11 gold badges157 silver badges194 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

michal2616 Over a year ago

You are right! I just made quick check and It turned out that I read only part of the messages that are delivered. As you mentioned some messages get concatenate and I don't read them properly. 'pickle.loads' didn't throw any exception so that tricked me. Now I will need to figure out how to read them all.

mhawke Over a year ago

OP could try socket.sendall() to ensure that all data is sent.

Collectives™ on Stack Overflow

Python socket recv() doesn't get every message if send too fast

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related