Python Bidirectional TCP Socket Hanging on socket.recv

Question

Referencing this example (and the docs): https://pymotw.com/2/socket/tcp.html I am trying to achieve bidirectional communication with blocking sockets between a client and a server using TCP.

I can get one-way communication to work from client->server or server->client, but the socket remains blocked or "hangs" when trying to receive messages on both the server and client. I am using a simple algorithm(recvall), which uses recv, to consolidate the packets into the full message.

I understand the sockets remain blocked by design until all the data is sent or read(right?), but isn't that what sendall and recvall take care of? How come disabling recv on either the client or server "unblocks" it and causes it to work? And ultimately what am I doing wrong that is causing the socket to stay blocked?

Here is my code, the only fundamental difference really being the messages that are sent:

recvall(socket)(shared between client and server):

def recvall(socket):
    data = ''
    while True:
        packet = socket.recv(16)
        if not packet: break
        data += packet

    return data

server.py (run first):

import socket

host = 'localhost'
port = 8080

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((host, port))
s.listen(5)

while True:
    (client, address) = s.accept()
    print 'client connected'

    try:
        print recvall(client)
        client.sendall('hello client')
    finally:
        client.close()

client.py: import socket

s = socket.create_connection((args.ip, args.port))

try:
    s.sendall('hello server')
    print recvall(s)
finally:
    s.close()

your recvall() function will accumulate all data until the TCP connection is closed, which is likely why your program appears to hang - you don't close the connection, so the other end doesn't know it should stop reading. (Remember also that in TCP, there are not packets - you just get a stream of bytes). Therefore you probably need some way of sending the length of your data so your recvall() function can know how much to read, or send something special to end your data (e.g. a newline) and have recvall() look for that special data. — nos
– nos, Commented May 22, 2017 at 20:53
If I have to close the socket then re-open it in order to then read from it, would that data not get lost? Isn't it attached to the socket? Or on the contrary, I would have to know the size of the message (was trying to avoid this)? I thought that was the point of recvall, to keep reading until no more bytes come through. Then it knows the data is finished. I'm confused as to why the example supposedly works but this code does not. They are almost identical. — Jordan
– Jordan, Commented May 22, 2017 at 21:01
recvall() doesn't know that you have sent "hello client" or "hello server" and that was all you wanted to send, it will keep reading as you say until no more bytes comes through, and the only way it knows for sure no more bytes will come is when the TCP connection is closed. If you want the recvall() function to stop reading without closing the connection, you must teach it when it should stop reading, such as first send a length and then have it stop after it has read that many bytes, or have it stop e.g. when it reads a newline character, or have it stop when it read a fixed amount of bytes — nos
– nos, Commented May 22, 2017 at 21:06

Jordan · Accepted Answer · 2017-05-23 16:05:28Z

From my understanding (epiphany here), the main problem is that recv inside recvall is only concerned with retrieving the stream (in the same way send is only concerned with sending the stream), it has no concept of a "message" and therefore cannot know when to finish reading. It read all the bytes and did not return any additional bytes, but that is NOT a signal that the message is finished sending, there could be more bytes waiting to be sent and it would not be safe to assume otherwise.

This requires us to have an explicit indicator for when to stop reading. recv and send are only concerned with managing the stream and therefore have no concept of a message (our "unit"). This article has some great solutions to this problem. Since I am sending fixed-length messages, I opted to check that the length is as expected before finishing recv. Here is the updated version of recvall, note MSG_LENGTH must be defined and enforced in order for recvall to not block the socket.

def recvall(socket):
    data = ''
    while len(data) < MSG_LENGTH:
        packet = socket.recv(BUFFER_SIZE)
        if not packet: break
        data += packet
    return data

Bidirectional communication now works, the only catch being the client and server must know the length of the message they will receive, again this is not an issue in my case. This is all new to me so someone please correct me on terminology and concepts.

Collectives™ on Stack Overflow

Python Bidirectional TCP Socket Hanging on socket.recv

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related