Python - Read remaining data from socket after TCP RST

Question

I'm implementing a file transfer protocol with the following use case:

The server sends the file chunk by chunk inside several frames.
The client might cancel the transfer: for this, it sends a message and disconnects at TCP level.

What happened in that case on server side (Python running on Windows) is that I catch a ConnectionResetException (this is normal, the client has disconnected the socket) while sending the data to the client. I would want to read the latest data sent by the client (the message used to abort the call), but calling mysocket.recv() still raises a ConnectionResetException.

With a wireshark capture, I can clearly see that the message was properly sent by the client prior to TCP disonnection.

Any idea floks? Thanks!

VR

There is no remaining data after an RST. It is all discarded. — user207421
– user207421, Commented May 12, 2020 at 12:31

JimD. · Accepted Answer · 2020-05-13 13:56:54Z

1

In order to understand what to do about this situation, you need to understand how a TCP connection is closed (see, e.g. this) and how the socket API relates to a clean shutdown (without fail, see this).

Your client is most likely calling close to terminate the connection. The problem with this is that there may be unread data in the socket receive queue or data arriving shortly from the other end that you will no longer be able to read, which is basically an error condition. To signal to the other end that data sent cannot be delivered to the receiving application, a reset is sent (well, technically, "SHOULD be sent" as per the RFC) and the TCP connection is abnormally terminated.

You might think that enabling SO_LINGER will help (many, many bits have been spilt over this so I won't elaborate further), but it won't solve the problem of unread data by the client causing the reset.

The client needs to instead call shutdown(SHUT_WR) to indicate that it is done sending, and then continue to call recv() until it reads 0 bytes indicating the other side is done sending. You may then call close().

Note that the Python 2 socket documentation states that

Depending on the platform, shutting down one half of the connection can also close the opposite half (e.g. on Mac OS X, shutdown(SHUT_WR) does not allow further reads on the other end of the connection).

This sounds like a bug to me. To get around this, you would have to send your cancel message, then keep reading until you get 0 bytes so that you know the server received the cancel message. You may then close the socket.

The Python 3.8 docs make no such disclaimer.

answered May 13, 2020 at 13:56

JimD.

2,4431 gold badge17 silver badges20 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

user207421 Over a year ago

Or be like FTP and use separate control and data connections, send a STOP on the control connection, await the reply, then close the data conection.

JimD. Over a year ago

@user207421 Indeed, there are many ways to solve this problem. I was primarily responding to the RST issue which is often not well understood.

Collectives™ on Stack Overflow

Python - Read remaining data from socket after TCP RST

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related