Sending various types of data through sockets in Python

Question

I currently have a Python client & server sending json object over socket as follows.

Client

# Create the socket & send the request
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
print 'Connecting to server at host: ' + (self.host) + ' port: ' + str(self.port)
s.connect((self.host, self.port))
print 'Sending signing request to the server'
s.sendall(request_data)
print 'Waiting for server response'
response_data = s.recv(10 * 1024)
print 'Got server response'
s.close()

Server

# Create a socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
print 'Starting the server at host: ' + (self.host) + ' port: ' + str(self.port)
s.bind((self.host, self.port))
s.listen(1)

while True:
    # Create a new connection
    print 'Listening for client requests...'
    conn, addr = s.accept()
    print 'Connected to: ' + str(addr)

    # Get the data
    request_data = conn.recv(10 * 1024)
    print 'Got message: ' + str(request_data)

    # Get the json object
    try:
        # Decode the data and do stuff
        # ...
        # ...
    except Exception as e:
        print e
    finally:
        # Close the connection
        conn.close()

However, besides the json object, I also need to send a file (which is not a json object). Inside the Server's while loop, the socket cannot distinguish when the json object ends and file starting receiving.

My questions here is about the methodology. What would be the usual approach to send two distinct types of data through the socket? Can we use the same socket to receive two data types in a serial order? Would that require two while loops (one for json, another for file) inside the current while loop?

Or are there any other ways of doing so?

Thanks.

The best approach would be using a higher level protocol than raw sockets, so the real question is, why are you reinventing the wheel? You must have a very good reason, for not using an stablished protocol like SSH/SFTP with paramiko, or use HTTP/FTP with twisted. — Paulo Scardine
– Paulo Scardine, Commented Dec 15, 2015 at 1:33

u354356007 · Accepted Answer · 2015-12-15 07:53:39Z

First things first, you cannot just do

response_data = s.recv(10 * 1024)
print 'Got server response'

or

# Get the data
request_data = conn.recv(10 * 1024)
print 'Got message: ' + str(request_data)

and then say you've got the data. Transmissions over TCP do not preserve their borders.

Regarding methodology, you need a protocol built over TCP. HTTP would be a great choice if you don't need your server to connect to clients without a request. In this case great libraries and frameworks are available.

If you want to build your own protocol, consider using control characters in your data stream. Something like this is possible:

json = b"{foo: ['b', 'a', 'r']}\n"  # \n here stands for end-of-the-json symbol
sock.send_byte(TYPE_JSON_MESSAGE)
sock.sendall(json)

sock.send_byte(TYPE_FILE_MESSAGE)
sock.send_int(file_size)  # so the server can determine where the file transmission ends
for chunk in chunked_file:
    sock.sendall(chunk)

Here it's up to you to implement send_byte and send_int. It's not really difficult, if you use struct module.

On the server side:

message_type = sock.recv(1)
if message_type == TYPE_JSON_MESSAGE:
    # ...
elif message_type == TYPE_FILE_MESSAGE:
    file_size = sock.recv_int()  # not implemented
    # ...

Collectives™ on Stack Overflow

Sending various types of data through sockets in Python

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related