0

I was trying to send some data over sockets, but I noticed that the bytes I send sometimes just get concatenated together.

Sorry if the wording is not great, but here’s some example code to reproduce this problem:

# SERVER CODE

import socket, pickle
from _thread import start_new_thread

IP = "0.0.0.0"  # Address to bind to
PORT = 5555  # Arbitrary non-privileged port
DEFAULT_BYTES = 2048
total_connections_so_far = 0


def send(data, conn, pickle_data=True):
    try:
        if pickle_data:
            data = pickle.dumps(data)
        conn.sendall(data)
    except Exception as e:
        print("ERROR TRYING TO SEND DATA: ", e)


def threaded_client(conn, addr, user_id):

    send(b"Hello there!", conn, False)
    send(b"2nd message", conn, False)
    send(b"Last message", conn, False)

    conn.close()


with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
    # bind the socket to the host address and port
    s.bind((IP, PORT))
    print("Server started at: ", s.getsockname())

    # listen for connections
    s.listen()
    print("Server has started. waiting for connections...")

    while True:

        conn, addr = s.accept()
        print("[CONNECTED]: ", addr)
        total_connections_so_far += 1  # increment the totoal connections

        # start a thread for the new client
        start_new_thread(threaded_client, (conn, addr, total_connections_so_far))



☝🏻 server.py

# CLIENT CODE

import socket, pickle


class Network:
    def __init__(self):
        self.client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        self.server = ""
        self.port = 5555
        self.addr = (
            self.server,
            self.port,
        )

    # function to connect to the server
    def connect(self):
        try:
            self.client.connect(self.addr)
            print("Connected!")

        except Exception as e:
            print("error while trying to connect:", e)
            return False

    # send some data to the server
    def send(self, data, pickle_data=True):
        try:
            if pickle_data:
                data = pickle.dumps(data)

            self.client.sendall(data)

        except Exception as e:
            print("error while trying to send data:", e)
            return False

    # recieve some data from the server
    def recv(self, buffer_size=2048):
        try:
            data = self.client.recv(buffer_size)
            return data
        except Exception as e:
            print("error while recieving:", e)


client = Network()
client.connect()
data = client.recv()
print(data)


☝🏻 client.py


Try running the client code a few times and you’ll notice that sometimes, the data received is a concatenation of all 3 messages sent from the server.

So to get around this problem, I have been using time.sleep(1) after every time I send something, but this is obviously not a great idea.

I understand that this happens cuz ( correct me if I’m wrong ) I’m only sending tiny bits of data from the server, and expecting to receive 2048 bits on the client side, so the client waits for a while to try and receive the full amount of data.

But this is a pretty common problem and there must be a neat solution to it right?

I know I’m a total noob, but please help me!

8
  • 4
    TCP is a stream protocol, there are no message boundaries. Commented Dec 29, 2021 at 5:01
  • So, is there no legitimate method to add message boundaries? Commented Dec 29, 2021 at 5:02
  • 4
    Sure add them any way you want. Put nulls or new lines between messages, or send a fixed length message, or send a size and then the message. The only guarantee you get from TCP is the bytes arrive in the same order sent. You define what a complete message looks like and you are responsible for receiving until you have a complete message Commented Dec 29, 2021 at 5:52
  • Oh alright! I guess I was just hoping that python would do all hard work for me… Thanks for you comment tho, I’ll try sending the size first and then receiving those many bytes on the client side. Commented Dec 29, 2021 at 6:08
  • hey @MarkTolonen, this might be a dumb question, but I just realised if I’m gonna send the length of the message first, and then the message itself… wont the “length of the message” and the “message” get concatenated? Commented Dec 29, 2021 at 9:00

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.