In a multi-thread code, if there are several threads trying to send data to a tcp socket at the same time, what will happen? will their data be mixed or will different threads will end up sending data one by one?
-
How are you writing to the socket? Are you using write(2) or some C++ abstraction?kdgregory– kdgregory2011-10-30 00:38:06 +00:00Commented Oct 30, 2011 at 0:38
-
thanks, please see below for my comments, it's send/sendto for packets ~ 100 bytesDaniel– Daniel2011-10-30 00:47:11 +00:00Commented Oct 30, 2011 at 0:47
2 Answers
It depends upon which primitives you're using to submit data to the socket.
If you're using write(2), send(2), sendto(2), or sendmsg(2) and the size of your message is small enough to fit entirely within the kernel buffers for the socket, then that entire write will be sent as a block without other data being interspersed.
If you're using fwrite(3) (or any other higher-level buffered IO abstraction), then there is a chance that your data will be sent without any other data being interspersed, but I would not rely upon this behavior.
I can't speak to sendfile(2) behavior. I'd like to think that the sendfile(2) operation "writes" the entire contents of the file to the socket before any other write(2) requests on the socket, but the documentation I've read doesn't say a word about it, so you better not make the assumption that it is in any sense "atomic".
The safest mechanism is for only a single thread to ever be submitting data to a socket.
7 Comments
send(2) return for -1, errno == EMSGSIZE, in case you're sending data faster than the network can deliver it.Indeed as the previous answer says: "The safest mechanism is for only a single thread to ever be submitting data to a socket."
But if you want to have multiple threads call read/write, you must guarantee thread safety yourself. I've written a wrapper around send and recv below that does this for you.
Send() and Recv() are thread safe in the sense that they wont cause a crash, but there is no guarantee that that data wont be "intermixed" with data sent from other threads. Since we most likely don't want that, we must block until all the thread's data is confirmed sent or received, or errored out. Such a long blocking call can be problematic, so make sure you're doing this on threads that can handle long blocking operations.
For linux:
#include <sys/types.h>
#include <sys/socket.h>
#include <map>
#include <mutex>
/* blocks until the full amount of bytes requested are read
* thread safe
* throws exception on error */
void recv_bytes(int sock, char* buf, int len, int flags){
static std::map<int, std::mutex> mtx;
std::lock_guard<std::mutex> lock(mtx[sock]);
int bytes_received = 0;
while (bytes_received != len){
int bytes = recv(sock, buf + bytes_received, len - bytes_received, flags);
//error check
if (bytes == -1){
throw std::runtime_error("Network Exception");
}
bytes_received += bytes;
}
}
/* blocks until the full amount of bytes requested are sent
* thread safe
* throws exception on error */
void send_bytes(int sock, char* buf, int len, int flags){
static std::map<int, std::mutex> mtx;
std::lock_guard<std::mutex> lock(mtx[sock]);
int bytes_sent = 0;
while (bytes_sent != len){
int bytes_s0 = send(sock, buf, len, flags);
if (bytes_sent == -1) {
throw std::runtime_error("Network Exception");
}
bytes_sent += bytes_s0;
}
}