Grpc not increasing max size of the message: received message larger than max

Question

I am using grpc to send some pretty large messages (the parameters of a machine learning model over the network). The problem is that I am getting the following error when I make a grpc call:

grpc: received message larger than max (261268499 vs. 4194304)

As suggested in other posts I tried to increase the max message size on the channel and the grpc server, but I keep getting the same error. Any idea on how to get this to work?

My code for the server:

maxMsgLength = 1024 * 1024 * 1024
server = grpc.server(futures.ThreadPoolExecutor(max_workers=10),
                     options=[('grpc.max_message_length', maxMsgLength),
                              ('grpc.max_send_message_length', maxMsgLength),
                              ('grpc.max_receive_message_length', maxMsgLength)])

The client:

maxMsgLength = 1024 * 1024 * 1024
channel = grpc.insecure_channel(ip_port, 
                                options=[('grpc.max_message_length', maxMsgLength),
                                         ('grpc.max_send_message_length', maxMsgLength),
                                         ('grpc.max_receive_message_length', maxMsgLength)])

Edit:

Not a solution, but maybe gives a little bit more insight into the problem. For some reason, if I set the max message size to 1024 * 1024 * 1024 it ends up defaulting to 4194304 as the error message implies. Not really sure why that happens. But anyways, I tried reducing the max message size to 1024 * 1024 * 200 and it shows the correct max message size in the error message (209715200). It seems like there is a problem where grpc is not setting the max message size properly. Not sure how to get around this though.

The maximum number I can use where the error message shows the proper max value is 2^28. If I put a max message size of 2^29 it defaults to 4194304.

why instead of sending all the parameters in a single message you don't send each parameter at a time in an individual message? — tr0fin0
– tr0fin0, Commented Jun 29 at 6:05

Pankaj Saini · Accepted Answer · 2023-04-02 03:02:22Z

0

You can send your data in chunk using grpc streaming API as mentioned in the below example. Although, more details are needed for better clarity of your exact requirement.

Protobuf File

syntax = "proto3";

package pb.foo_service;

service FooService {
    rpc TestRpc(TestRequest) returns (stream TestResponse) {}
}

message TestRequest {}

message TestResponse {
  bytes data = 1;
}

Servicer

class FooService(pb_grpc.FooServiceServicer):

    def TestRpc(
        self,
        request: pb.TestRequest,
        context: grpc.ServicerContext
    ) -> Iterator[pb.TestResponse]:
        with open("test_file", 'rb') as content_file:
            content = content_file.read()
    yield my_generated_module_pb2.Response(data=content)

answered Apr 2, 2023 at 3:02

Pankaj Saini

8107 silver badges7 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Daniel Hornik Over a year ago

But the question is different. Often you are not the one who implements the server :(

TASK · Accepted Answer · 2025-06-15 01:44:21Z

0

Try to set the options on insecure_channel of client:

channel = grpc.insecure_channel(ip_port, 
                                options=[('grpc.max_message_length', -1),
                                         ('grpc.max_send_message_length', -1),
                                         ('grpc.max_receive_message_length', -1)])

It's worked for me.

answered Jun 15 at 1:44

TASK

3451 gold badge4 silver badges14 bronze badges

2 Comments

tr0fin0 Jun 29 at 6:03

why -1 would work?

TASK Jun 30 at 2:52

@tr0fin0, from grpc docs grpc.github.io/grpc/python/glossary.html#term-channel_arguments and github repository github.com/grpc/grpc/blob/…

tr0fin0 · Accepted Answer · 2025-06-29 06:16:12Z

0

according to this GitHub thread there is a 4Mb ( 33 554 432 bits ) message limit on gRPC which could be related with the limts you encounter.

I would suggest to send each parameter on an individual gRPC message which would avoid the message limit and could be easier to maintain.

edited Jun 29 at 6:16

answered Jun 29 at 6:14

tr0fin0

52 silver badges6 bronze badges

Collectives™ on Stack Overflow

Grpc not increasing max size of the message: received message larger than max

3 Answers 3

Protobuf File

Servicer

1 Comment

2 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Protobuf File

Servicer

1 Comment

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related