grpc client python: How to create grpc client connection pool for better throughput?

Question

Our usecase is to make a large number of requests. Each request return 1 MB of data. Right now, on client side, we create a single GRPC channel and the run the following function in a loop

content_grpc_channel = grpc.insecure_channel(content_netloc)
test_stub = test_pb2_grpc.ContentServiceInternalStub(
    content_grpc_channel)

def get_feature_data_future(feature_id, span_context=()):
  req_feature = test_pb2.GetFeatureRequest()
  req_feature.feature_id = feature_id
  resp_feature_future = test_stub.GetFeature.future(
      req_feature, metadata=span_context)
  return resp_feature_future

My question is in python how I can create grpc client connection pool for better throughput?

In golang I see this https://godoc.org/google.golang.org/api/option#WithGRPCConnectionPool but I have a hard time to find the doc in python.

Is there such a utility in python to create grpc connection pool? Or should I create multiple grpc channels and manage those myself? I assume each channel will have different tcp connection, correct?

Not a full answer, but grpc uses http/2 and can easily handle multiple channels on a single tcp connection. — Hans Musgrave
– Hans Musgrave, Commented Aug 13, 2020 at 23:01
@HansMusgrave Understood. Are you saying using multiple TCP connections (each connection also load balance across multiple servers) will not yield better throughput than a single connection? — SonGohan Nyc
– SonGohan Nyc, Commented Aug 14, 2020 at 0:22
They may or may not. Network performance can vary a ton based on configuration details and usage patterns. As a rough heuristic though, all the data is going through a single NIC at both ends, and the channel multiplexing overhead has to happen somewhere, be it the tcp stack or the http/2 implementation -- I wouldn't a priori expect one to typically be better than the other for arbitrary hardware and network settings. Multiple connections have more warm-up overhead, but if you're "[making] a large number of requests" then that's negligible, so start with what's easiest, benchmark, and iterate. — Hans Musgrave
– Hans Musgrave, Commented Aug 14, 2020 at 3:29

Alexsandro Souza · Accepted Answer · 2020-08-16 10:40:27Z

1

gRPC uses HTTP/2 and can multiplex many requests on one connection and gRPC client connections should be re-used for the lifetime of the client app.

The Golang link you mentioned, says that WithGRPCConnectionPool would be used to balance the requests. You might search for load balancing if it is what you need but remember that load balancing only makes sense if you have multiple gRPC server instances.

If you are searching for a connection pool inspired by what is done when working with databases, I would say you don't need to worry about it as the opening connection overhead doesn't exist when working with gRPC

answered Aug 16, 2020 at 10:40

Alexsandro Souza

1,0058 silver badges15 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

grpc client python: How to create grpc client connection pool for better throughput?

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related