How should I implement a timeout in a GRPC call?

The doc of WithTimeout says that it is used to set timeout of connection initializing, and not to set timeout to the calls. context in the DialContext is the same.

WithTimeout returns a DialOption that configures a timeout for dialing a ClientConn initially. This is valid if and only if WithBlock() is present. Deprecated: use DialContext instead of Dial and context.WithTimeout instead. Will be supported throughout 1.x.

To set timeout to the calls you can pass context to invoke like:

ctx, cancel := context.WithDeadline(ctx, time.Now().Add(time.Duration(2000)*time.Millisecond))
defer cancel()
clientConn.Invoke(ctx, "/YourEndpoint", in, out, opts...)

The context.WithTimeout is used in the grpc.DialContext to control the timeout of all RPC calls of the current DialContext. It is inconvenient to handle the different timeouts of one/some specific RPC call.

We could define one custom timeout callOption to handle the forced timeout value of some RPC calls in the clientInterceptor

Define the custom timeout callOption through EmptyCallOption

type TimeoutCallOption struct {
    grpc.EmptyCallOption

    forcedTimeout time.Duration
}

func WithForcedTimeout(forceTimeout time.Duration) TimeoutCallOption {
    return TimeoutCallOption{forcedTimeout: forceTimeout}
}

Handle the forcedTimeout in the UnaryClientInterceptor

func getForcedTimeout(callOptions []grpc.CallOption) (time.Duration, bool) {
    for _, opt := range callOptions {
        if co, ok := opt.(TimeoutCallOption); ok {
            return co.forcedTimeout, true
        }
    }

    return 0, false
}

func TimeoutInterceptor(t time.Duration) grpc.UnaryClientInterceptor {
    return func(ctx context.Context, method string, req, reply interface{}, cc *grpc.ClientConn,
        invoker grpc.UnaryInvoker, opts ...grpc.CallOption) error {
        timeout := t
        if v, ok := getForcedTimeout(opts); ok {
            timeout = v
        }

        if timeout <= 0 {
            return invoker(ctx, method, req, reply, cc, opts...)
        }

        ctx, cancel := context.WithTimeout(ctx, timeout)
        defer cancel()

        return invoker(ctx, method, req, reply, cc, opts...)
    }
}

Usage samples

// The default timeout of RPC call of this conn is 3 seconds
conn, err := grpc.Dial(
        address,
grpc.WithUnaryInterceptor(TimeoutInterceptor(time.Duration(3)*time.Second)), ...)

....

c := pb.NewGreeterClient(conn)
c.SayHello(context.Background(), &pb.HelloRequest{Name: "world"},
            WithForcedTimeout(time.Duration(10)*time.Second))
// The timeout of SayHello RPC call is 10 seconds

The old syntax using grpc.WithTimeout is deprecated.

opts := []grpc.DialOption{
  // you would also need grpc.WithBlock() which can be an anti-pattern
  // grpc.WithBlock(),
  grpc.WithTimeout(timeoutValue),
  // ...
}
clientConn, err := grpc.Dial(serverAddress, opts...)
if err != nil {
    return err
}

https://pkg.go.dev/google.golang.org/grpc#WithTimeout

WithTimeout returns a DialOption that configures a timeout for dialing a ClientConn initially. This is valid if and only if WithBlock() is present.

Deprecated: use DialContext instead of Dial and context.WithTimeout instead. Will be supported throughout 1.x.

So if you use the new syntax, that is still only applicable to a timeout for dialing the initial ClientConn.

opts := []grpc.DialOption{
  ...
}
ctx, cancel := context.WithTimeout(context.Background(), timeoutValue)
clientConn, err := grpc.DialContext(ctx, serverAddress, opts...)
if err != nil {
    return err
}

I am using this code to connect to a grpc server and clientConn object is used for all subsequent rpc calls. maxDelay is set to 5 seconds. Now because of some issue at server, it is not responding for a grpc call. So my client is waiting for a long time for each rpc call. Do I need to set timeout in a different way?

If you want a timeout on each call; then that's beyond the initial ClientConn. You would want instead to provide use context.WithTimeout and pass that to ClientConn.Invoke. So you could either do that yourself, or configure an interceptor for the timeout.

https://pkg.go.dev/google.golang.org/grpc#ClientConn.Invoke

There are 2 nice packages to use for configuring common client interceptors for timeouts and retries:

• https://github.com/grpc-ecosystem/go-grpc-middleware/tree/main/interceptors/retry
• https://github.com/grpc-ecosystem/go-grpc-middleware/tree/main/interceptors/timeout

import (
    "context"
    "github.com/grpc-ecosystem/go-grpc-middleware/v2/interceptors/timeout"
)

opts := []grpc.DialOption{
    // timeout for each unary call
    timeout.UnaryClientInterceptor(timeoutValue),
    // ...
}

// timeout for initial ClientConn
ctx, cancel := context.WithTimeout(context.Background(), timeoutValue)
clientConn, err := grpc.DialContext(ctx, serverAddress, opts...)
if err != nil {
    return err
}

These solutions were deprecated (again) in March, 2024 (grpc commit). Below is how to achieve timeouts with grpc.NewClient in grpc v1.17.0.

import (
    "google.golang.org/grpc"
    "google.golang.org/grpc/backoff"
    "google.golang.org/grpc/credentials/insecure"
)

// Does not connect immediately
//   Waits for RPC call or conn.Connect()
dialOpts := []grpc.DialOption{
    grpc.WithTransportCredentials(insecure.NewCredentials()), // Disable TLS
    grpc.WithConnectParams(grpc.ConnectParams{
        Backoff: backoff.Config{
            BaseDelay:  100 * time.Millisecond,
            Multiplier: 1.6,
            Jitter:     0.2,
            MaxDelay:   3 * time.Second,
        },
        MinConnectTimeout: 5 * time.Second,
    }),
}

conn, err := grpc.NewClient("localhost:50051", dialOpts...)
defer conn.Close()

For connection/error logging, add an interceptor to dialOpts (function below):

import "google.golang.org/grpc/connectivity"
 
dialOpts := []grpc.DialOption{
    // other options
    grpc.WithUnaryInterceptor(logInterceptor), 
}

An example log interceptor function implementing UnaryClientInterceptor.

func logInterceptor(
    ctx context.Context,
    method string,
    req, reply interface{},
    cc *grpc.ClientConn,
    invoker grpc.UnaryInvoker,
    opts ...grpc.CallOption,
) error {
    log.Printf("Attempting to call method: %s", method)
    err := invoker(ctx, method, req, reply, cc, opts...)
    if err != nil {
        log.Printf("Error during RPC call: %v", err)
    } else {
        log.Printf("Successfully called method: %s", method)
    }

    state := cc.GetState()
    
    switch state {
    case connectivity.TransientFailure:
        log.Println("Transient failure detected. Retrying...")
    case connectivity.Shutdown:
        log.Println("Connection has been shut down.")
    case connectivity.Ready:
        log.Println("Connection ready.")
    default:
        log.Printf("Other state: %v", state)
    }
    
    return err
}