Interceptors

As you build gRPC applications, you may want to execute some common logic before or after the execution of the remote function, for either client or server applications. In gRPC you can intercept that RPC’s execution to meet certain requirements such as logging, authentication, metrics, etc., using an extension mechanism called an interceptor. gRPC provides simple APIs to implement and install interceptors in your client and server gRPC applications. They are one of the key extension mechanisms in gRPC and are quite useful in use cases such as logging, authentication, authorization, metrics, tracing, and any other customer requirements.

gRPC interceptors can be categorized into two types based on the type of RPC calls they intercept. For unary RPC you can use unary interceptors, while for streaming RPC you can use streaming interceptors. These interceptors can be used on the gRPC server side or on the gRPC client side. First, let’s start by looking at using interceptors on the server side.

Server-Side Interceptors

When a client invokes a remote method of a gRPC service, you can execute a common logic prior to the execution of the remote methods by using a server-side interceptor. This helps when you need to apply certain features such as authentication prior to invoking the remote method. As shown in Figure 5-1, you can plug one or more interceptors into any gRPC server that you develop. For example, to plug a new server-side interceptor into your OrderManagement gRPC service, you can implement the interceptor and register it when you create the gRPC server.

Figure 5-1. Server-side interceptors

On the server side, the unary interceptor allows you to intercept the unary RPC call while the streaming interceptor intercepts the streaming RPC. Let’s first discuss server-side unary interceptors.

Unary interceptor

If you want to intercept the unary RPC of your gRPC service at the server side, you’ll need to implement a unary interceptor for your gRPC server. As shown in the Go code snippet in Example 5-1, you can do this by implementing a function of type UnaryServerInterceptor and registering that function when you create a gRPC server. UnaryServerInterceptor is the type for a server-side unary interceptor with the following signature:

func(ctx context.Context, req interface{}, info *UnaryServerInfo,
	                     handler UnaryHandler) (resp interface{}, err error)

Inside this function you get full control of all unary RPC calls that are coming to your gRPC server.

Example 5-1. gRPC server-side unary interceptor

// Server - Unary Interceptor
func orderUnaryServerInterceptor(ctx context.Context, req interface{},
                             info *grpc.UnaryServerInfo, handler grpc.UnaryHandler)
                             (interface{}, error) {

	// Preprocessing logic
	// Gets info about the current RPC call by examining the args passed in
	log.Println("======= [Server Interceptor] ", info.FullMethod) 


	// Invoking the handler to complete the normal execution of a unary RPC.
	m, err := handler(ctx, req) 

	// Post processing logic
	log.Printf(" Post Proc Message : %s", m) 
	return m, err 
}


// ...

func main() {

...
     // Registering the Interceptor at the server-side.
	s := grpc.NewServer(
		grpc.UnaryInterceptor(orderUnaryServerInterceptor)) 
...

Preprocessing phase: this is where you can intercept the message prior to invoking the respective RPC.

Invoking the RPC method via UnaryHandler.

Postprocessing phase: you can process the response from the RPC invocation.

Sending back the RPC response.

Registering the unary interceptor with the gRPC server.

The implementation of a server-side unary interceptor can usually be divided into three parts: preprocessing, invoking the RPC method, and postprocessing. As the name implies, the preprocessor phase is executed prior to invoking the remote method intended in the RPC call. In the preprocessor phase, users can get info about the current RPC call by examining the args passed in, such as RPC context, RPC request, and server information. Thus, during the preprocessor phase you can even modify the RPC call.

Then, in the invoker phase, you have to call the gRPC UnaryHandler to invoke the RPC method. Once you invoke the RPC, the postprocessor phase is executed. This means that the response for the RPC call goes through the postprocessor phase. In the phase, you can deal with the returned reply and error when required. Once the postprocessor phase is completed, you need to return the message and the error as the return parameters of your interceptor function. If no postprocessing is required, you can simply return the handler call (handler(ctx, req)).

Next, let’s discuss streaming interceptors.

Stream interceptor

The server-side streaming interceptor intercepts any streaming RPC calls that the gRPC server deals with. The stream interceptor includes a preprocessing phase and a stream operation interception phase.

As shown in the Go code snippet in Example 5-2, suppose that we want to intercept streaming RPC calls of the OrderManagement service. StreamServerInterceptor is the type for server-side stream interceptors. orderServerStreamInterceptor is an interceptor function of type StreamServerInterceptor with the signature:

func(srv interface{}, ss ServerStream, info *StreamServerInfo,
                                     handler StreamHandler) error

Similar to a unary interceptor, in the preprocessor phase, you can intercept a streaming RPC call before it goes to the service implementation. After the preprocessor phase, you can then invoke the StreamHandler to complete the execution of RPC invocation of the remote method. After the preprocessor phase, you can intercept the streaming RPC message by using an interface known as a wrapper stream that implements the grpc.ServerStream interface. You can pass this wrapper structure when you invoke grpc.StreamHandler with handler(srv, newWrappedStream(ss)). The wrapper of grpc.ServerStream intercepts the streaming messages sent or received by the gRPC service. It implements the SendMsg and RecvMsg functions, which will be invoked when the service receives or sends an RPC streaming message.

Example 5-2. gRPC server-side streaming interceptor

// Server - Streaming Interceptor
// wrappedStream wraps around the embedded grpc.ServerStream,
// and intercepts the RecvMsg and SendMsg method call.

type wrappedStream struct { 
	grpc.ServerStream
}


func (w *wrappedStream) RecvMsg(m interface{}) error {
	log.Printf("====== [Server Stream Interceptor Wrapper] " +
		"Receive a message (Type: %T) at %s",
		m, time.Now().Format(time.RFC3339))
	return w.ServerStream.RecvMsg(m)
}


func (w *wrappedStream) SendMsg(m interface{}) error {
	log.Printf("====== [Server Stream Interceptor Wrapper] " +
		"Send a message (Type: %T) at %v",
		m, time.Now().Format(time.RFC3339))
	return w.ServerStream.SendMsg(m)
}


func newWrappedStream(s grpc.ServerStream) grpc.ServerStream {
	return &wrappedStream{s}
}


func orderServerStreamInterceptor(srv interface{},
        ss grpc.ServerStream, info *grpc.StreamServerInfo,
        handler grpc.StreamHandler) error {
	log.Println("====== [Server Stream Interceptor] ",
		info.FullMethod) 
	err := handler(srv, newWrappedStream(ss)) 
	if err != nil {
		log.Printf("RPC failed with error %v", err)
	}
	return err
}


...
// Registering the interceptor
s := grpc.NewServer(
		grpc.StreamInterceptor(orderServerStreamInterceptor)) 

…

Wrapper stream of the grpc.ServerStream.

Implementing the RecvMsg function of the wrapper to process messages received with stream RPC.

Implementing the SendMsg function of the wrapper to process messages sent with stream RPC.

Creating an instance of the new wrapper stream.

Streaming interceptor implementation.

Preprocessor phase.

Invoking the streaming RPC with the wrapper stream.

Registering the interceptor.

To understand the behavior of the streaming interceptor on the server side, look at the following output from the gRPC server logs. Based on the order in which each log message is printed you can identify the behavior of the streaming interceptor. The streaming remote method that we have invoked here is SearchOrders, which is a server-streaming RPC:

[Server Stream Interceptor]  /ecommerce.OrderManagement/searchOrders
[Server Stream Interceptor Wrapper] Receive a message

Matching Order Found : 102 -> Writing Order to the stream ...
[Server Stream Interceptor Wrapper] Send a message...
Matching Order Found : 104 -> Writing Order to the stream ...
[Server Stream Interceptor Wrapper] Send a message...

Client-side interceptor terminology is quite similar to that of server-side interceptors, with some subtle variations as to the interfaces and function signatures. Let’s move on to the details of client-side interceptors.

Client-Side Interceptors

When a client invokes an RPC call to invoke a remote method of a gRPC service, you can intercept those RPC calls on the client side. As shown in Figure 5-2, with client-side interceptors, you can intercept unary RPC calls as well as streaming RPC calls.

Figure 5-2. Client-side interceptors

This is particularly useful when you need to implement certain reusable features, such as securely calling a gRPC service outside the client application code.

Unary interceptor

A client-side unary RPC interceptor is used for intercepting the unary RPC client side. UnaryClientInterceptor is the type for a client-side unary interceptor that has a function signature as follows:

func(ctx context.Context, method string, req, reply interface{},
         cc *ClientConn, invoker UnaryInvoker, opts ...CallOption) error

As we saw with the server-side unary interceptor, the client-side unary interceptor has different phases. Example 5-3 shows the basic Go implementation of a unary interceptor on the client side. In the preprocessor phase, you can intercept the RPC calls before invoking the remote method. Here you will have access to the information about the current RPC call by examining the args passed in, such as RPC context, method string, request to be sent, and CallOptions configured. So, you can even modify the original RPC call before it is sent to the server application. Then using the UnaryInvoker argument you can invoke the actual unary RPC. In the postprocessor phase, you can access the response or the error results of the RPC invocation.

Example 5-3. gRPC client-side unary interceptor

func orderUnaryClientInterceptor(
	ctx context.Context, method string, req, reply interface{},
	cc *grpc.ClientConn,
	invoker grpc.UnaryInvoker, opts ...grpc.CallOption) error {
	// Preprocessor phase
	log.Println("Method : " + method) 

	// Invoking the remote method
	err := invoker(ctx, method, req, reply, cc, opts...) 

	// Postprocessor phase
	log.Println(reply) 

	return err 
}
...

func main() {
	// Setting up a connection to the server.
	conn, err := grpc.Dial(address, grpc.WithInsecure(),
		grpc.WithUnaryInterceptor(orderUnaryClientInterceptor)) 
...

Preprocessing phase has access to the RPC request prior to sending it out to the server.

Invoking the RPC method via UnaryInvoker.

Postprocessing phase where you can process the response or error results.

Returning an error back to the gRPC client application along with a reply, which is passed as an argument.

Setting up a connection to the server by passing a unary interceptor as a dial option.

Registering the interceptor function is done inside the grpc.Dial operation using grpc.WithUnaryInterceptor.

Stream interceptor

The client-side streaming interceptor intercepts any streaming RPC calls that the gRPC client deals with. The implementation of the client-side stream interceptor is quite similar to that of the server side. StreamClientInterceptor is the type for a client-side stream interceptor; it is a function type with this signature:

func(ctx context.Context, desc *StreamDesc, cc *ClientConn,
                                      method string, streamer Streamer,
                                      opts ...CallOption) (ClientStream, error)

As shown in Example 5-4, the client-side stream interceptor implementation includes preprocessing and stream operation interception.

Example 5-4. gRPC client-side stream interceptor

func clientStreamInterceptor(
	ctx context.Context, desc *grpc.StreamDesc,
	cc *grpc.ClientConn, method string,
	streamer grpc.Streamer, opts ...grpc.CallOption)
        (grpc.ClientStream, error) {
	log.Println("======= [Client Interceptor] ", method) 
	s, err := streamer(ctx, desc, cc, method, opts...) 
	if err != nil {
		return nil, err
	}
	return newWrappedStream(s), nil 
}


type wrappedStream struct { 
	grpc.ClientStream
}

func (w *wrappedStream) RecvMsg(m interface{}) error { 
	log.Printf("====== [Client Stream Interceptor] " +
		"Receive a message (Type: %T) at %v",
		m, time.Now().Format(time.RFC3339))
	return w.ClientStream.RecvMsg(m)
}

func (w *wrappedStream) SendMsg(m interface{}) error { 
	log.Printf("====== [Client Stream Interceptor] " +
		"Send a message (Type: %T) at %v",
		m, time.Now().Format(time.RFC3339))
	return w.ClientStream.SendMsg(m)
}

func newWrappedStream(s grpc.ClientStream) grpc.ClientStream {
	return &wrappedStream{s}
}

...

func main() {
	// Setting up a connection to the server.
	conn, err := grpc.Dial(address, grpc.WithInsecure(),
		grpc.WithStreamInterceptor(clientStreamInterceptor)) 
...

Preprocessing phase has access to the RPC request prior to sending it out to the server.

Calling the passed-in streamer to get a ClientStream.

Wrapping around the ClientStream, overloading its methods with intercepting logic, and returning it to the client application.

Wrapper stream of grpc.ClientStream.

Function to intercept messages received from streaming RPC.

Function to intercept messages sent from streaming RPC.

Registering a streaming interceptor.

Intercepting for stream operations is done via a wrapper implementation of the stream where you have to implement a new structure wrapping grpc.ClientStream. Here you implement two wrapped stream methods, RecvMsg and SendMsg, that can be used to intercept streaming messages received or sent from the client side. The registration of the interceptor is the same as for the unary interceptor and is done with the grpc.Dial operation.

Let’s look at deadlines, another capability you’ll often need to apply when calling gRPC services from the client application.

Deadlines

Deadlines and timeouts are two commonly used patterns in distributed computing. Timeouts allow you to specify how long a client application can wait for an RPC to complete before it terminates with an error. A timeout is usually specified as a duration and locally applied at each client side. For example, a single request may consist of multiple downstream RPCs that chain together multiple services. So we can apply timeouts, relative to each RPC, at each service invocation. Therefore, timeouts cannot be directly applied for the entire life cycle of the request. That’s where we need to use deadlines.

A deadline is expressed in absolute time from the beginning of a request (even if the API presents them as a duration offset) and applied across multiple service invocations. The application that initiates the request sets the deadline and the entire request chain needs to respond by the deadline. gRPC APIs supports using deadlines for your RPC. For many reasons, it is always good practice to use deadlines in your gRPC applications. gRPC communication happens over the network, so there can be delays between the RPC calls and responses. Also, in certain cases the gRPC service itself can take more time to respond depending on the service’s business logic. When client applications are developed without using deadlines, they infinitely wait for a response for RPC requests that are initiated and resources will be held for all in-flight requests. This puts the service as well as the client at risk of running out of resources, increasing the latency of the service; this could even crash the entire gRPC service.

The example scenario shown in Figure 5-3 illustrates a gRPC client application calling a product management service that again invokes the inventory service.

The client application sets a deadline offset (i.e., deadline = current time + offset) of 50 ms. The network latency between the client and ProductMgt service is 0 ms and the processing latency of the ProductMgt service is 20 ms. The product management service has to set a deadline offset of 30 ms. Since the inventory service takes 30 ms to respond, the deadline event would occur on both client sides (ProductMgt invokes the Inventory service and the client application).

The latency added from the business logic of the ProductMgt service is 20 ms. Then the ProductMgt service’s invocation logic triggers the deadline-exceeded scenario and propagates it back to the client application as well. Therefore, when using deadlines, make sure that they are applied across all services.

Figure 5-3. Using deadlines when calling services

A client application can set a deadline when it initiates a connection with a gRPC service. Once the RPC call is made, the client application waits for the duration specified by the deadline; if the response for the RPC call is not received within that time, the RPC call is terminated with a DEADLINE_EXCEEDED error.

Let’s look at a real-world example of using deadlines when invoking gRPC services. In the same OrderManagement service use case, suppose the AddOrder RPC takes a significant amount of time to complete (we’ve simulated this with the introduction of a delay into the AddOrder method of the OrderManagement gRPC service). But the client application only waits until the response is no longer useful to it. For example, the duration that AddOrder takes to respond is two seconds, while the client only waits two seconds for a response. To implement this (as shown in the Go code snippet shown in Example 5-5), the client application can set the two-second timeout with the context.WithDeadline operation. We have used the status package to process error code; we’ll discuss this in detail in the error-handling section.

conn, err := grpc.Dial(address, grpc.WithInsecure())
if err != nil {
    log.Fatalf("did not connect: %v", err)
}
defer conn.Close()
client := pb.NewOrderManagementClient(conn)

clientDeadline := time.Now().Add(
    time.Duration(2 * time.Second))
ctx, cancel := context.WithDeadline(
    context.Background(), clientDeadline) 

defer cancel()

// Add Order
order1 := pb.Order{Id: "101",
    Items:[]string{"iPhone XS", "Mac Book Pro"},
    Destination:"San Jose, CA",
    Price:2300.00}
res, addErr := client.AddOrder(ctx, &order1) 

if addErr != nil {
    got := status.Code(addErr) 
    log.Printf("Error Occured -> addOrder : , %v:", got) 
} else {
    log.Print("AddOrder Response -> ", res.Value)
}

Setting a two-second deadline on the current context.

Invoking the AddOrder remote method and capturing any possible errors into addErr.

Using the status package to determine the error code.

If the invocation exceeds the specified deadline, it should return an error of the type DEADLINE_EXCEEDED.

So how should we determine the ideal value for the deadline? There is no single answer to that question, but you need to consider several factors in making that choice; mainly, the end-to-end latency of each service that we invoke, which RPCs are serial and which RPCs can be made in parallel, the latency of the underlying network, and the deadline values of the downstream services. Once you are able to come up with the initial value for the deadline, fine-tune it based on the operating condition of the gRPC applications.

When it comes to deadlines in gRPC, both the client and server can make their own independent and local determination about whether the RPC was successful; this means their conclusions may not match. For instance, in our example, when the client meets the DEADLINE_EXCEEDED condition, the service may still try to respond. So, the service application needs to determine whether the current RPC is still valid or not. From the server side, you can also detect when the client has reached the deadline specified when invoking the RPC. Inside the AddOrder operation, you can check for ctx.Err() == context.DeadlineExceeded to find out whether the client has already met the deadline exceeded state, and then abandon the RPC at the server side and return an error (this is often implemented using a nonblocking select construct in Go).

Similar to deadlines, there can be certain situations in which your client or server application wants to terminate the ongoing gRPC communication. This is where gRPC cancellation becomes useful.

Cancellation

In a gRPC connection between a client and server application, both the client and server make independent and local determinations of the success of the call. For instance, you could have an RPC that finishes successfully on the server side but fails on the client side. Similarly, there can be various conditions where the client and server may end up with different conclusions on the results of an RPC. When either the client or server application wants to terminate the RPC this can be done by canceling the RPC. Once the RPC is canceled, no further RPC-related messaging can be done and the fact that one party has canceled the RPC is propagated to the other side.

Let’s take the example of bidirectional streaming between the client and server applications. In the Go code sample shown in Example 5-6, you can obtain the cancel function from the context.WithTimeout call. Once you have the reference to cancel, you can call it at any location where you intend to terminate the RPC.

ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second) 


streamProcOrder, _ := client.ProcessOrders(ctx) 
_ = streamProcOrder.Send(&wrapper.StringValue{Value:"102"}) 
_ = streamProcOrder.Send(&wrapper.StringValue{Value:"103"})
_ = streamProcOrder.Send(&wrapper.StringValue{Value:"104"})

channel := make(chan bool, 1)

go asncClientBidirectionalRPC(streamProcOrder, channel)
time.Sleep(time.Millisecond * 1000)

// Canceling the RPC
cancel() 
log.Printf("RPC Status : %s", ctx.Err()) 

_ = streamProcOrder.Send(&wrapper.StringValue{Value:"101"})
_ = streamProcOrder.CloseSend()

<- channel

func asncClientBidirectionalRPC (
    streamProcOrder pb.OrderManagement_ProcessOrdersClient, c chan bool) {
...
		combinedShipment, errProcOrder := streamProcOrder.Recv()
		if errProcOrder != nil {
			log.Printf("Error Receiving messages %v", errProcOrder) 
...
}

Obtaining the reference to cancel.

Invoking the streaming RPC.

Sending messages to the service via the stream.

Canceling RPC/terminating RPC from the client side.

Status of the current context.

Returning context canceled error when trying to receive messages from a canceled context.

When one party cancels the RPC, the other party can determine it by checking the context. In this example, the server application can check whether the current context is canceled by using stream.Context().Err() == context.Canceled.

As you have seen in the application of deadlines as well as cancellation, handling errors with RPC is a very common requirement. In the next section, we look at gRPC error-handling techniques in detail.

Error Handling

When we invoke a gRPC call, the client receives a response with a successful status or an error with the corresponding error status. The client application needs to be written in such a way that you handle all the potential errors and error conditions. The server application requires you to handle errors as well as generate the appropriate errors with corresponding status codes.

When an error occurs, gRPC returns one of its error-status codes with an optional error message that provides more details of the error condition. The status object is composed of an integer code and a string message that are common to all gRPC implementations for different languages.

gRPC uses a set of well-defined gRPC-specific status codes. This includes status codes such as the following:

OK

Successful status; not an error.

CANCELLED

The operation was canceled, typically by the caller.

DEADLINE_EXCEEDED

The deadline expired before the operation could complete.

INVALID_ARGUMENT

The client specified an invalid argument.

Table 5-1 shows the available gRPC error codes and the description of each error code. The complete list of error codes can be found in the gRPC official documentation, or in the documentation for Go and Java.

The error model provided with gRPC out of the box is quite limited and independent of the underlying gRPC data format (where the most common format is protocol buffers). If you are using protocol buffers as your data format then you can leverage the richer error model the Google APIs provide under the google.rpc package. However, the error model is supported only in the C++, Go, Java, Python, and Ruby libraries, so be mindful of this if you plan to use other languages than these.

Let’s look at how these concepts can be used in a real-world gRPC error-handling use case. In our order management use case, suppose that we need to handle a request with invalid order IDs in the AddOrder remote method. As shown in Example 5-7, suppose that if the given order ID equals -1 then you need to generate an error and return it to the consumer.

if orderReq.Id == "-1" { 
    log.Printf("Order ID is invalid! -> Received Order ID %s",
        orderReq.Id)

    errorStatus := status.New(codes.InvalidArgument,
        "Invalid information received") 
    ds, err := errorStatus.WithDetails( 
        &epb.BadRequest_FieldViolation{
            Field:"ID",
            Description: fmt.Sprintf(
                "Order ID received is not valid %s : %s",
                orderReq.Id, orderReq.Description),
        },
    )
    if err != nil {
        return nil, errorStatus.Err()
    }

    return nil, ds.Err() 
    }
    ...

Invalid request, needs to generate an error and send it back to the client.

Create a new error status with error code InvalidArgument.

Include any error details with an error type BadRequest_FieldViolation from google.golang.org/genproto/googleapis/rpc/errdetails.

Returning the generated error.

You can simply create an error status from grpc.status packages with the required error code and details. In the example here we have used status.New(codes.InvalidArgument, "Invalid information received"). You just need to send this error back to the client with return nil, errorStatus.Err(). However, to include a richer error model, you can use Google API’s google.rpc package. In this example, we have set an error detail with a specific error type from google.golang.org/genproto/googleapis/rpc/errdetails.

For error handling on the client side, you simply process the error returned as part of your RPC invocation. For example, in Example 5-8, you can find the Go implementation of the client application of this order management use case. Here we invoked the AddOrder method and assigned the returned error to the addOrderError variable. So, the next step is to inspect the results of addOrderError and gracefully handle the error. For that, you can obtain the error code and specific error type that we have set from the server side.

order1 := pb.Order{Id: "-1",
	Items:[]string{"iPhone XS", "Mac Book Pro"},
	Destination:"San Jose, CA", Price:2300.00} 
res, addOrderError := client.AddOrder(ctx, &order1) 


if addOrderError != nil {
	errorCode := status.Code(addOrderError) 
	if errorCode == codes.InvalidArgument { 
		log.Printf("Invalid Argument Error : %s", errorCode)
		errorStatus := status.Convert(addOrderError) 
		for _, d := range errorStatus.Details() {
			switch info := d.(type) {
			case *epb.BadRequest_FieldViolation: 
				log.Printf("Request Field Invalid: %s", info)
			default:
				log.Printf("Unexpected error type: %s", info)
			}
		}
	} else {
		log.Printf("Unhandled error : %s ", errorCode)
	}
} else {
	log.Print("AddOrder Response -> ", res.Value)
}

This is an invalid order.

Invoke the AddOrder remote method and assign the error to addOrderError.

Obtain the error code using the grpc/status package.

Check for InvalidArgument error code.

Obtain the error status from the error.

Check for BadRequest_FieldViolation error type.

It’s always good practice to use the appropriate gRPC error codes and a richer error model whenever possible for your gRPC applications. gRPC error status and details are normally sent via the trailer headers at the transport protocol level.

Now let’s look at multiplexing, a service-hosting mechanism on the same gRPC server runtime.

Multiplexing

In terms of gRPC services and client applications, we’ve seen so far a given gRPC server with a gRPC service registered on it and a gRPC client connection being used by a single client stub only. However, gRPC allows you to run multiple gRPC services on the same gRPC server (see Figure 5-4), as well as use the same gRPC client connection for multiple gRPC client stubs. This capability is known as multiplexing.

Figure 5-4. Multiplexing multiple gRPC services in the same server application

For example, in our OrderManagement service example, suppose that you want to run another service that is required for order-management purposes on the same gRPC server, so that the client application can reuse the same connection to invoke both the services as required. Then you can register both services on the same gRPC server by using their respective server register functions (i.e., ordermgt_pb.RegisterOrderManagementServer and hello_pb.RegisterGreeterServer). Using this method, you can register one or more gRPC services on the same gRPC server (as shown in Example 5-9).

func main() {
	initSampleData()
	lis, err := net.Listen("tcp", port)
	if err != nil {
		log.Fatalf("failed to listen: %v", err)
	}
	grpcServer := grpc.NewServer() 

	// Register Order Management service on gRPC orderMgtServer
	ordermgt_pb.RegisterOrderManagementServer(grpcServer, &orderMgtServer{}) 

	// Register Greeter Service on gRPC orderMgtServer
	hello_pb.RegisterGreeterServer(grpcServer, &helloServer{}) 

      ...
}

Creating the gRPC server.

Registering the OrderManagement service with the gRPC server.

Registering the Hello service with the same gRPC server.

Similarly, from the client side you can share the same gRPC connection between two gRPC client stubs.

As shown in Example 5-10, since both gRPC services are running in the same gRPC server, you can create a gRPC connection and use it when creating the gRPC client instance for different services.

// Setting up a connection to the server.
conn, err := grpc.Dial(address, grpc.WithInsecure()) 
...

orderManagementClient := pb.NewOrderManagementClient(conn) 

...

// Add Order RPC
	...
res, addErr := orderManagementClient.AddOrder(ctx, &order1)

...


helloClient := hwpb.NewGreeterClient(conn) 

	...
	// Say hello RPC
helloResponse, err := helloClient.SayHello(hwcCtx,
	&hwpb.HelloRequest{Name: "gRPC Up and Running!"})
...

Running multiple services or using the same connection between multiple stubs is a design choice that is independent of gRPC concepts. In most everyday use cases such as microservices, it is quite common to not share the same gRPC server instance between two services.

In the next section, we’ll talk about how to exchange data that is not part of RPC parameters and responses between client and service applications.

Metadata

gRPC applications usually share information via RPC calls between gRPC services and consumers. In most cases, information directly related to the service’s business logic and consumer is part of the remote method invocation arguments. However, in certain conditions, you may want to share information about the RPC calls that are not related to the business context of the RPC, so they shouldn’t be part of the RPC arguments. In such cases, you can use gRPC metadata that you can send or receive from either the gRPC service or the gRPC client. As illustrated in Figure 5-5, the metadata that you create on either the client or server side can be exchanged between the client and server applications using gRPC headers. Metadata is structured in the form of a list of key(string)/value pairs.

One of the most common usages of metadata is to exchange security headers between gRPC applications. Similarly, you can use it to exchange any such information between gRPC applications. Often gRPC metadata APIs are heavily used inside the interceptors that we develop. In the next section, we’ll explore how gRPC supports sending metadata between the client and server.

Figure 5-5. Exchanging gRPC metadata between client and server applications

// Metadata Creation : option I
md := metadata.New(map[string]string{"key1": "val1", "key2": "val2"})

// Metadata Creation : option II
md := metadata.Pairs(
    "key1", "val1",
    "key1", "val1-2", // "key1" will have map value []string{"val1", "val1-2"}
    "key2", "val2",
)

func (s *server) AddOrder(ctx context.Context, orderReq *pb.Order)
    (*wrappers.StringValue, error) {

md, metadataAvailable := metadata.FromIncomingContext(ctx)
// read the required metadata from the ‘md’ metadata map.

md := metadata.Pairs(
	"timestamp", time.Now().Format(time.StampNano),
	"kn", "vn",
) 
mdCtx := metadata.NewOutgoingContext(context.Background(), md) 

ctxA := metadata.AppendToOutgoingContext(mdCtx,
      "k1", "v1", "k1", "v2", "k2", "v3") 

// make unary RPC
response, err := client.SomeRPC(ctxA, someRequest) 

// or make streaming RPC
stream, err := client.SomeStreamingRPC(ctxA) 

var header, trailer metadata.MD 

// ***** Unary RPC *****

r, err := client.SomeRPC( 
    ctx,
    someRequest,
    grpc.Header(&header),
    grpc.Trailer(&trailer),
)

// process header and trailer map here.

// ***** Streaming RPC *****

stream, err := client.SomeStreamingRPC(ctx)

// retrieve header
header, err := stream.Header() 

// retrieve trailer
trailer := stream.Trailer() 

// process header and trailer map here.

func (s *server) SomeRPC(ctx context.Context,
    in *pb.someRequest) (*pb.someResponse, error) { 
    md, ok := metadata.FromIncomingContext(ctx) 
    // do something with metadata
}

func (s *server) SomeStreamingRPC(
    stream pb.Service_SomeStreamingRPCServer) error { 
    md, ok := metadata.FromIncomingContext(stream.Context()) 
    // do something with metadata
}

func (s *server) SomeRPC(ctx context.Context,
    in *pb.someRequest) (*pb.someResponse, error) {
    // create and send header
    header := metadata.Pairs("header-key", "val")
    grpc.SendHeader(ctx, header) 
    // create and set trailer
    trailer := metadata.Pairs("trailer-key", "val")
    grpc.SetTrailer(ctx, trailer) 
}

func (s *server) SomeStreamingRPC(stream pb.Service_SomeStreamingRPCServer) error {
    // create and send header
    header := metadata.Pairs("header-key", "val")
    stream.SendHeader(header) 
    // create and set trailer
    trailer := metadata.Pairs("trailer-key", "val")    stream.SetTrailer(trailer) 
}

type exampleResolverBuilder struct{} 

func (*exampleResolverBuilder) Build(target resolver.Target,
	cc resolver.ClientConn,
	opts resolver.BuildOption) (resolver.Resolver, error) {

	r := &exampleResolver{ 
		target: target,
		cc:     cc,
		addrsStore: map[string][]string{
           exampleServiceName: addrs, 
		},
	}
	r.start()
	return r, nil
}
func (*exampleResolverBuilder) Scheme() string { return exampleScheme } 

type exampleResolver struct { 
	target     resolver.Target
	cc         resolver.ClientConn
	addrsStore map[string][]string
}

func (r *exampleResolver) start() {
	addrStrs := r.addrsStore[r.target.Endpoint]
	addrs := make([]resolver.Address, len(addrStrs))
	for i, s := range addrStrs {
		addrs[i] = resolver.Address{Addr: s}
	}
	r.cc.UpdateState(resolver.State{Addresses: addrs})
}
func (*exampleResolver) ResolveNow(o resolver.ResolveNowOption) {}
func (*exampleResolver) Close()                                 {}

func init() {
	resolver.Register(&exampleResolverBuilder{})
}

Load Balancing

Often when you develop production-ready gRPC applications, you need to make sure that your application can cater to high availability and scalability needs. Therefore, you always run more than one gRPC server in production. So, distributing RPC calls between these services needs to be taken care of by some entity. That’s where load balancing comes into play. Two main load-balancing mechanisms are commonly used in gRPC: a load-balancer (LB) proxy and client-side load balancing. Let’s start by discussing the LB proxy.

Load-Balancer Proxy

In proxy load balancing (Figure 5-6), the client issues RPCs to the LB proxy. Then the LB proxy distributes the RPC call to one of the available backend gRPC servers that implements the actual logic for serving the call. The LB proxy keeps track of load on each backend server and offers a different load-balancing algorithm for distributing the load among the backend services.

Figure 5-6. Client application invokes a load balancer that fronts multiple gRPC services

The topology of the backend services is not transparent to the gRPC clients, and they are only aware of the load balancer’s endpoint. Therefore, on the client side, you don’t need to make any changes to cater to a load-balancing use case, apart from using the load balancer’s endpoint as the destination for all your gRPC connections. The backend services can report the load status back to the load balancer so that it can use that information for the load-balancing logic.

In theory, you can select any load balancer that supports HTTP/2 as the LB proxy for your gRPC applications. However, it must have full HTTP/2 support. Thus it’s always a good idea to specifically choose load balancers that explicitly offer gRPC support. For instance, you can use load-balancing solutions such as Nginx, Envoy proxy, etc., as the LB proxy for your gRPC applications.

If you don’t use a gRPC load balancer, then you can implement the load-balancing logic as part of the client applications you write. Let’s look more closely at client-side load balancing.

Client-Side Load Balancing

Rather than having an intermediate proxy layer for load balancing, you can implement the load-balancing logic at the gRPC client level. In this method, the client is aware of multiple backend gRPC servers and chooses one to use for each RPC. As illustrated in Figure 5-7, the load-balancing logic may be entirely developed as part of the client application (also known as thick client) or it can be implemented in a dedicated server known as lookaside load balancer. Then the client can query it to obtain the best gRPC server to connect to. The client directly connects to the gRPC server address obtained by the lookaside load balancer.

Figure 5-7. Client-side load balancing

To understand how you can implement client-side load balancing, let’s look at an example of a thick client implemented using Go. In this use case, suppose we have two backend gRPC services running an echo server on :50051 and :50052. These gRPC services will include the serving address of the server as part of the RPC response. So we can consider these two servers as two members of an echo gRPC service cluster. Now, suppose we want to build a gRPC client application that uses the round-robin (executed in turn against every other) algorithm when selecting the gRPC server endpoint and another client that uses the first endpoint of the server endpoint list. Example 5-16 shows the thick client load-balancing implementation. Here you can observe that the client is dialing example:///lb.example.grpc.io. So, we are using the example scheme name and lb.example.grpc.io as the server name. Based on this scheme, it will look for a name resolver to discover the absolute value for the backend service address. Based on the list of values the name resolver returns, gRPC runs different load-balancing algorithms against those servers. The behavior is configured with grpc.WithBalancerName("round_robin").

Example 5-16. Client-side load balancing with a thick client

pickfirstConn, err := grpc.Dial(
		fmt.Sprintf("%s:///%s",
        // 	exampleScheme      = "example"
        //	exampleServiceName = "lb.example.grpc.io"
        exampleScheme, exampleServiceName), 
        // "pick_first" is the default option. 
		grpc.WithBalancerName("pick_first"),

		grpc.WithInsecure(),)
if err != nil {
    log.Fatalf("did not connect: %v", err)
}
defer pickfirstConn.Close()

log.Println("==== Calling helloworld.Greeter/SayHello " +
	"with pick_first ====")
makeRPCs(pickfirstConn, 10)

// Make another ClientConn with round_robin policy.
roundrobinConn, err := grpc.Dial(
    fmt.Sprintf("%s:///%s", exampleScheme, exampleServiceName),
    // "example:///lb.example.grpc.io"
    grpc.WithBalancerName("round_robin"), 
    grpc.WithInsecure(),
)
if err != nil {
    log.Fatalf("did not connect: %v", err)
}
defer roundrobinConn.Close()

log.Println("==== Calling helloworld.Greeter/SayHello " +
	"with round_robin ====")
makeRPCs(roundrobinConn, 10)

Creating a gRPC connection with a scheme and the service name. The scheme is resolved from a scheme resolver, which is part of the client application.

Specifying a load-balancing algorithm that picks the first server on the server endpoint list.

Using the round-robin load-balancing algorithm.

There are two load-balancing policies supported in gRPC by default: pick_first and round_robin. pick_first tries to connect to the first address, uses it for all RPCs if it connects, or tries the next address if it fails. round_robin connects to all the addresses it sees and sends an RPC to each backend one at a time in order.

In the client-side load-balancing scenario in Example 5-16, we have a name resolver to resolve scheme example, which contains the logic of discovering the actual values of the endpoint URLs. Now let’s talk about compression, another commonly used feature of gRPC, for sending large amounts of content over RPC.

Summary

Building production-ready, real-world gRPC applications often requires you to include various capabilities besides defining the service interface, generating the server and client code, and implementing the business logic. As you saw in this chapter, gRPC offers a wide range of capabilities that you will need when building gRPC applications, including interceptors, deadlines, cancellations, and error handling.

However, we haven’t yet discussed how to secure gRPC applications and how to consume them. So, in the next chapter we’ll cover this topic in detail.