Testing a gRPC Server

gRPC service testing is often done using a gRPC client application as part of the test cases. The server-side testing consists of starting a gRPC server with the required gRPC service and then connecting to the server using the client application where you implement your test cases. Let’s take a look at a sample test case written for the Go implementation of our ProductInfo service. In Go, the implementation of the gRPC test case should be implemented as a generic test case of Go using the testing package (see Example 7-1).

func TestServer_AddProduct(t *testing.T) { 
	grpcServer := initGRPCServerHTTP2() 
	conn, err := grpc.Dial(address, grpc.WithInsecure()) 
	if err != nil {

           grpcServer.Stop()
           t.Fatalf("did not connect: %v", err)
	}
	defer conn.Close()
	c := pb.NewProductInfoClient(conn)

	name := "Sumsung S10"
	description := "Samsung Galaxy S10 is the latest smart phone, launched in
	February 2019"
	price := float32(700.0)
	ctx, cancel := context.WithTimeout(context.Background(), time.Second)
	defer cancel()
	r, err := c.AddProduct(ctx, &pb.Product{Name: name,
	                                Description: description, Price: price}) 
	if err != nil { 
		t.Fatalf("Could not add product: %v", err)
	}

	if r.Value == "" {
		t.Errorf("Invalid Product ID %s", r.Value)
	}
	log.Printf("Res %s", r.Value)
      grpcServer.Stop()
}

Conventional test that starts a gRPC server and client to test the service with RPC.

Starting a conventional gRPC server running on HTTP/2.

Connecting to the server application.

Sends RPC for AddProduct method.

Verification of the response message.

As gRPC test cases are based on standard language test cases, the way in which you execute them will not be different from a standard test case. One special thing about the server-side gRPC tests is that they require the server application to open up a port the client application connects to. If you prefer not to do this, or your testing environment doesn’t allow it, you can use a library to help avoid starting up a service with a real port number. In Go, you can use the bufconn package, which provides a net.Conn implemented by a buffer and related dialing and listening functionality. You can find the full code sample in the source code repository for this chapter. If you are using Java you can use a test framework such as JUnit and follow the exact same procedure to write a server-side gRPC test. However, if you prefer to write the test case without starting a gRPC server instance, then you can use the gRPC in-process server of the Java implementation. You can find a complete Java code example for this in the code repository of this book.

It is also possible to unit test the business logic of the remote functions that you develop without going through the RPC network layer. You can instead directly test the functions by invoking them without using a gRPC client.

With this, we have learned how to write tests for gRPC services. Now let’s talk about how to test your gRPC client applications.

Testing a gRPC Client

When we are developing tests for a gRPC client, one of the possible approaches to testing would be to start a gRPC server and implement a mock service. However, this won’t be a very straightforward task as it will have the overhead of opening a port and connecting to a server. Therefore, to test client-side logic without the overhead of connecting to a real server, you can use a mocking framework. Mocking of the gRPC server side enables developers to write lightweight unit tests to check functionalities on the client side without invoking RPC calls to a server.

If you are developing a gRPC client application with Go, you can use Gomock to mock the client interface (using the generated code) and programmatically set its methods to expect and return predetermined values. Using Gomock, you can generate mock interfaces for the gRPC client application using:

mockgen github.com/grpc-up-and-running/samples/ch07/grpc-docker/go/proto-gen 
ProductInfoClient > mock_prodinfo/prodinfo_mock.go

Here, we’ve specified ProductInfoClient as the interface to be mocked. Then the test code you write can import the package generated by mockgen along with the gomock package to write unit tests around client-side logic. As shown in Example 7-2, you can create a mock object to expect a call to its method and return a response.

func TestAddProduct(t *testing.T) {
	ctrl := gomock.NewController(t)
	defer ctrl.Finish()
	mocklProdInfoClient := NewMockProductInfoClient(ctrl) 
     ...
	req := &pb.Product{Name: name, Description: description, Price: price}

	mocklProdInfoClient. 
	 EXPECT().AddProduct(gomock.Any(), &rpcMsg{msg: req},). 
	 Return(&wrapper.StringValue{Value: "ABC123" + name}, nil) 

	testAddProduct(t, mocklProdInfoClient) 
}

func testAddProduct(t *testing.T, client pb.ProductInfoClient) {
	ctx, cancel := context.WithTimeout(context.Background(), time.Second)
	defer cancel()
	...

	r, err := client.AddProduct(ctx, &pb.Product{Name: name,
    Description: description, Price: price})

	// test and verify response.
}

Creating a mock object to expect calls to remote methods.

Programming the mock object.

Expect a call to the AddProduct method.

Return a mock value for product ID.

Call the actual test method that invokes the remote method of the client stub.

If you are using Java, you can test the client application using Mockito and the in-process server implementation for the Java implementation of gRPC. You can refer to the source code repository for more details of these samples. Once you have the required server- and client-side testing in place you can integrate them with the continuous integration tools that you use.

It is important to keep in mind that mocking gRPC servers will not give you the exact same behavior as with a real gRPC server. So certain capabilities may not be able to be verified via test unless you re-implement all the error logic present in gRPC servers. In practice, you can verify a selected set of capabilities via mocking and the rest needs to be verified against the actual gRPC server implementation. Now let’s look at how you can do load testing and benchmarking of your gRPC applications.

Load Testing

It is difficult to conduct load testing and benchmarking for gRPC applications using conventional tools, as these applications are more or less bound to specific protocols such as HTTP. Therefore, for gRPC we need tailor-made load-testing tools that can load test the gRPC server by generating a virtual load of RPCs to the server.

ghz is such a load-testing tool; it is implemented as a command-line utility using Go. It can be used for testing and debugging services locally, and also in automated continuous integration environments for performance regression testing. For example, using ghz you can run a load test with the following command:

ghz --insecure 
  --proto ./greeter.proto 
  --call helloworld.Greeter.SayHello 
  -d '{"name":"Joe"}'
  -n 2000 
  -c 20 

  0.0.0.0:50051

Here we invoke a SayHello remote method of the Greeter service insecurely. We can specify the total number of requests (-n 2000) and concurrency (20 threads). The results can also be generated in various output formats.

Once you have the required server- and client-side testing in place, you can integrate them with the continuous integration tools that you use.

Continuous Integration

If you are new to continuous integration (CI), it is a development practice that requires developers to frequently integrate code into a shared repository. During each check-in the code is then verified by an automated build, allowing teams to detect problems early. When it comes to gRPC applications, often the server- and client-side applications are independent and may be built with disparate technologies. So, as part of the CI process, you will have to verify the gRPC client- or server-side code using the unit and integration testing techniques that we learned in the previous section. Then based on the language that you use to build the gRPC application, you can integrate the testing (e.g., Go testing or Java JUnit) of those applications with the CI tool of your choice. For instance, if you have written tests using Go, then you can easily integrate your Go tests with tools such as Jenkins, TravisCI, Spinnaker, etc.

Once you establish a testing and CI procedure for your gRPC application, the next thing that you need to look into is the deployment of your gRPC applications.

Deploying on Docker

Docker is an open platform for developing, shipping, and running applications. Using Docker, you can separate your applications from your infrastructure. It offers the ability to package and run an application in an isolated environment called a container so that you can run multiple containers on the same host. Containers are much more lightweight than conventional VMs and run directly within the host machine’s kernel.

Let’s look at some examples of deploying a gRPC application as a Docker container.

Once you develop a gRPC server application, you can create a Docker container for it. Example 7-3 shows a Dockerfile of a Go-based gRPC server. There are many gRPC-specific constructs in the Dockerfile. In this example, we have used a multistage Docker build where we build the application in stage 1, and then run the application in stage 2 as a much more lightweight runtime. The generated server-side code is also added into the container prior to building the application.

# Multistage build

# Build stage I: 
FROM golang AS build
ENV location /go/src/github.com/grpc-up-and-running/samples/ch07/grpc-docker/go
WORKDIR ${location}/server

ADD ./server ${location}/server
ADD ./proto-gen ${location}/proto-gen

RUN go get -d ./... 
RUN go install ./... 

RUN CGO_ENABLED=0 go build -o /bin/grpc-productinfo-server 

# Build stage II: 
FROM scratch
COPY --from=build /bin/grpc-productinfo-server /bin/grpc-productinfo-server 

ENTRYPOINT ["/bin/grpc-productinfo-server"]
EXPOSE 50051

Only the Go language and Alpine Linux is needed to build the program.

Download all the dependencies.

Install all the packages.

Building the server application.

Go binaries are self-contained executables.

Copy the binary that we built in the previous stage to the new location.

Once you create the Dockerfile you can build the Docker image using:

docker image build -t grpc-productinfo-server -f server/Dockerfile

The gRPC client application can be created using the same approach. One exception here is that, since we are running our server application on Docker, the hostname and port that the client application uses to connect to gRPC is now different.

When we run both the server and client gRPC applications on Docker, they need to communicate with each other and the outside world via the host machine. So there has to be a layer of networking involved. Docker supports different types of networks, each fit for certain use cases. So, when we run the server and client Docker containers, we can specify a common network so that the client application can discover the location of the server application based on the hostname. This means that the client application code has to change so that it connects to the hostname of the server. For example, our Go gRPC application must be modified to call the service hostname instead of localhost:

conn, err := grpc.Dial("productinfo:50051", grpc.WithInsecure())

You may read the hostname from the environment rather than hardcoding it in your client application. Once you are done with the changes to the client application, you need to rebuild the Docker image and then run both the server and client images as shown here:

docker run -it --network=my-net --name=productinfo 
    --hostname=productinfo
    -p 50051:50051  grpc-productinfo-server 

docker run -it --network=my-net 
    --hostname=client grpc-productinfo-client 

Running the gRPC server with hostname productinfo, port 50051 on Docker network my-net.

Running the gRPC client on Docker network my-net.

When starting Docker containers, you can specify a Docker network that a given container runs on. If the service shares the same network, then the client application can discover the actual address of the host service using the hostname provided along with the docker run command.

When the number of containers you run is small and their interactions are relatively simple, then you can possibly build your solution entirely on Docker. However, most real-world scenarios require the management of multiple containers and their interactions. Building such solutions solely based on Docker is quite tedious. That’s where a container orchestration platform comes into the picture.

Deploying on Kubernetes

Kubernetes is an open source platform for automating deployment, scaling, and management of containerized applications. When you run a containerized gRPC application using Docker, there’s no scalability or high-availability guarantee provided out of the box. You need to build those things outside the Docker containers. Kubernetes provides a wide range of such capabilities, so that you can offload most container-management and orchestration tasks to the underlying Kubernetes platform.

Let’s look at how your gRPC server application can be deployed into Kubernetes.

apiVersion: apps/v1
kind: Deployment 
metadata:
  name: grpc-productinfo-server 
spec:
  replicas: 1 
  selector:
    matchLabels:
      app: grpc-productinfo-server
  template:
    metadata:
      labels:
        app: grpc-productinfo-server
    spec:
      containers:
      - name: grpc-productinfo-server 
        image: kasunindrasiri/grpc-productinfo-server 
        resources:
          limits:
            memory: "128Mi"
            cpu: "500m"
        ports:
        - containerPort: 50051
          name: grpc

apiVersion: v1
kind: Service 
metadata:
  name: productinfo 
spec:
  selector:
    app: grpc-productinfo-server 
  ports:
  - port: 50051 
    targetPort: 50051
    name: grpc
  type: NodePort

apiVersion: batch/v1
kind: Job 
metadata:
  name: grpc-productinfo-client 
spec:
  completions: 1 
  parallelism: 1 
  template:
    spec:
      containers:
      - name: grpc-productinfo-client 
        image: kasunindrasiri/grpc-productinfo-client 
      restartPolicy: Never
  backoffLimit: 4

apiVersion: extensions/v1beta1
kind: Ingress 
metadata:
  annotations: 
    kubernetes.io/ingress.class: "nginx"
    nginx.ingress.kubernetes.io/ssl-redirect: "false"
    nginx.ingress.kubernetes.io/backend-protocol: "GRPC"
  name: grpc-prodinfo-ingress 
spec:
  rules:
  - host: productinfo 
    http:
      paths:
      - backend:
          serviceName: productinfo 
          servicePort: grpc 

Metrics

Metrics are a numeric representation of data measured over intervals of time. When talking about metrics, there are two types of data we can collect. One is system-level metrics like CPU usage, memory usage, etc. The other one is application-level metrics like inbound request rate, request error rate, etc.

System-level metrics are normally captured when the application is running. These days, there are lots of tools to capture those metrics, and they’re usually captured by the DevOps team. But application-level metrics differ between applications. So when designing a new application, it is the task of an application developer to decide what kind of application-level metrics need to be captured to get an understanding of the behavior of a system. In this section, we are going to focus on how to enable application-level metrics in our applications.

package main

import (
  "errors"
  "log"
  "net"
  "net/http"

  pb "productinfo/server/ecommerce"
  "google.golang.org/grpc"
  "go.opencensus.io/plugin/ocgrpc" 
  "go.opencensus.io/stats/view"
  "go.opencensus.io/zpages"
  "go.opencensus.io/examples/exporter"
)

const (
  port = ":50051"
)

// server is used to implement ecommerce/product_info.
type server struct {
  productMap map[string]*pb.Product
}


func main() {

  go func() { 
     mux := http.NewServeMux()
     zpages.Handle(mux, "/debug")
     log.Fatal(http.ListenAndServe("127.0.0.1:8081", mux))
  }()

   view.RegisterExporter(&exporter.PrintExporter{}) 

  if err := view.Register(ocgrpc.DefaultServerViews...); err != nil { 
     log.Fatal(err)
  }

  grpcServer := grpc.NewServer(grpc.StatsHandler(&ocgrpc.ServerHandler{})) 
  pb.RegisterProductInfoServer(grpcServer, &server{}) 

  lis, err := net.Listen("tcp", port)
  if err != nil {
     log.Fatalf("Failed to listen: %v", err)
  }

  if err := grpcServer.Serve(lis); err != nil { 
     log.Fatalf("failed to serve: %v", err)
  }
}

package main

import (
  "context"
  "log"
  "time"

  pb "productinfo/server/ecommerce"
  "google.golang.org/grpc"
  "go.opencensus.io/plugin/ocgrpc" 
  "go.opencensus.io/stats/view"
  "go.opencensus.io/examples/exporter"
)

const (
  address = "localhost:50051"
)

func main() {
  view.RegisterExporter(&exporter.PrintExporter{}) 

   if err := view.Register(ocgrpc.DefaultClientViews...); err != nil { 
       log.Fatal(err)
   }

  conn, err := grpc.Dial(address, 
        grpc.WithStatsHandler(&ocgrpc.ClientHandler{}),
          grpc.WithInsecure(),
          )
  if err != nil {
     log.Fatalf("Can't connect: %v", err)
  }
  defer conn.Close() 

  c := pb.NewProductInfoClient(conn) 

  .... // Skip RPC method invocation.
}

package main

import (
  ...
  "github.com/grpc-ecosystem/go-grpc-prometheus" 
  "github.com/prometheus/client_golang/prometheus"
  "github.com/prometheus/client_golang/prometheus/promhttp"
)

var (
  reg = prometheus.NewRegistry() 

  grpcMetrics = grpc_prometheus.NewServerMetrics() 

   customMetricCounter = prometheus.NewCounterVec(prometheus.CounterOpts{
       Name: "product_mgt_server_handle_count",
       Help: "Total number of RPCs handled on the server.",
   }, []string{"name"}) 
)

func init() {
   reg.MustRegister(grpcMetrics, customMetricCounter) 
}

func main() {
  lis, err := net.Listen("tcp", port)
  if err != nil {
     log.Fatalf("failed to listen: %v", err)
  }

  httpServer := &http.Server{
      Handler: promhttp.HandlerFor(reg, promhttp.HandlerOpts{}),
        Addr:  fmt.Sprintf("0.0.0.0:%d", 9092)} 

  grpcServer := grpc.NewServer(
     grpc.UnaryInterceptor(grpcMetrics.UnaryServerInterceptor()), 
  )

  pb.RegisterProductInfoServer(grpcServer, &server{})
  grpcMetrics.InitializeMetrics(grpcServer) 

  // Start your http server for prometheus.
  go func() {
     if err := httpServer.ListenAndServe(); err != nil {
        log.Fatal("Unable to start a http server.")
     }
  }()

  if err := grpcServer.Serve(lis); err != nil {
     log.Fatalf("failed to serve: %v", err)
  }
}

// AddProduct implements ecommerce.AddProduct
func (s *server) AddProduct(ctx context.Context,
     in *pb.Product) (*wrapper.StringValue, error) {
     customMetricCounter.WithLabelValues(in.Name).Inc()
  ...
}

package main

import (
  ...
  "github.com/grpc-ecosystem/go-grpc-prometheus" 
  "github.com/prometheus/client_golang/prometheus"
  "github.com/prometheus/client_golang/prometheus/promhttp"
)

const (
  address = "localhost:50051"
)

func main() {
  reg := prometheus.NewRegistry() 
  grpcMetrics := grpc_prometheus.NewClientMetrics() 
  reg.MustRegister(grpcMetrics) 

  conn, err := grpc.Dial(address,
        grpc.WithUnaryInterceptor(grpcMetrics.UnaryClientInterceptor()), 
          grpc.WithInsecure(),
          )
  if err != nil {
     log.Fatalf("did not connect: %v", err)
  }
  defer conn.Close()

   // Create a HTTP server for prometheus.
   httpServer := &http.Server{
        Handler: promhttp.HandlerFor(reg, promhttp.HandlerOpts{}),
          Addr: fmt.Sprintf("0.0.0.0:%d", 9094)} 

   // Start your http server for prometheus.
   go func() {
       if err := httpServer.ListenAndServe(); err != nil {
           log.Fatal("Unable to start a http server.")
       }
   }()

  c := pb.NewProductInfoClient(conn)
  ...
}

Logs

Logs are immutable, time-stamped records of discrete events that happened over time. We, as application developers, normally dump data into logs to tell where and what the internal state of the system is at a given point. The benefit of logs is they are the easiest to generate and more granular than metrics. We can attach specific actions or a bunch of context to it like unique IDs, what we are going to do, stack traces, etc. The downside is that they are very expensive because we need to store and index them in a way that makes it easy to search and use them.

In gRPC applications, we can enable logging using interceptors. As we discussed in Chapter 5, we can attach a new logging interceptor on both the client side and server side and log request and response messages of each remote call.

Once you add logs in your gRPC application, they’ll print in either the console or logfile, depending on how you configure logging. How to configure logging depends on the logging framework you used.

We’ve now discussed two pillars of observability: metrics and logs. These are sufficient for understanding the performance and behavior of individual systems, but they aren’t sufficient to understand the lifetime of a request that traverses multiple systems. Distributed tracing is a technique that brings visibility of the lifetime of a request across several systems.

Tracing

A trace is a representation of a series of related events that constructs the end-to-end request flow through a distributed system. As we discussed in the section “Using gRPC for Microservices Communication”, in a real-world scenario we have multiple microservices serving different and specific business capabilities. Therefore, a request starting from the client is normally going through a number of services and different systems before the response going back to the client. All these intermediate events are part of the request flow. With tracing, we gain visibility into both the path traversed by a request as well as the structure of a request.

In tracing, a trace is a tree of spans, which are the primary building blocks of distributed tracing. The span contains the metadata about the task, the latency (the time spent to complete the task), and other related attributes of the task. A trace has its own ID called TraceID and it is a unique byte sequence. This traceID groups and distinguishes spans from each other. Let’s try to enable tracing in our gRPC application.

Like metrics, the OpenCensus library provides support to enable tracing in gRPC applications. We will use OpenCensus to enable tracing in our Product Management application. As we said earlier, we can plug any supported exporters to export tracing data to different backends. We will use Jaeger for the distributed tracing sample.

By default, tracing is enabled in gRPC Go. So it only requires registering an exporter to start collecting traces with gRPC Go integration. Let’s initiate a Jaeger exporter in both client and server applications. Example 7-13 illustrates how we can initiate the OpenCensus Jaeger exporter using the library.

package tracer

import (
  "log"

  "go.opencensus.io/trace" 
  "contrib.go.opencensus.io/exporter/jaeger"

)

func initTracing() {

   trace.ApplyConfig(trace.Config{DefaultSampler: trace.AlwaysSample()})
   agentEndpointURI := "localhost:6831"
   collectorEndpointURI := "http://localhost:14268/api/traces" 
    exporter, err := jaeger.NewExporter(jaeger.Options{
            CollectorEndpoint: collectorEndpointURI,
            AgentEndpoint: agentEndpointURI,
            ServiceName:    "product_info",

    })
    if err != nil {
       log.Fatal(err)
    }
    trace.RegisterExporter(exporter) 

}

Import the OpenTracing and Jaeger libraries.

Create the Jaeger exporter with the collector endpoint, service name, and agent endpoint.

Register the exporter with the OpenCensus tracer.

Once we register the exporter with the server, we can instrument the server by tracing. Example 7-14 illustrates how to instrument tracing in service method.

// GetProduct implements ecommerce.GetProduct
func (s *server) GetProduct(ctx context.Context, in *wrapper.StringValue) (
         *pb.Product, error) {
  ctx, span := trace.StartSpan(ctx, "ecommerce.GetProduct") 
  defer span.End() 
  value, exists := s.productMap[in.Value]
  if exists {
     return value, status.New(codes.OK, "").Err()
  }
  return nil, status.Errorf(codes.NotFound, "Product does not exist.", in.Value)
}

Start new span with span name and context.

Stop the span when everything is done.

Similar to the gRPC server, we can instrument the client by tracing as shown in Example 7-15.

package main

import (
  "context"
  "log"
  "time"

  pb "productinfo/client/ecommerce"
  "productinfo/client/tracer"
  "google.golang.org/grpc"
  "go.opencensus.io/plugin/ocgrpc" 
  "go.opencensus.io/trace"
  "contrib.go.opencensus.io/exporter/jaeger"

)

const (
  address = "localhost:50051"
)

func main() {
  tracer.initTracing() 

  conn, err := grpc.Dial(address, grpc.WithInsecure())
  if err != nil {
     log.Fatalf("did not connect: %v", err)
  }
  defer conn.Close()
  c := pb.NewProductInfoClient(conn)

  ctx, span := trace.StartSpan(context.Background(),
          "ecommerce.ProductInfoClient") 

  name := "Apple iphone 11"
  description := "Apple iphone 11 is the latest smartphone,
            launched in September 2019"
  price := float32(700.0)
  r, err := c.AddProduct(ctx, &pb.Product{Name: name,
      Description: description, Price: price}) 
  if err != nil {
     log.Fatalf("Could not add product: %v", err)
  }
  log.Printf("Product ID: %s added successfully", r.Value)

  product, err := c.GetProduct(ctx, &pb.ProductID{Value: r.Value}) 
  if err != nil {
    log.Fatalf("Could not get product: %v", err)
  }
  log.Printf("Product: ", product.String())
  span.End() 

}

Import the OpenTracing and Jaeger libraries.

Call the initTracing function and initialize the Jaeger exporter instance and register with trace.

Start new span with span name and context.

Stop the span when everything is done.

Invoke addProduct remote method by passing new product details.

Invoke getProduct remote method by passing productID.

Once we run the server and client, trace spans are published to the Jaeger agent for which a daemon process acts as a buffer to abstract out batch processing and routing from the clients. Once the Jaeger agent receives trace logs from the client, it forwards them to the collector. The collector processes the logs and stores them. From the Jaeger server, we can visualize tracing.

From that, we are going to conclude the discussion of observability. Logs, metrics, and traces serve their own unique purpose, and it’s better to have all three pillars enabled in your system to gain maximum visibility of the internal state.

Once you have a gRPC-based observable application running in production, you can keep watching its state and easily find out whenever there is an issue or system outage. When you diagnose an issue in the system, it is important to find the solution, test it, and deploy it to production as soon as possible. To accomplish that goal, you need to have good debugging and troubleshooting mechanisms. Let’s look into the details of these mechanisms for gRPC applications.

Enabling Extra Logging

We can enable extra logs and traces to diagnose the problem of your gRPC application. In the gRPC Go application, we can enable extra logs by setting the following environment variables:

GRPC_GO_LOG_VERBOSITY_LEVEL=99 
GRPC_GO_LOG_SEVERITY_LEVEL=info 

Verbosity means how many times any single info message should print every five minutes. The verbosity is set to 0 by default.

Sets log severity level to info. All the informational messages will be printed.

In the gRPC Java application, there are no environment variables to control the log level. We can turn on extra logs by providing a logging.properties file with log-level changes. Let’s say we want to troubleshoot transport-level frames in our application. We can create a new logging.properties file in our application and set the lower log level to a specific Java package (netty transport package) as follows:

handlers=java.util.logging.ConsoleHandler
io.grpc.netty.level=FINE
java.util.logging.ConsoleHandler.level=FINE
java.util.logging.ConsoleHandler.formatter=java.util.logging.SimpleFormatter

Then start up the Java binary with the JVM flag:

-Djava.util.logging.config.file=logging.properties

Once we set the lower log level in our application, all the logs in which the level is equal or higher than the configured log level will print in the console/logfile. We can gain valuable insight into the state of the system by reading the logs.

With that, we have covered most of what you should know when running a gRPC application in production.