Defining Messages

The message is the data structure that is exchanged between client and service. As you can see in Figure 2-1, our ProductInfo use case has two message types. One is the product information (Product), which is required when adding a new product to the system and is returned when retrieving a particular product. The other is a unique identification (ProductID) of the product, which is required when retrieving a particular product from the system and is returned when adding a new product:

ProductID

ProductID is a unique identifier of the product that can be a string value. We can either define our own message type that contains a string field or use the well-known message type google.protobuf.StringValue, provided by the protocol buffer library. In this example, we are going to define our own message type that contains a string field. The ProductID message type definition is shown in Example 2-1.

message ProductID {
   string value = 1;
}

Product

Product is a custom message type that represents the data that should exist in a product in our online retail application. It can have a set of fields that represent the data associated with each product. Suppose the Product message type has the following fields:

ID

Unique identifier of the product

Name

Product name

Description

Product description

Price

Product price

Then we can define our custom message type using a protocol buffer as shown in Example 2-2.

message Product {
   string id = 1;
   string name = 2;
   string description = 3;
   float price = 4;
}

Here the number assigned to each message field is used to uniquely identify the field in the message. So, we can’t use the same number in two different fields in the same message definition. We will further dive into the details of the message definition techniques of protocol buffers and explain why we need to provide a unique number for each field in Chapter 4. For now, you can think of it as a rule when defining a protocol buffer message.

Since we have completed defining message types for the ProductInfo service, we can move on to the service interface definition.

Defining Services

A service is a collection of remote methods that are exposed to a client. In our sample, the ProductInfo service has two remote methods: addProduct(product) and getProduct(productId). According to the protocol buffer rule, we can only have one input parameter in a remote method and it can return only one value. If we need to pass multiple values to the method like in the addProduct method, we need to define a message type and group all the values as we have done in the Product message type:

addProduct

Creates a new Product in the system. It requires the details of the product as input and returns the product identification number of the newly added product, if the action completed successfully. Example 2-3 shows the definition of the addProduct method.

rpc addProduct(Product) returns (google.protobuf.StringValue);

getProduct

Retrieves product information. It requires the ProductID as input and returns Product details if a particular product exists in the system. Example 2-4 shows the definition of the getProduct method.

rpc getProduct(google.protobuf.StringValue) returns (Product);

Combining all messages and services, we now have a complete protocol buffer definition for our ProductInfo use case, as shown in Example 2-5.

syntax = "proto3"; 
package ecommerce; 

service ProductInfo { 
    rpc addProduct(Product) returns (ProductID); 
    rpc getProduct(ProductID) returns (Product); 
}

message Product { 
    string id = 1; 
    string name = 2;
    string description = 3;
}

message ProductID { 
    string value = 1;
}

The service definition begins with specifying the protocol buffer version (proto3) that we use.

Package names are used to prevent name clashes between protocol message types and also will be used to generate code.

Definition of the service interface of the service.

Remote method to add a product that returns the product ID as the response.

Remote method to get a product based on the product ID.

Definition of the message format/type of Product.

Field (name-value pair) that holds the product ID with unique field numbers that are used to identify your fields in the message binary format.

Definition of the message format/type of ProductID.

In the protocol buffer definition, we can specify a package name (e.g., ecommerce), which helps to prevent naming conflicts between different projects. When we generate code for our services or clients using this service definition with a package, the same packages (unless we explicitly specify a different package for code generation) are created in the respective programming language (of course only if the language supports the notion of a package) with which our code is generated. We can also define package names with version numbers like ecommerce.v1 and ecommerce.v2. So future major changes to the API can coexist in the same codebase.

One other process that should be mentioned here is importing from another proto file. If we need to use the message types defined in other proto files, we can import them and our protocol buffer definition. For example, if we want to use the StringValue type (google.protobuf.StringValue) defined in the wrappers.proto file, we can import the google/protobuf/wrappers.proto file in our definition as follows:

syntax = "proto3";

import "google/protobuf/wrappers.proto";

package ecommerce;
...

Once you complete the specification of the service definition, you can proceed to the implementation of the gRPC service and the client.

Developing a Service

When you generate the service skeleton code, you will get the low-level code required to establish the gRPC communication, relevant message types, and interfaces. The task of service implementation is all about implementing the interfaces that are generated with the code generation step. Let’s start with implementing the Go service and then we will look at how to implement the same service in the Java language.

go mod init productinfo/service

└─ productinfo
           └─ service
                 ├── go.mod
                 ├ . . .
                 └── ecommerce
                         └── . . .

module productinfo/service

require (
  github.com/gofrs/uuid v3.2.0
  github.com/golang/protobuf v1.3.2
  github.com/google/uuid v1.1.1
  google.golang.org/grpc v1.24.0
)

go get -u google.golang.org/grpc

go get -u github.com/golang/protobuf/protoc-gen-go

protoc -I ecommerce  
  ecommerce/product_info.proto  
  --go_out=plugins=grpc:<module_dir_path>/ecommerce 

package main

import (
  "context"
  "errors"
  "log"

"github.com/gofrs/uuid"
pb "productinfo/service/ecommerce" 

)

// server is used to implement ecommerce/product_info.
type server struct{ 
  productMap map[string]*pb.Product
}

// AddProduct implements ecommerce.AddProduct
func (s *server) AddProduct(ctx context.Context,
                    in *pb.Product) (*pb.ProductID, error) { 
  out, err := uuid.NewV4()
  if err != nil {
     return nil, status.Errorf(codes.Internal,
         "Error while generating Product ID", err)
  }
  in.Id = out.String()
  if s.productMap == nil {
     s.productMap = make(map[string]*pb.Product)
  }
  s.productMap[in.Id] = in
  return &pb.ProductID{Value: in.Id}, status.New(codes.OK, "").Err()

}

// GetProduct implements ecommerce.GetProduct
func (s *server) GetProduct(ctx context.Context, in *pb.ProductID)
                              (*pb.Product, error) { 
  value, exists := s.productMap[in.Value]
  if exists {
    return value, status.New(codes.OK, "").Err()
  }
  return nil, status.Errorf(codes.NotFound, "Product does not exist.", in.Value)

}

package main

import (
  "log"
  "net"

  pb "productinfo/service/ecommerce" 
  "google.golang.org/grpc"
)

const (
  port = ":50051"
)

func main() {
  lis, err := net.Listen("tcp", port) 
  if err != nil {
     log.Fatalf("failed to listen: %v", err)
  }
  s := grpc.NewServer() 
  pb.RegisterProductInfoServer(s, &server{}) 

  log.Printf("Starting gRPC listener on port " + port)
  if err := s.Serve(lis); err != nil { 
     log.Fatalf("failed to serve: %v", err)
  }
}

 product-info-service

  ├── build.gradle
  ├ . . .
  └── src
      ├── main
      │   ├── java
      │   └── resources
      └── test
          ├── java
          └── resources

apply plugin: 'java'
apply plugin: 'com.google.protobuf'

repositories {
   mavenCentral()
}


def grpcVersion = '1.24.1' 

dependencies { 
   compile "io.grpc:grpc-netty:${grpcVersion}"
   compile "io.grpc:grpc-protobuf:${grpcVersion}"
   compile "io.grpc:grpc-stub:${grpcVersion}"
   compile 'com.google.protobuf:protobuf-java:3.9.2'
}

buildscript {
   repositories {
       mavenCentral()
   }
   dependencies { 

       classpath 'com.google.protobuf:protobuf-gradle-plugin:0.8.10'
   }
}

protobuf { 
   protoc {
       artifact = 'com.google.protobuf:protoc:3.9.2'
   }
   plugins {
       grpc {
           artifact = "io.grpc:protoc-gen-grpc-java:${grpcVersion}"
       }
   }
   generateProtoTasks {
       all()*.plugins {
           grpc {}
       }
   }
}

sourceSets { 
   main {
       java {
           srcDirs 'build/generated/source/proto/main/grpc'
           srcDirs 'build/generated/source/proto/main/java'
       }
   }
}

jar { 
   manifest {
       attributes "Main-Class": "ecommerce.ProductInfoServer"
   }
   from {
       configurations.compile.collect { it.isDirectory() ? it : zipTree(it) }
   }
}

apply plugin: 'application'

startScripts.enabled = false

$ ./gradle build

package ecommerce;

import io.grpc.Status;
import io.grpc.StatusException;

import java.util.HashMap;
import java.util.Map;
import java.util.UUID;

public class ProductInfoImpl extends ProductInfoGrpc.ProductInfoImplBase { 

   private Map productMap = new HashMap<String, ProductInfoOuterClass.Product>();

   @Override
  public void addProduct(
       ProductInfoOuterClass.Product request,
      io.grpc.stub.StreamObserver
           <ProductInfoOuterClass.ProductID> responseObserver ) { 
       UUID uuid = UUID.randomUUID();
       String randomUUIDString = uuid.toString();
       productMap.put(randomUUIDString, request);
       ProductInfoOuterClass.ProductID id =
           ProductInfoOuterClass.ProductID.newBuilder()
           .setValue(randomUUIDString).build();
       responseObserver.onNext(id); 
       responseObserver.onCompleted(); 
   }

   @Override
  public void getProduct(
       ProductInfoOuterClass.ProductID request,
       io.grpc.stub.StreamObserver
            <ProductInfoOuterClass.Product> responseObserver ) { 
       String id = request.getValue();
       if (productMap.containsKey(id)) {
           responseObserver.onNext(
                (ProductInfoOuterClass.Product) productMap.get(id)); 
           responseObserver.onCompleted(); 
       } else {
           responseObserver.onError(new StatusException(Status.NOT_FOUND)); 
       }
   }
}

package ecommerce;

import io.grpc.Server;
import io.grpc.ServerBuilder;

import java.io.IOException;

public class ProductInfoServer {

   public static void main(String[] args)
           throws IOException, InterruptedException {
       int port = 50051;
       Server server = ServerBuilder.forPort(port) 
               .addService(new ProductInfoImpl())
               .build()
               .start();
       System.out.println("Server started, listening on " + port);
       Runtime.getRuntime().addShutdownHook(new Thread(() -> { 
           System.err.println("Shutting down gRPC server since JVM is " +
                "shutting down");
           if (server != null) {
               server.shutdown();
           }
           System.err.println(“Server shut down");
       }));
       server.awaitTermination(); 
   }
}

Developing a gRPC Client

As we did with the gRPC service implementation, we can now discuss how to create an application to talk with the server. Let’s start off with the generation of the client-side stubs from the service definition. On top of the generated client stub, we can create a simple gRPC client to connect with our gRPC server and invoke the remote methods that it offers.

In this sample, we are going to write client applications in both the Java and Go languages. But you don’t need to create your server and client in the same language, or run them on the same platform. Since gRPC works across languages and platforms, you can create them in any supported language. Let’s discuss the Go implementation first. If you are interested in the Java implementation, you may skip the next section and go directly into the Java client.

package main

import (
  "context"
  "log"
  "time"

  pb "productinfo/client/ecommerce" 
  "google.golang.org/grpc"

)

const (
  address = "localhost:50051"
)

func main() {

  conn, err := grpc.Dial(address, grpc.WithInsecure()) 
  if err != nil {
     log.Fatalf("did not connect: %v", err)
  }
  defer conn.Close() 
  c := pb.NewProductInfoClient(conn) 

  name := "Apple iPhone 11"
  description := `Meet Apple iPhone 11. All-new dual-camera system with
              Ultra Wide and Night mode.`
  price := float32(1000.0)
  ctx, cancel := context.WithTimeout(context.Background(), time.Second) 
  defer cancel()
  r, err := c.AddProduct(ctx,
         &pb.Product{Name: name, Description: description, Price: price})  
  if err != nil {
     log.Fatalf("Could not add product: %v", err)
  }
  log.Printf("Product ID: %s added successfully", r.Value)

  product, err := c.GetProduct(ctx, &pb.ProductID{Value: r.Value}) 
  if err != nil {
    log.Fatalf("Could not get product: %v", err)
  }
  log.Printf("Product: ", product.String())

}

package ecommerce;

import io.grpc.ManagedChannel;
import io.grpc.ManagedChannelBuilder;

import java.util.logging.Logger;

/**
* gRPC client sample for productInfo service.
*/
public class ProductInfoClient {

   public static void main(String[] args) throws InterruptedException {
       ManagedChannel channel = ManagedChannelBuilder
           .forAddress("localhost", 50051) 
           .usePlaintext()
           .build();

       ProductInfoGrpc.ProductInfoBlockingStub stub =
              ProductInfoGrpc.newBlockingStub(channel); 

       ProductInfoOuterClass.ProductID productID = stub.addProduct(    
               ProductInfoOuterClass.Product.newBuilder()
                       .setName("Apple iPhone 11")
                       .setDescription("Meet Apple iPhone 11. " +
                            All-new dual-camera system with " +
                            "Ultra Wide and Night mode.");
                       .setPrice(1000.0f)
                       .build());
       System.out.println(productID.getValue());

       ProductInfoOuterClass.Product product = stub.getProduct(productID);  
       System.out.println(product.toString());
       channel.shutdown(); 
   }
}

Building a Go Server

When we implement a Go service, the final package structure in the workspace looks like the following:

└─ productinfo
           └─ service
                 ├─ go.mod
                 ├─ main.go
                 ├─ productinfo_service.go
                 └─ ecommerce
                    └── product_info.pb.go

We can build our service to generate a service binary (bin/server). In order to build, first go to the Go module root directory location (productinfo/service) and execute the following shell command:

$ go build -i -v -o bin/server

Once the build is successful, an executable file (bin/server) is created under the bin directory.

Next, let’s set up the Go client!

Building a Go Client

When we implement a Go client, the package structure in the workspace looks like:

└─ productinfo
           └─ client
                ├─ go.mod
                ├──main.go
                └─ ecommerce
                       └── product_info.pb.go

We can build the client code the same way we built the Go service using the following shell command:

$ go build -i -v -o bin/client

Once the build is successful, an executable file (bin/client) is created under the bin directory. The next step is to run the files!

Running a Go Server and Client

We’ve just built a client and a server. Let’s run them on separate terminals and make them talk to each other:

// Running Server
$ bin/server
2019/08/08 10:17:58 Starting gRPC listener on port :50051

// Running Client
$ bin/client
2019/08/08 11:20:01 Product ID: 5d0e7cdc-b9a0-11e9-93a4-6c96cfe0687d
added successfully
2019/08/08 11:20:01 Product: id:"5d0e7cdc-b9a0-11e9-93a4-6c96cfe0687d"
        name:"Apple iPhone 11"
        description:"Meet Apple iPhone 11. All-new dual-camera system with
        Ultra Wide and Night mode."
        price:1000

Next we’ll build a Java server.

Building a Java Server

Since we implement the Java service as a Gradle project, we can easily build the project using the following command:

$ gradle build

Once the build is successful, the executable JAR (server.jar) file is created under the build/libs directory.

Building a Java Client

Just as with a service, we can easily build the project using the following command:

$ gradle build

Once the build is successful, the executable JAR (client.jar) file is created under the build/libs directory.

Running a Java Server and Client

We’ve now built both a client and server in the Java language. Let’s run them:

$ java -jar build/libs/server.jar
INFO: Server started, listening on 50051

$ java -jar build/libs/client.jar
INFO: Product ID: a143af20-12e6-483e-a28f-15a38b757ea8 added successfully.
INFO: Product: name: "Apple iPhone 11"
description: "Meet Apple iPhone 11. All-new dual-camera system with
Ultra Wide and Night mode."
price: 1000.0

Now we have built and run our sample successfully on local machines. Once we successfully run the client and server, the client application first invokes the addProduct method with product details and receives the product identifier of the newly added product as the response. Then it retrieves the newly added product details by calling the getProduct method with the product identifier. As we mentioned earlier in this chapter, we don’t need to write the client in the same language to talk with the server. We can run a gRPC Java server and Go client and it will work without any issue.

That brings us to the end of the chapter!