The microservices architecture inherently has to deal with dozens to hundreds or thousands of services. When you operate at that scale, you need to have some governance processes in place. However, using a strict centralized governance process will hinder the autonomy of the microservices architecture. Therefore, we need to rethink a strategy for microservices governance.
In this chapter, we walk through the requirements for microservices governance and delve deep into a few key aspects of it, such as service registry and governance. The rest of the governance topics are covered in the upcoming chapters.
Why Microservices Governance?
Service Oriented Architecture (SOA) governance was one of the key driving forces behind the operational success of SOA and it provides cooperation and coordination between different entities in an organization (development teams, service consumers, etc.). Although it defines a comprehensive set of theoretical concepts as part of SOA governance, only a handful of concepts are being actively used in practice.
When we shift into a microservices architecture, most of the useful governance concepts are also being discarded. The governance concept in microservices is just interpreted1 as a decentralized process, which gives each team/entity the freedom to govern their own domain, in the way they prefer. Decentralized governance is applicable to the service development, deployment, and execution processes, but there’s a lot more to it than that. Let’s take a closer look at various aspects of microservices governance and see how we can implement them in practice.
Aspects of Microservices Governance
Microservices governance comprises various practices that are coordinated together to realize a real-world scenario. Most of these concepts are not new but ones that we have successfully used in the SOA governance. They are equally applicable under the microservices architecture.
Service Definition
Any microservice that we develop must have enough information to uniquely identify itself, its functionality, and how a consumer should consume it. Therefore, it must have a mechanism to specify the service definition and it should be readily available to the service consumers.
There are several technologies (discussed in Chapter 3, “Inter-service Communication”)—such as OpenAPI (Swagger), GrahQL schema, gRPC, and protocol buffer—that define service interfaces. These allow you to define the service identifiers, service interfaces (i.e., the available service functionalities), and service message models (schema or message format of the service requests and responses). Other service metadata, such as service ownership and service level agreements (SLAs), can also be part of the service definition.
Service definitions are often stored in a central repository, to which the consumers have access and the service owners can publish.
Service Registry and Discovery
The service registry is the place where you can store the service definitions, so that the service providers can make their services available and known to the consumers. Service consumers locate the services that they want to invoke using the service registry. The service metadata, which are part of the service definition (such as service URL, message models, supported functionalities, etc.), can be retrieved via the service registry.
The service registry defines an API to publish and access service definitions. When a service is created or updated, the service owners should publish the service definition to the service registry and the consumers can discover services during the runtime using a service discovery mechanism. Later in the chapter, we take a closer look at the most commonly used service registries and discovery mechanisms in the microservices architecture.
Service Lifecycle Management
Microservices have different lifecycle stages, which include planning, designing, implementing, deploying, maintaining, and decommissioning. Given the decentralized nature of microservices, often these tasks are owned by the team who owns each microservice. In most practical scenarios, it is common to have uniform lifecycle stages for your microservices irrespective of the business scope and the technologies that you use to develop them. The service lifecycle management techniques are centrally applied to your microservices architecture. This includes the deployment lifecycle management, how to version your services, etc. Most of these capabilities are implemented as part of API management or control planes in the service mesh. We discuss them in detail in Chapter 9, “Service Mesh,” and Chapter 10, “APIs, Events, and Streams”.
Quality of Service (QoS)
There are several Quality of Services (QoS) aspects that you need to consider prior to exposing a service to your consumers. The service may be exposed as a secured service, which leverages various security protocols and standards (starting from transport layer security, access tokens, etc.). Also you may control access to the service using rate-limiting and throttling. Caching and incorporating various hooks into monitoring and monetization are some other prominent QoS features. Most of these requirements are directly related to the governance of your microservices and often centrally controlled.
We cover the microservices security fundamentals and use cases in detail in Chapter 11, “Microservice Security Fundamentals,” and Chapter 12, “Securing Microservices”.
Service Observability
When your application interacts with multiple microservices, it is vital to have metrics, tracing, logging, visualization, and alerting capabilities for all your services, so that you will have a clear picture of their interactions and be able to troubleshoot when something goes wrong. All these requirements are consolidated under one concept, called observability.
With a microservices architecture, it is likely to have hundreds or thousands of services communicating with each other. The ability to get service metrics, trace messages and service interactions, get service logs, understanding runtime dependencies of services, troubleshoot in the event of a failure, and set alerting for anomalies can all be considered under the umbrella of observability.
Most of the observability tools operate as a centralized entity in which all services can push the required data, which are useful for metrics, tracing, logging, visualization, etc. It’s up to the observability tools to analyze the data and process them so that they can derive the required observability related information. We discuss all the technologies and tools that are used for microservices observability in Chapter 13, “Observability”.
With that, we have discussed most of the key aspects of microservices governance. Now it’s time to take a look at how those concepts can be implemented in practice.
Implementing Microservice Governance
Key components for the realization of microservices governance
Let’s revisit the governance aspects that we discussed in the previous sections and see how and where they are implemented with respect to these components.
Service Registry and Discovery
If we consider service definitions, the definition of service identifiers, message models, interfaces, etc., are the things that you can do without centralized governance. However, these service definitions must be published into a centralized service registry. The service registry is a centralized component, which will also define a canonical model to describe a service. All the service owners should publish their service definitions in that canonical form to the service registry. Even the services are implemented with drastically different technologies (e.g. OpenAPI vs gRPC), you can still find a common metadata for such services and add them to the service registry.
The service registry also comes with a service discovery protocol (or API), which can be used to retrieve service information during runtime. (We will discuss service registry and discovery in detail in the next section.)
Development Lifecycle Management
Service lifecycle management is often implemented at the service deployment level. For example, when a given service has to be deployed across multiple environments, the deployment process will address those requirements of replicating or moving the same service code across those environments. This also includes various DevOps-related deployment methodologies (blue-green, canary, AB testing, etc.) and describes how to manage different environments such as development (dev), test, quality assurance (QA), staging, production (prod), and so on.
API Management/API Gateway
API management plays a key role in the realization of several microservices governance aspects. As discussed in Chapter 1, “The Case for Microservices,” the API management layer or API gateway is used to expose your microservices to your consumers as managed APIs. This includes all the quality of service aspects that we discussed in the previous sections as well as several other API management specific details, such as monetization. As part of API management, we can apply security, service versioning, throttling, caching, monetization, etc. for services during the runtime. It’s important to understand that most of these capabilities have to be applied centrally for service invocations. Therefore, API gateways are centrally governed or managed and application of those capabilities can be either centralized or decentralized. Also, API gateways can be used for external or internal consumers. These capabilities can be equally applicable when microservices talk to each other via an internal API gateway.
API management solutions often work hand-in-hand with service registries to discover services as well as to use them as the API repository (i.e., the information related to APIs can be also published and discovered via a service registry). This is also quite useful when we use existing services and create new APIs out of them.
One other important aspect is that API management solutions provide a rich capability to discover and consume APIs. So, it is possible to leverage API management to manage all your microservices (rather than selecting the service that you want to expose externally).
We’ll defer the detailed discussion on these topics to Chapter 10, “APIs, Events, and Streams”.
Observability
Observability is something that is generically applicable to all your microservices. Each microservice can push data to any of those observability tools using the recommended observability agents. Observability tools offer a centralized view of all the interactions of your microservices and they operate as passive entities which would not intervene the original flow of business messages. The API management solution or API gateway also work hand-in-hand with observability tools. Most of the observability aspects that we discussed in the previous sections are often applicable to APIs as well.
There are specialized observability tools for each observability aspect, such as metrics, tracing, logging, service visualization, alerting, etc. We’ll delve deep into microservices observability in Chapter 13.
Service Registry and Discovery
When you run hundreds or thousands of microservices, it is quite important to have a central place to get the details of services. That’s where service registry and discovery come into the picture. In many resources that describe services registry and discovery in the context of microservices, it is explained as a mechanism to obtain the location of the microservice during execution. However, the service registry has a much broader meaning and can be used much more effectively.
Because you have so many services to deal with, it’s very important to have a central place where you can obtain all service information. As we discussed in previous sections, service registry is that place where we store all the service definitions (not just the service URLs, as some microservices articles explain). Consumers can get all the service definitions by accessing the service registry. Then the service discovery defines a way to access the service registry. Service owners should register the service at the service registry so that the consumers can discover them. Also, the owners are responsible for updating and maintaining the information of services. The most commonly practiced use cases for service registry and discovery in a microservices architecture is to have addressable names for services that make them independent of the infrastructure that they are running on. For example, when we are doing a service call, we use a name with a logical reference to the service and the service discovery resolves that name to the actual endpoint address of the service. Therefore, external services or consumers do not need to change their code when there’s a change in the actual endpoint address.
In general, we can consider service registry to be the repository to publish and retrieve canonical service definitions of all services in your microservices architecture. The mechanism, which we use to retrieve service definitions, is known as service discovery.
Client-side service discovery
Server-side service discovery
Client-side discovery is commonly used when the client is fully aware of the existence of the service registry and client code contains the service discovery logic. Server-side discovery is often used with container management systems such as Kubernetes and Docker Swarm, where service discovery is transparent to the client side.
Service registry implementations are not dependent on any of the service discovery mechanisms. In fact, in both cases, either the client or an intermediate application (load balancer) uses the same service registry APIs.
The service registry is often used as the centralized component for service discovery and therefore it can be the single point of failure. Therefore, It’s important to ensure high availability of the service registry as part of the deployment.
In the next couple of sections, we’ll take a closer look at the commonly used service registry solutions.
Consul
Consul2 is a distributed, highly available system that is designed for discovering and configuring services. It offers most of the service registry capabilities, so that services can publish service definitions, and clients can use Consul to discover a given service. Using either DNS or HTTP, applications can find the services they depend upon. In addition to the service registry and discovery capabilities, Consul also supports service health checking, a key-value store that can be used for dynamic configuration, feature flagging, coordination, leader election, and more.
Consul architecture
Every node that provides services to Consul should run a Consul agent.
An agent is the long-running daemon on every member of the Consul cluster. It is started by running the Consul agent. The agent can run in client or server mode. However, an agent is not required for a client that discovers services.
The agent is responsible for running health checks and keeping services in sync. The agents talk to one or more Consul servers.
A Consul client is an agent that forwards all RPCs to a server. The client is relatively stateless and the only background activity a client performs is taking part in the LAN gossip pool. (The gossip protocol involves random node-to-node communication over UDP.)
A Consul server is where data is stored and replicated. The servers elect a leader. Servers are responsible for maintaining the cluster state, responding to RPC queries, exchanging WAN gossip with other datacenters, and forwarding queries to leaders or remote datacenters.
Consul defines a canonical service definition, which is used for registering and discovering services.
A service can be registered either by providing a service definition as a configuration (inside the consul.d directory) or by making the appropriate calls to the HTTP API.
Similarly, services can be discovered using the service discovery REST API.
Let’s suppose you have installed3 Consul. Let’s go through the steps involved in service registration and discovery. (You can try ch06/sample01, which is available in the samples repository.)
Registering a Service
Alternatively, you can register the service via the catalog REST API4.
Discovering a Service
Consul REST API provides a convenient way to retrieve the service definitions.
All these operations are also exposed via DNS interfaces. There is a wide range of other capabilities offered from Consul, which are not directly related to service registry, and which are not in the scope of this book. But just keep in mind that if you need some sort of key-value pair-based repository with coordination and high-availability support, Consul is a very good solution to incorporate into your microservice architecture.
Eureka
Eureka architecture
Eureka server is a web application that you can deploy into Tomcat. You can then connect to it via a Eureka client or the REST API. A Eureka client is a Java client that can be used to register heartbeats. As shown in Figure 6-5, you can embed the Eureka client as part of service code or client code. Application Services can use the Eureka client to register services, and application client can use it for discovering services. Services register with Eureka and then send heartbeats to renew their leases every 30 seconds. If the client cannot renew the lease, it is taken out of the server registry after about 90 seconds. The registration information and the renewals are replicated across all the Eureka nodes in the cluster.
There is one Eureka cluster per region (geographic locations such as us-east, us-west, etc.), which knows only about the instances in its region. There is at the least one Eureka server per zone (one region can have multiple zones, which can be considered isolated datacenters) to handle zone failures. The clients from any zone can look up the registry information (which happens every 30 seconds) to locate their services (which could be in any zone) and make remote calls.
Using Eureka with Spring Boot
Spring Boot offers native support for using Eureka as a service registry. Let’s take a close look at how you can use Eureka in your Spring Boot applications and use service registry and discovery capabilities in practice.
With some annotations you can quickly enable and configure the service registry and discovery patterns inside your application with Eureka. Eureka instances can be registered, and clients can discover the instances using Spring-managed beans. An embedded Eureka server can be created with a declarative Java configuration.
This application will boot up a Eureka service registry instance and you can change various behaviors via the application.properties file.
This service retrieves the application name from the Eureka service registry and returns as part of the response.
Etcd
Etcd7 is a general-purpose, distributed key-value store designed to reliably and quickly, preserve and provide access to critical data. It enables reliable distributed coordination through distributed locking, leader elections, and write-barriers. An etcd cluster is intended for high availability and permanent data storage and retrieval. Therefore, etcd is also being used as a service registry implementation. However, it offers a wide range of capabilities, which are beyond the service registration and discovery. etcd offers a CLI tool called etdcctl and a gRPC API to interact with it.
etcd v3 uses gRPC for its messaging protocol. The etcd project includes a gRPC-based Go client and a command-line utility, etcdctl, for communicating with an etcd cluster through gRPC. For languages with no gRPC support, etcd provides a JSON grpc-gateway. This gateway serves a RESTful proxy that translates HTTP/JSON requests into gRPC messages.
etcd is being widely used as part of most of the existing registry and deployment orchestrator solutions, such as Kubernetes.
Service Discovery with Kubernetes
In the Kubernetes environment, when you call one service from another, you don’t need to worry about the actual location of your service. Kubernetes by default uses DNS names to discover the pods. Therefore, if you want to call the bar service from the foo service, in the foo service’s code, you can just refer to http://bar:<port> as the service endpoint. Kubernetes will resolve and map the name to the actual endpoint. Kubernetes internally uses etcd as its distributed key-value store.
It’s important to keep in mind that, although Kubernetes offers out-of-the-box capabilities to seamless discovery of a service, it is not meant to be used as the repository and interface for service developers or consumers to interact with. That’s where you may want to manage such service definitions in an external service registry.
We discuss Kubernetes in detail in Chapter 8, “Deploying and Running Microservices,” and will dive into a real-world example that uses service discovery inside Kubernetes.
Summary
In this chapter, we discussed microservices governance in a broad perspective. Rather than just abstracting it as a decentralized process, we took a detailed look at various aspects of microservices governance, such as service definitions, lifecycle management, registry and discovery, quality of service, and observability. While service design, development, and deployment can be done as a fully decentralized process, there are several concepts that you need to centrally apply to your microservices governance. In that context, we introduced key aspects of the implementation of microservices governance—service registry and discovery, development lifecycle management, API management, and observability. We set the foundation for most of these concepts in the context of microservice governance and we deferred a detailed discussion to the upcoming chapters that are dedicated to each of those topics.
We had a detailed discussion of service registry and discovery aspects of microservices governance in this chapter. We discussed the importance of service registry and discovery, and covered a couple of commonly used patterns. Then we discussed some of the most popular service registry solutions, such as Consul and Eureka, with real-world examples.